Copyright and Licensing
Articles accepted for publication will be licensed under the Creative Commons BY-NC-SA. Authors must sign a non-exclusive distribution agreement after article acceptance.
Towards more efficient initialization methods for Convolutional Neural Networks via K-Means and Principal Components
DOI:
https://doi.org/10.24215/16666038.25.e04Keywords:
Clustering, CNN, Deep learning,, K-Means Clustering, Principal Component AnalysisAbstract
This paper presents an exploration of unsupervised methods for initializing and training filters in convolutional layers, aiming to reduce the dependency on labeled data and computational resources. We propose two unsupervised methods based on the distribution of input data and evaluate their performance against traditional Glorot Uniform initialization. By initializing solely the initial layer of a basic CNN network with one of our proposed methods, we attained a 0.78\% enhancement in final accuracy compared to traditional Glorot Uniform initialization. Our findings suggest that these unsupervised methods could serve as effective alternatives for filter initialization, potentially leading to more efficient training processes and a better understanding of CNNs.
Downloads
Download data is not yet available.
References
T.-H. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, and Y.
Ma, “PCANet: A simple deep learning baseline
for image classification,” IEEE Trans. Image Pro-
cess., vol. 24, no. 12, pp. 5017–5032, Dec. 2015.
doi: 10.1109/TIP.2015.2475625.
X.-D. Ren, H.-N. Guo, G.-C. He, X. Xu, C. Di,
and S.-H. Li, “Convolutional neural network based
on principal component analysis initialization for
image classification,” in Proc. IEEE First Int. Conf.
Data Science Cyberspace (DSC), 2016, pp. 329–
doi: 10.1109/DSC.2016.18.
F. C. Soon, H. Y. Khaw, J. H. Chuah, and J. Kane-
san, “Semisupervised PCA convolutional network
for vehicle type classification,” IEEE Trans. Veh.
Technol., vol. 69, no. 8, pp. 8267–8277, Aug. 2020.
doi: 10.1109/TVT.2020.3000306.
C. Zhang, M. Mei, M. Zhuolin, J. Zhang, A. Deng,
et al., “PLDANet: Reasonable combination of
PCA and LDA convolutional networks,” Int. J.
Comput. Commun. Control, vol. 17, no. 2, Apr.
doi: 10.15837/ijccc.2022.2.4541.
L. Sun, Y. Liu, S. Chen, B. Luo, Y. Li, and C.
Liu, “Pig detection algorithm based on sliding
windows and PCA convolution,” IEEE Access,
vol. 7, pp. 44229–44238, 2019. doi: 10.1109/AC-
CESS.2019.2907748.
L. Tian, C. Fan, Y. Ming, and Y. Jin,
“Stacked PCA network (SPCANet): An effec-
tive deep learning for face recognition,” in
Proc. Int. Conf. DSP, 2015, pp. 1039–1043. doi:
1109/ICDSP.2015.7252036.
H. Y. Khaw, F. C. Soon, J. H. Chuah, and C. O.
Chow, “Image noise types recognition using con-
volutional neural network with principal compo-
nents analysis,” IET Image Process., vol. 11, no.
, pp. 1238–1245, Dec. 2017. doi: 10.1049/iet-
ipr.2017.0374.
A. Coates and A. Y. Ng, “Learning feature repre-
sentations with K-means,” in Neural Networks:
Tricks of the Trade, G. Montavon, G. B. Orr,
and K.-R. Muller, Eds., 2nd ed. Berlin, Germany:
Springer, LNCS 7700, 2012, pp. 561–580.
Downloads
Published
2025-04-30
Issue
Section
Original Articles
License
Copyright (c) 2025 Federico Rabinovich, Facundo Quiroga, Franco Ronchetti
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
How to Cite
[1]
“Towards more efficient initialization methods for Convolutional Neural Networks via K-Means and Principal Components”, JCS&T, vol. 25, no. 1, p. e04, Apr. 2025, doi: 10.24215/16666038.25.e04.
