NEURAL NETWORKS TRAINING BASED ON RANDOM SEARCH
Article Sidebar
Main Article Content
Abstract
The paper deals with a state-of-art problem, associated with neural networks training. Training algorithm (with special parallelization procedure) implementing the annealing method is proposed. The training efficiency is demonstrated by the example of a neural network architecture focused on parallel data processing. For the color image compression problem, it is shown that the proposed algorithm significantly outperforms gradient methods in terms of efficiency. The results obtained make it possible to improve the neural networks training quality in general, and can be used to solve a wide class of applied problems.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Matskevich, V. V. (2022). Vozmozhnosti metoda otzhiga v zadache obucheniya neironnykh setei [Annealing method possibilities in the neural networks training problem]. In Tekhnologii peredachi i obrabotki informatsii [Information transmission and processing technologies] (69–73). Minsk: BSUIR (in Russ., abstr. in Engl.).
Hajek, B. (1988). Cooling Schedules for Optimal Annealing. Mathematics of Operations Research, 13(2), 311–329. http://www.jstor.org/stable/3689827.
Li, W., Han, M., & Wang, J. (2020). Recurrent restricted Boltzmann machine for chaotic time-series prediction. In 12 th International Conference on Advanced Computational Intelligence (ICACI) (439–445). DOI: 10.1109/ICACI49185.2020.9177510.
Sharma, Bh., Tomer, M., & Kriti, Kr. (2020). Extractive text summarization using F-RBM. Journal of Statistics and Management Systems, 23(6), 1093–1104. DOI: 10.1080/09720510.2020.1808353.
Zhou, D., Wang, X., Tian, Y., & Wang, R. (2017). A novel radar signal recognition method based on a deep restricted Boltzmann machine. Engineering Review, 37(2), (165–171).
Devi, Ch., Chen, R-Ch, Hendry, & Hung, H.-T. (2021). Experiment improvement of restricted Boltzmann machine methods for image classification. Vietnam Journal of Computer Science, 8(3), (417–432). DOI: 10.1142/S2196888821500184.
Zhai, J., Zhou, X., Zhang, S., & Wang. T. (2019). Ensemble RBM-based classifier using fuzzy integral for big data classification. International Journal of machine learning and cybernetics, (10), 3327–3337. DOI: 10.1007/s13042-019-00960-3.
Nakashika, T. (2018) LSTBM: a novel sequence representation of speech spectra using restricted Boltzmann machine with long short-term memory. In Proc. Interspeech 2018 (2529–2533). DOI: 10.21437/Interspeech.2018-1753.
Krasnoproshin, V. V., & Matskevich, V. V. (2020). Neural Network Data Processing Based on Deep Belief Networks. In Communications in Computer and Information Science. Vol. 1282: “Open Semantic Technologies for Intelligent System” (234–244). Springer. DOI: 10.1007/978-3-030-60447-9_15.
Kingma, D. P., & Ba, J. L. (2015). Adam: A Method for Stochastic Optimization. In Proc. of the 3rd International Conference on Learning Representations (1–15).
Hamis, S., Zaharia, T., & Rousseau, O. (2019). Image Compression at Very Low Bitrate Based on Deep Learned Super-Resolution. In IEEE 23rd International Symposium on Consumer Technologies (ISCT) (128–133). DOI: 10.1109/ISCE.2019.8901038.
Oswin, K., Fischer, A., & Igel, Ch. (2018). Population-Contrastive-Divergence: Does consistency help with RBM training? Pattern Recognition Letters, (102), 1–7. DOI: 10.48550/arXiv.1510.01624.
Li, X., Gao, X, & Wang, Ch. (2021). A Novel Restricted Boltzmann Machine Training Algorithm With Dynamic Tempering Chains. IEEE ACCESS, (9), 21939–21950. DOI: 10.1109/ACCESS.2020.3043599.
Brugge, K., Fischer, A., & Igel, Ch. (2013). The flip-the-state transition operator for restricted Boltzmann machines. Machine Learning, 93(1), 53–69. DOI: https://doi.org/10.1007/s10994-013-5390-3.