АЛГОРИТМИЧЕСКАЯ УСТОЙЧИВОСТЬ  НЕЙРОННЫХ СЕТЕЙ ГЛУБОКОГО ОБУЧЕНИЯ  ПРИ РАСПОЗНАВАНИИ МИКРОСТРУКТУРЫ МАТЕРИАЛОВ

R. A. Klestov; A. V. Klyuev; V. Yu. Stolbov

doi:10.14529/ctcr210114

Authors

R. A. Klestov Perm National Research Polytechnic University, Perm, Russian Federation
A. V. Klyuev Perm National Research Polytechnic University, Perm, Russian Federation
V. Yu. Stolbov Perm National Research Polytechnic University, Perm, Russian Federation

DOI:

https://doi.org/10.14529/ctcr210114

Keywords:

deep neural networks, material microstructure, image recognition, deep learning, algorithmic stability

Abstract

The division of data for training a neural network into training and test data in various proportions to each other is investigated. The question is raised about how the quality of data distribution and their correct annotation can affect the final result of constructing a neural network model. The paper investigates the algorithmic stability of training a deep neural network in problems of recognition of the microstructure of materials. The study of the stability of the learning process makes it possible to estimate the performance of a neural network model on incomplete data distorted by up to 10%. Purpose. Research of the stability of the learning process of a neural network in the classification of microstructures of functional materials. Materials and methods. Artificial neural network is the main instrument on the basis of which produced the study. Different subtypes of deep convolutional networks are used such as VGG and ResNet. Neural networks are trained using an improved backpropagation method. The studied model is the frozen state of the neural network after a certain number of learning epochs. The amount of data excluded from the study was randomly distributed for each class in five different distributions. Results. Investigated neural network learning process. Results of experiments conducted computing training with gradual decrease in
the number of input data. Distortions of calculation results when changing data with a step of 2 percent are investigated. The percentage of deviation was revealed, equal to 10, at which the trained neural network model loses its stability. Conclusion. The results obtained mean that with an established quantitative or qualitative deviation in the training or test set, the results obtained by training the network can hardly be trusted. Although the results of this study are applicable to a particular case, i.e., microstructure recognition problems using ResNet-152, the authors propose a simpler technique for studying the stability of deep learning neural networks based on the analysis of a test, not a training set.

References

Recommendation System for Material Scientists Based on Deep Learn Neural Network / A. Kliuev, R. Klestov, M. Bartolomey, A. Rogozhnikov // Advances in Intelligent Systems and Computing. – 2019. – Vol. 850. – P. 216–223.

Evaluation of mechanical characteristics of hardened steels based on neural network analysis of digital images of microstructures / A.V. Klyuev, V.Yu. Stolbov, N.V. Kopceva, Yu.Yu. Jefimova // Cherny Metally. – 2020. – No. 6. – P. 50–56.

Klestov, R. About some approaches to problem of metals and alloys microstructures classification based on neural network technologies / R. Klestov, A. Klyuev, V. Stolbov // Advances in Engineering Research. – 2018. – Vol. 157. – P. 292–296.

Хайкин, С. Нейронные сети: полный курс: пер. с англ. / С. Хайкин. – 2-е изд. – М.: Издат. дом «Вильямс», 2006. – 1104 с.

Борисов, В.В. Искусственные нейронные сети. Теория и практика / В.В. Борисов, В.В. Круглов. – М.: Горячая линия – Телеком, 2003. – 382 с.

Hecht-Nielsen, R. Kolmogorov’s mapping neural network existence theorem / R. Hecht-Nielsen // IEEE First Annual Int. Conf. on Neural Networks. – San Diego, 1987. – Vol. 3. – P. 11–13.

Going deeper with convolutions / Christian Szegedy, Wei Liu, Yangqing Jia et al. // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). – 2015. – P. 1–9.

Deep Residual Learning for Image Recognition / Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). – 2016. – P. 770–778.

Squeeze-and-Excitation Networks / Jie Hu, Li Shen, Samuel Albanie et al. – 2017. – ILSVRC 2017 image classification winner. arXiv: 1709.01507.

Вапник, В.Н. Теория распознавания образов / В.Н. Вапник, А.Я. Червоненкис. – М.: Наука, 1974. – 416 с.

Bousquet, O. Algorithmic Stability and Generalization Performance / O. Bousquet, A. Elisseeff // Advances in Neural Information Processing Systems. – 2001. – No. 13. – P. 196–202.

Bousquet, O. Stability and Generalization / O. Bousquet, A. Elisseeff // Journal of Machine Learning Research. – 2002. – No. 2. – P. 499–526.

Brownlee, J. Impact of Dataset Size on Deep Learning Model Skill and Performance Estimates / J. Brownlee. – https://machinelearningmastery.com/impact-of-dataset-size-on-deep-learning-model-skill-and-performance-estimates (дата обращения: 04.02.2019).

How much data is needed to train a medical image deep learning system to achieve necessary high accuracy? / Junghwan Cho, Kyewook Lee, Ellie Shin et al. – 2015. – arXiv: 1511.06348, preprint arXiv.

Warden, P. How many images do you need to train a neural network? / Pete Warden. – https://petewarden.com/2017/12/14/how-many-images-do-you-need-to-train-a-neural-network (дата обращения: 04.11.2019).

Soekhoe, D. On the Impact of data set Size in Transfer Learning using Deep Neural Networks / Deepak Soekhoe, Peter van der Putten, Aske Plaat // IDA 2016: Advances in Intelligent Data Analysis XV. – 2016. – p. 50–60.