چکیده مقاله
Neural network training can be rigorously analyzed as a discrete time dynamical system, allowing formal derivation of stability and convergence conditions without relying solely on empirical simulations In this paper, we model gradient descent GD and momentum based updates as iterative maps and identify equilibrium points corresponding to critical points of the loss function Using Lyapunov stability theory, linearization, and spectral analysis of the Hessian, we derive sufficient conditions for local stability and linear convergence Extensions include overparameterized networks, stochastic gradient descent SGD , and multi layer architectures We also provide an analytical contribution for ReLU networks, showing that stability depends only on the spectral properties of active Hessian blocks in piecewise linear regions The results provide a mathematically sound framework for understanding training dynamics and predicting convergence behavior of neural networks Boyd & Vandenberghe, Y Khalil, Y Polyak, 1975 Bottou, Y
کلیدواژهها
نویسندگان
Parviz Asefi Nazarlou
Azarbaijan Shahid Madani University ASMU , Tabriz, Iran
شیوه ارجاع
Asefi Nazarlou, Parviz,1404,Stability and Convergence Analysis of Neural Networks Using Discrete Dynamical Systems Theory,The 8th international conference on artificial intelligence and its future prospects in electrical, computer, mechanical and telecommunication engineering sciences,Mashhad
ارائهشده در
مجموعه مقالات هشتمین کنفرانس بین المللی هوش مصنوعی و چشم انداز آینده آن در علوم مهندسی برق ، کامپیوتر ، مکانیک و مخابرات5 آذر 1404 · مشهد