title: Ð¡ÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Adam
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Comparative analysis of the Adam algorithm
Creators: name: ÐÐ°ÑÐ»ÐµÐ½Ð½Ð¸ÐºÐ¾Ð²Ð°, ÐÐ»Ð¸Ð½Ð°
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2021
Subjects: Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸
adam
adagrad
rmsprop
adadelta
nadam
Ð½ÐµÐ¹ÑÐ¾Ð½Ð½Ð°Ñ ÑÐµÑÑ
Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
optimization algorithms
neural network
deep learning

description: ÐÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ Adam Ð¸ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÐµÐ³Ð¾ Ð¿ÑÐ¾Ð¸Ð·Ð²Ð¾Ð´Ð¸ÑÐµÐ»ÑÐ½Ð¾ÑÑÐ¸ Ñ Ð¿ÑÐ¾Ð¸Ð·Ð²Ð¾Ð´Ð¸ÑÐµÐ»ÑÐ½Ð¾ÑÑÑÑ Ð´ÑÑÐ³Ð¸Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð² Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸, ÑÐ°ÐºÐ¸Ñ ÐºÐ°Ðº: Nadam, AdaGrad, RMSProp, AdaDelta, Ð¸Ð¼Ð¿ÑÐ»ÑÑ ÐÐµÑÑÐµÑÐ¾Ð²Ð°. Ð¢ÑÐµÐ±ÑÐµÑÑÑ Ð²ÑÑÑÐ½Ð¸ÑÑ, ÑÐ²Ð»ÑÐµÑÑÑ Ð»Ð¸ Adam Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ ÑÑÑÐµÐºÑÐ¸Ð²Ð½ÑÐ¼ Ð´Ð»Ñ Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹. ÐÑÐ¾Ð²ÐµÐ´ÐµÐ½ ÑÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð², Ð¿ÑÑÐµÐ¼ Ð¸Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸. ÐÑÐ¾Ð²ÐµÐ´ÐµÐ½Ñ ÑÐµÑÑÑ Ð´Ð»Ñ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÑÐ°Ð·Ð»Ð¸ÑÐ½ÑÑ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸. ÐÐ°Ð½Ð½Ð°Ñ ÑÐµÐ¼Ð° ÑÐ²Ð»ÑÐµÑÑÑ Ð°ÐºÑÑÐ°Ð»ÑÐ½Ð¾Ð¹, ÑÐ°Ðº ÐºÐ°Ðº ÑÐµÐ¹ÑÐ°Ñ Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ Ð¸ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ Ð²Ð¾ÑÑÑÐµÐ±Ð¾Ð²Ð°Ð½Ñ. Ð¢Ð°ÐºÐ¶Ðµ, ÑÐ°Ðº ÐºÐ°Ðº Adam ÑÐ²Ð»ÑÐµÑÑÑ Ð¾Ð´Ð½Ð¸Ð¼ Ð¸Ð· ÑÐ°Ð¼ÑÑ Ð¿Ð¾Ð¿ÑÐ»ÑÑÐ½ÑÑ ÑÐ¿Ð¾ÑÐ¾Ð±Ð¾Ð² Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹ Ð²Ð°Ð¶Ð½Ð¾ Ð¿ÑÐ¾Ð²ÐµÑÑÐ¸ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ, Ð¿Ð¾ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÐµÐ³Ð¾ Ñ Ð´ÑÑÐ³Ð¸Ð¼Ð¸ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð°Ð¼Ð¸ Ð¸ Ð²ÑÑÐ²Ð»ÐµÐ½Ð¸Ñ Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾Ð³Ð¾ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð°. Ð ÑÐ°Ð±Ð¾ÑÐµ Ð±ÑÐ»Ð¸ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ñ Ð¸ ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð±Ð¸Ð±Ð»Ð¸Ð¾ÑÐµÐºÐ¸ Keras Ð½Ð° ÑÐ·ÑÐºÐµ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Python Ð² Ð¸Ð½ÑÐµÐ³ÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð½Ð¾Ð¹ ÑÑÐµÐ´Ðµ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ¸ PyCharm. ÐÑÐ»Ð¸ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ñ ÑÐµÑÑÑ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ ÑÐ·ÑÐºÐ¾Ð²Ð¾Ð¹ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¸ ÑÑÐ°Ð½Ð´Ð°ÑÑÐ½ÑÐ¹ ÑÐµÑÑ Ð¿Ð¾ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹. Ð¢Ð°ÐºÐ¶Ðµ Ð±ÑÐ» Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½Ñ Ð´Ð»Ñ Ð±Ð¾Ð»ÑÑÐµÐ¹ Ð½Ð°Ð³Ð»ÑÐ´Ð½Ð¾ÑÑÐ¸ ÑÐ°Ð±Ð¾ÑÑ Ð²ÑÐ±ÑÐ°Ð½Ð½ÑÑ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð²: ÑÐµÑÐµÐ½Ð° Ð·Ð°Ð´Ð°ÑÐ° Ð½Ð°ÑÐ¾Ð¶Ð´ÐµÐ½Ð¸Ñ Ð¼Ð¸Ð½Ð¸Ð¼ÑÐ¼Ð° ÑÑÐ½ÐºÑÐ¸Ð¸ Ð¾Ñ Ð´Ð²ÑÑ Ð¿ÐµÑÐµÐ¼ÐµÐ½Ð½ÑÑ. Ð¡Ð´ÐµÐ»Ð°Ð½Ñ Ð²ÑÐ²Ð¾Ð´Ñ Ð¸Ð· ÑÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½Ð¾Ð³Ð¾ Ð°Ð½Ð°Ð»Ð¸Ð·Ð° Ð¾Ð± ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ ÑÐ°ÑÑÐ¼Ð°ÑÑÐ¸Ð²Ð°ÐµÐ¼ÑÑ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð².
description: The final qualification work of the bachelor is devoted to the study of the Adam optimization algorithm and the comparison of its performance with the performance of other optimization algorithms, such as: Nadam, AdaGrad, RMSProp, AdaDelta, Nesterov's impulse. It is necessary to find out whether Adam is the most effective for deep learning of neural networks. A comparative analysis of the algorithms, by their implementation, is carried out. Tests were conducted to compare different optimization methods. This topic is relevant, since now the topic of machine learning and deep learning optimization algorithms is most in demand. And also, since Adam is one of the most popular ways to optimize neural networks, it is important to conduct research comparing it with other algorithms and identifying the most effective algorithm. In this paper, optimization algorithms were developed and implemented using the Keras library in the Python programming language in the integrated development environment PyCharm. Language model recognition tests and a standard image recognition test were performed. An experiment was also conducted to make the selected algorithms more clear: the problem of finding the minimum of a function of two variables was solved. Conclusions are drawn from the comparative analysis of the effectiveness of the algorithms under consideration.