title: ÐÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ðµ ÑÐ°Ð±Ð¾ÑÐ¾ÑÐ¿Ð¾ÑÐ¾Ð±Ð½Ð¾ÑÑÐ¸ Ð°Ð´Ð°Ð¿ÑÐ¸Ð²Ð½Ð¾Ð³Ð¾ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð¼ÐµÑÐ¾Ð´Ð° AMSGrad Ð² ÑÑÐ»Ð¾Ð²Ð¸ÑÑ Ð¶ÐµÑÑÐºÐ¾ÑÑÐ¸ ÑÐµÐ»ÐµÐ²ÑÑ ÑÑÐ½ÐºÑÐ¸Ð¾Ð½Ð°Ð»Ð¾Ð²
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Performance study of the adaptive gradient method AMSGrad in conditions of target functionals rigidity
Creators: name: ÐÑÐºÐ¾Ð²Ð°, ÐÑÐµÐ½Ð¸Ñ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2021
Subjects: AMSGrad
Ð¼ÐµÑÐ¾Ð´Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸
Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸
ÑÑ Ð¾Ð´Ð¸Ð¼Ð¾ÑÑÑ
Ð¶ÐµÑÑÐºÐ¾ÑÑÑ
Ð¿Ð»Ð¾Ñ Ð°Ñ Ð¾Ð±ÑÑÐ»Ð¾Ð²Ð»ÐµÐ½Ð½Ð¾ÑÑÑ
optimization methods
optimization algorithms
convergence
rigidity
ill-conditioned problems

description: ÐÐ°Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¸ Ð¾ÑÐµÐ½ÐºÐµ Ð¾Ð´Ð½Ð¾Ð¹ Ð¸Ð· ÑÐ°ÑÐ°ÐºÑÐµÑÐ¸ÑÑÐ¸Ðº Ð°Ð´Ð°Ð¿ÑÐ¸Ð²Ð½Ð¾Ð³Ð¾ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð¼ÐµÑÐ¾Ð´Ð° AMSGrad â ÐµÐ³Ð¾ ÑÐ°Ð±Ð¾ÑÐ¾ÑÐ¿Ð¾ÑÐ¾Ð±Ð½Ð¾ÑÑÐ¸ Ð¿ÑÐ¸ Ð¿Ð¾Ð¸ÑÐºÐµ Ð¾Ð¿ÑÐ¸Ð¼ÑÐ¼Ð° Ð¶ÐµÑÑÐºÐ¸Ñ ÑÐµÐ»ÐµÐ²ÑÑ ÑÑÐ½ÐºÑÐ¸Ð¾Ð½Ð°Ð»Ð¾Ð². Ð ÑÐ°Ð±Ð¾ÑÐµ Ð¿ÑÐ¸Ð²ÐµÐ´ÐµÐ½ Ð¾Ð±Ð·Ð¾Ñ ÑÑÐ¾Ð³Ð¾ Ð¼ÐµÑÐ¾Ð´Ð°, Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ ÐµÐ³Ð¾ ÑÐµÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ Ð² ÑÑÐ»Ð¾Ð²Ð¸ÑÑ Ð¶ÐµÑÑÐºÐ¾ÑÑÐ¸, Ð° ÑÐ°ÐºÐ¶Ðµ ÑÐ´ÐµÐ»Ð°Ð½ Ð²ÑÐ²Ð¾Ð´ Ð¾ Ð²Ð¾Ð·Ð¼Ð¾Ð¶Ð½Ð¾ÑÑÑÑ ÐµÐ³Ð¾ Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸Ñ Ð´Ð»Ñ ÑÐµÑÐµÐ½Ð¸Ñ Ð¶ÐµÑÑÐºÐ¸Ñ Ð·Ð°Ð´Ð°Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸. Ð ÑÐ¾Ð´Ðµ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ»Ð¸ ÑÐµÑÐµÐ½Ñ ÑÐ»ÐµÐ´ÑÑÑÐ¸Ðµ Ð·Ð°Ð´Ð°ÑÐ¸: 1. ÐÐ·ÑÑÐµÐ½Ð¸Ðµ Ð¾ÑÐ¾Ð±ÐµÐ½Ð½Ð¾ÑÑÐµÐ¹ Ð°Ð´Ð°Ð¿ÑÐ¸Ð²Ð½Ð¾Ð³Ð¾ Ð³ÑÐ°Ð´Ð¸ÐµÐ½ÑÐ½Ð¾Ð³Ð¾ Ð¼ÐµÑÐ¾Ð´Ð° AMSGrad. 2. Ð ÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð¼ÐµÑÐ¾Ð´Ð° AMSGrad Ð½Ð° ÑÐ·ÑÐºÐµ Python. 3. Ð¢ÐµÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ Ð¼ÐµÑÐ¾Ð´Ð° AMSGrad Ð½Ð° Ð½Ð°Ð±Ð¾ÑÐµ ÑÐµÑÑÐ¾Ð²ÑÑ ÑÑÐ½ÐºÑÐ¸Ð¹ (ÑÑÐ½ÐºÑÐ¸Ð¸ Ð Ð¾Ð·ÐµÐ½Ð±ÑÐ¾ÐºÐ° Ñ ÑÐ°Ð·Ð»Ð¸ÑÐ½ÑÐ¼Ð¸ ÐºÐ¾ÑÑÑÐ¸ÑÐ¸ÐµÐ½ÑÐ°Ð¼Ð¸ Ð¸ ÑÑÐ½ÐºÑÐ¸Ð¸ ÐÐ°ÑÑÐ»Ð»Ð°). 4. ÐÑÐµÐ½ÐºÐ° Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½Ð¾Ð¹ ÑÐ¾ÑÐ½Ð¾ÑÑÐ¸ Ñ ÑÐ¾ÑÐºÐ¸ Ð·ÑÐµÐ½Ð¸Ñ ÑÑÐ¾Ð´Ð¸Ð¼Ð¾ÑÑÐ¸ ÐºÐ°Ðº Ð¿Ð¾ Ð°ÑÐ³ÑÐ¼ÐµÐ½ÑÑ, ÑÐ°Ðº Ð¸ Ð¿Ð¾ ÑÑÐ½ÐºÑÐ¸Ð¾Ð½Ð°Ð»Ñ. ÐÑÐ»Ð¾ ÑÑÑÐ°Ð½Ð¾Ð²Ð»ÐµÐ½Ð¾, ÑÑÐ¾ Ð¿ÑÐ¸ Ð½ÐµÐºÐ¾ÑÐ¾ÑÐ¾Ð¼ ÑÑÐ¾Ð²Ð½Ðµ Ð¶ÐµÑÑÐºÐ¾ÑÑÐ¸ AMSGrad ÑÐ¿Ð¾ÑÐ¾Ð±ÐµÐ½ ÑÐµÑÐ°ÑÑ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¾Ð½Ð½ÑÐµ Ð·Ð°Ð´Ð°ÑÐ¸, Ð¾Ð´Ð½Ð°ÐºÐ¾ Ð¾Ð½ Ð¾ÐºÐ°Ð·Ð°Ð»ÑÑ Ð½ÐµÑÐ°Ð±Ð¾ÑÐ¾ÑÐ¿Ð¾ÑÐ¾Ð±ÐµÐ½ Ð² ÑÑÐ»Ð¾Ð²Ð¸ÑÑ Ð²ÑÑÐ¾ÐºÐ¾Ð³Ð¾ ÑÑÐ¾Ð²Ð½Ñ Ð¶ÐµÑÑÐºÐ¾ÑÑÐ¸ ÑÐµÐ»ÐµÐ²ÑÑ ÑÑÐ½ÐºÑÐ¸Ð¾Ð½Ð°Ð»Ð¾Ð². Ð ÐµÐ·ÑÐ»ÑÑÐ°ÑÑ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¼Ð¾Ð³ÑÑ Ð¿ÑÐ¸Ð¼ÐµÐ½ÑÑÑÑÑ Ð¿ÑÐ¸ ÑÐµÑÐµÐ½Ð¸Ð¸ Ð¿ÑÐ°ÐºÑÐ¸ÑÐµÑÐºÐ¸Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¾Ð½Ð½ÑÑ Ð·Ð°Ð´Ð°Ñ Ð´Ð»Ñ Ð¿ÑÐ¸Ð½ÑÑÐ¸Ñ ÐºÐ¾ÑÑÐµÐºÑÐ½Ð¾Ð³Ð¾ ÑÐµÑÐµÐ½Ð¸Ñ Ð¾ Ð²Ð¾Ð·Ð¼Ð¾Ð¶Ð½Ð¾ÑÑÐ¸ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¼ÐµÑÐ¾Ð´Ð° Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ AMSGrad.
description: This work is devoted to studying and evaluating one of the adaptive gradient method AMSGrad characteristics â its performance in the search for the optimum of rigid target functionals. This work provides an overview of this method, the method was tested under rigidity conditions, and also the work contains a conclusion about possibilities of the method application for solving rigid optimization problems. During the study, the following problems were solved: 1. Study of the adaptive gradient method AMSGrad features. 2. Implementation of the AMSGrad method in Python. 3. Testing the AMSGrad method on a set of test functions (Rosenbrock function with different coefficients and Powell function). 4. Estimation of the obtained accuracy from the point of view of convergence both in argument and in functional. It was found that at a certain level of rigidity AMSGrad is able to solve optimization problems, but it turned out to be inoperable in conditions of the target functionals high rigidity level. The research results can be used in solving practical optimization problems to make the correct decision about the possibility of using the AMSGrad optimization method.