# Adam Algorithm with Step Adaptation
## 4 Numerical Results and Discussion
### Test Functions
#### 4.2.3 Functions with Ellipsoidal Ravine  
To study the effect of problem dimension, the following function was used:  
\[f_{EEL}(x,[a\text{max},b\text{max}])=(1-x_{1})^{2}+a\text{max}\left(1-\sum_{i=1}^ {n}x_{i}^{2}/\ b_{i}\right)^{2},\ \ \ \ \ b_{i}=b\text{max}\frac{i-1}{n-1}, \tag{20}\]  
The initial point is \(x_{1}^{0}=-1\), \(x_{i}^{0}=i\), \(i=2\), \(3\), \(\ldots\)\(n\), \(\varepsilon=10^{-4}\), \(n\)_max_ = 3,100,000. As \(n\) increases, the dimension of the problem increases. Table 10 shows the results of minimization.  
Here, we can see that the efficiency of the algorithms increase in the following order: H_GR, ADAM, H_MOMENTUM and H_ADAM. In this problem, the minimum point is degenerate. Therefore, the gradient method moves into the extremum region and loops. The ADAM algorithm has a slow convergence, and with large noise, it tends to start looping. The H_ADAM algorithm copes with all tasks. This example shows the advantages of the H_ADAM algorithm compared to the H_MOMENTUM algorithm, obtained through the use of diagonal metric matrices.  
The following function has a multidimensional ellipsoidal ravine. Minimization occurs when moving along a curvilinear ravine to a minimum point.  
\[f_{EX}(x,[a\text{max},\ b\text{max}])=(1-x_{1})^{2}+a\text{max}\left(1-\sum_{i =1}^{n}x_{i}^{2}/\ b_{i}\right)^{2}+\frac{1}{2}\sum_{i=2}^{n}x_{i}^{2}/\ b_{i}, \ \ b_{i}=b\text{max}^{n-1}. \tag{21}\]  
The initial point is \(x_{1}^{0}=-1\), \(x_{i}^{0}=i\), \(i=2\), \(3\), \(\ldots n\), \(\varepsilon=10^{-10}\), \(n\)_max_ = 41,000,000. Due to the additional quadratic term in \(f_{EX}\), the minimum point ceases to be degenerate. This allows the gradient method to find the minimum of the function with higher accuracy compared to the function \(f_{EEL}\). Table 11 shows the results of minimization.  
\begin{table}
\begin{tabular}{c r r r} \hline \(\mathbf{r}\) & **ADAM** & **H\_GR** & **H\_MOMENTUM** & **H\_ADAM** \\ \hline
0 & 123,881 & 4704 \(\times\)\(10^{-4}\) & 3670 & 5592 \\
1 & 232,563 & 4480 \(\times\)\(10^{-4}\) & 445,481 & 41,511 \\
2 & 398,545 & 3654 \(\times\)\(10^{-4}\) & 560,187 & 77,574 \\
3 & 717,788 & 3654 \(\times\)\(10^{-4}\) & 994,309 & 205,299 \\
4 & 1,723,520 & 4125 \(\times\)\(10^{-4}\) & 1,110,249 & 320,391 \\
5 & 6009 \(\times\)\(10^{-4}\) & 5028 \(\times\)\(10^{-4}\) & 1,187,059 & 364,720 \\
6 & 6163 \(\times\)\(10^{-4}\) & 6156 \(\times\)\(10^{-4}\) & 1,687,797 & 720,857 \\
7 & 9079 \(\times\)\(10^{-4}\) & 7666 \(\times\)\(10^{-4}\) & 1,922,210 & 1,010,981 \\
8 & 9766 \(\times\)\(10^{-4}\) & 9447 \(\times\)\(10^{-4}\) & 2,830,218 & 1,877,945 \\
9 & 1941 \(\times\)\(10^{-3}\) & 1165 \(\times\)\(10^{-3}\) & 1809 \(\times\)\(10^{-4}\) & 2,257,204 \\
10 & 2129 \(\times\)\(10^{-3}\) & 1417 \(\times\)\(10^{-3}\) & 2446 \(\times\)\(10^{-4}\) & 3,023,663 \\ \hline \end{tabular}
\end{table}
Table 10: Number of iterations for the function \(f_{EEL}(x,[a_{max}=100,\ b_{max}=1000])\) minimization with gradient interference.  
Here, we can see that the efficiency of the algorithms increase in the following order: ADAM, H_GR, H_MOMENTUM and H_ADAM. In this non-convex problem, the minimum point is non-degenerate, but with a large spread of the eigenvalues of the matrix of second derivatives in the minimum region. Therefore, the gradient method has a slow convergence. The ADAM algorithm does not work in the presence of interference. The H_MOMENTUM algorithm works here. The H_ADAM algorithm copes with all problems. This example demonstrates the advantages of the H_ADAM algorithm compared to the H_MOMENTUM algorithm, obtained by using diagonal metric matrices, which are especially noticeable in conditions of small interference.