# Adam Algorithm with Step Adaptation
## 4 Numerical Results and Discussion
### Test Functions
#### 4.2.2 Quadratic Functions  
To study the effect of problem conditioning, the following quadratic function was used:  
\[f_{Q}(x,[\alpha\text{max}])=\frac{1}{2}\sum_{i=1}^{n}a_{i}\ x_{i}^{2},\ \ a_{i}= \alpha\text{max}\frac{i-1}{n-1}. \tag{18}\] \begin{table}
\begin{tabular}{c c c} \hline \hline \(\beta_{2}\) & ADAM & H\_ADAM \\ \hline
0.3 & \(\frac{1901\times 10^{-2}}{1063\times 10^{-3}}\) & 1934 \\
0.5 & \(\frac{1684}{1379\times 10^{-4}}\) & 1533 \\
0.9 & \(\frac{6094\times 10^{-5}}{2253\times 10^{-5}}\) & 1502 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Number of iterations for the \(f_{Q}(x,\)\(\{a_{max}=10\})\) function minimization depending on the parameter \(\beta_{2}\). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \(r\) & ADAM & H\_GR & H\_MOMENTUM & H\_ADAM \\ \hline
0 & 12,315 & 11,252 & 2094 & 10,001 \\
1 & 13,721 & 11,269 & 7140 & 11,021 \\
2 & 116,779 & 31,477 & 18,938 & 18,921 \\
3 & 44,111 & 29,050 & 38,862 & 41,404 \\
4 & 260,210 & 147,051 & 1064 & 148,752 \\
5 & 934,950 & 168,840 & 112,545 & 112,284 \\
6 & \(\frac{1301\times 10^{-5}}{2872\times 10^{-4}}\) & 429,597 & 83,790 & 80,820 \\
7 & \(\frac{2872\times 10^{-4}}{2872\times 10^{-4}}\) & 571,974 & 152,464 & 182,693 \\
8 & 71,419 & \(\frac{3233\times 10^{-9}}{2872\times 10^{-5}}\) & 662,428 & 288,912 \\
9 & \(\frac{3868\times 10^{-5}}{2872\times 10^{-3}}\) & 761,875 & 350,220 & 256,479 \\
10 & \(\frac{1231\times 10^{-3}}{2872\times 10^{-3}}\) & 499,957 & 58,956 & 347,315 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Number of iterations for the Rosenbrock function minimization with gradient interference. Eigenvalues \(a_{i}\) of this function have the boundaries \(\lambda_{min}\) = 1 and \(\lambda_{max}\) = \(a_{max}\). Increasing the \(a_{max}\) value leads to an increase in problem conditioning. The initial point is \(x_{0}\) = (100, 100,..., 100), \(\varepsilon\) = 10\({}^{-10}\), \(n\_max\) = 3,000,000. Tables 6-8 show the results of minimization. Here, we can see that the efficiency of the algorithms increase in the following order: ADAM, H_GM, H_ADAM and H_MOMENTUM. A slight deterioration in conditionality results in an equalization of the convergence rate of the gradient method and the methods using a smoothed gradient. The metric transformation does not play a significant role here. Here, we can see that the efficiency of the algorithms increase in the following order: ADAM, H_GR, H_ADAM and H_MOMENTUM. The metric transformation does not play a significant role here. The H_GR step adaptation methods also cope with the task successfully. \begin{table}
\begin{tabular}{c c c c} \hline \hline \(r\) & **ADAM** & **H\_GR** & **H\_MOMENTUM** & **H\_ADAM** \\ \hline
0 & 108,850 & 1272 & 1729 \(\times\) 10\({}^{-3}\) & 3222 \\
1 & 253,080 & 1795 & 2231 & 2368 \\
2 & \(1133\times 10^{-3}\) & 1994 & 5124 & 5534 \\
3 & \(3678\times 10^{-3}\) & 2953 & 10,878 & 11,133 \\
4 & \(5230\times 10^{-3}\) & 4517 & 16,731 & 18,416 \\
5 & \(7113\times 10^{-3}\) & 6725 & 28,560 & 29,747 \\
6 & \(1226\times 10^{-3}\) & 9448 & 48,540 & 48,546 \\
7 & \(1448\times 10^{-3}\) & 12,240 & 71,915 & 70,744 \\
8 & \(2727\times 10^{-3}\) & 15,898 & 97,062 & 97,362 \\
9 & \(4985\times 10^{-3}\) & 19,896 & 120,143 & 119,395 \\
10 & \(8304\times 10^{-3}\) & 23,851 & 148,182 & 148,132 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Number of iterations for the function\(f_{Q}(x,\{a_{max}=100\})\) minimization with gradient interference.