## 6 Results  
In this part of our study, we will characterize the convergence rate of the three algorithms, and compare the convergence/divergence behavior in relation to the parameters of simulation, and, finally, we will examine the quality of the resulting reconstructions of the three optimizers. First of all, we choose in the context of this present analysis, the definition of divergence to be the state of the running optimization when the error minimization didn't improve for longer than 200 iterations in total. The first subject of focus is the convergence rate of each algorithm. Let \(X_{AD}\), \(X_{NAD}\), and \(X_{AMS}\) be three random variables representing the state of convergence for Adam, Nadam, and AmsGrad optimizers. These variables take values 0 or 1, depending on either the corresponding algorithm diverges or converges, such that:  
\[ \begin{cases}X_{AD}=0\text{ if Adam diverge}\\ X_{AD}=1\text{ if Adam converge}\end{cases} \]  
\[ \begin{cases}X_{NAD}=0\text{ if Nadam diverge}\\ X_{NAD}=1\text{ if Nadam converge}\end{cases} \]  
\[ \begin{cases}X_{AMS}=0\text{ if AmsGrad diverge}\\ X_{AMS}=1\text{ if AmsGrad converge}\end{cases} \]  
The simulation provided us with three samples of independent and identically distributed \((X_{AD,n})_{n\leq 1340\cdot}\)\((X_{NAD,n})_{n\leq 1340\cdot}\), and \((X_{AMS,n})_{n\leq 1340\cdot}\). To statistically estimate the rates of convergence, namely \(P(X_{AD}=1)=p_{AD}\), \(P(X_{NAD}=1)=p_{NAD}\), and \(P(X_{AMS}=1)=p_{AMS}\). We use the three estimators  
\[ \begin{cases}\hat{p}_{AD}=\frac{\sum\#(X_{AD}=1)}{N}\\ \hat{p}_{NAD}=\frac{\sum\#(X_{AD}=1)}{N}\\ \hat{p}_{AMS}=\frac{\sum\#(X_{AMS}=1)}{N}\end{cases} \]  
where # denotes the count function, in order to construct 95% confidence intervals based on the large number normal approximation, as presented in Table 2.  
To shed light on the influence of simulation parameters on convergence rates, we run a logistic regression to estimate the conditional distributions \(P(X|\beta,n,\delta)\), where \(X\in\{X_{AD},X_{NAD},X_{AMS}\}\), n denotes the number of anomalies in the image, and the remaining variables \(\beta\) and \(\delta\) are as mentioned earlier. The result of this procedure is depicted in Table 3 presenting \(p\)-values for the statistical significance of each regression parameter. From Table 3, we conclude that the main parameter that also has a significant influence on convergence of these algorithms is the learning rate hyper-parameter. Since the logistic coefficient for\(\beta\) is positive for Adam and Nadam, the larger the learning rate, the more guarantee there is for the algorithm to converge. This statement is reversed for AmsGrad, as we observe a negative coefficient for learning rate. The AmsGrad is also impacted negatively with the number of anomalies in the image. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Target Proportion** & **Confidence Interval Lower Bound** & **Confidence Interval Upper Bound** \\ \hline \(\hat{p}_{AD}\) & 0.87 & 0.9 \\ \hline \(\hat{p}_{NAD}\) & 0.87 & 0.91 \\ \hline \(\hat{p}_{AMS}\) & 0.88 & 0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The 95% non-parametric confidence intervals. Running our experiment simulation provided us with 1340 convergent instances in total (this means where \((X_{AD}=1,X_{NAD}=1,X_{AMS}=1)\)), to evaluate the comparative performance between optimizers in terms of the speed of convergence as measured by the number of iterations taken by each optimizer to reach the solution, We will conduct a statistical analysis on the generated data, comparing first the speed globally between optimizers and then relating it to the variables of simulation such as the initial guess error, the choice of learning rate and the number of anomalies in the image.