# An Improved Adam's Algorithm for Stomach Image Classification
## 2 Design of the CG-Adam Algorithm
### The CG-Adam Algorithm  
In the use of optimization algorithms, various parameters need to be considered, and the learning rate is crucial. The learning rate needs to be determined by a combination of factors and is usually set at different values during different training phases. The Adam algorithm employs an adaptive adjustment of the learning rate technique to ensure that the learning rate does not decay too quickly. In order to achieve this goal, Adam performs an exponentially weighted average of the gradient and its square, which is essentially an accumulation calculation of the historical gradient and can effectively attenuate the influence of the historical gradient on the current gradient. In the initial stage of iteration, the Adam algorithm is prone to rapid convergence due to the rapid accumulation of the first-order momentum of the gradient, resulting in significant oscillations around the optimum and exhibiting instability. When the algorithm falls into a local optimum, it is also difficult to jump out of the local optimum. In this paper, a new adaptive learning rate algorithm is proposed for the above problem; CG-Adam is an improved version of the Adam optimization algorithm that combines the control restart strategy and Gradient norm joint clipping. The specific steps of this algorithm are shown in Algorithm 1. For a clearer description of Algorithm 1, the algorithm flowchart (Scheme 1) is shown below. ```
1:Input: initial point \(\theta_{t}\), first moment decay \(\beta_{1}\), second moment decay \(\beta_{2}\), Gradient clipping threshold clip_value, Restart period T.
2:Initialize: set the initial value of momentum \(m_{0}\) and \(v_{0}\) to 0. Initialize \(t^{\prime}\) to 0 for cycle counting. Initialize restart_step to 0 to control the restart strategy. Set the initial step of the optimizer to 0. i.e., \(m_{0}=0,v_{0}=0\), \(t^{\prime}=0\), restart_step = 0, step = 0. 3: For \(t=1\) to \(T\)do
4:\(g_{t}=\
abla_{\theta}f_{t}(\theta_{t})\)
5:\(m_{t}=\beta_{1}m_{t-1}+(1-\beta_{1})g_{t}\)
6:\(v_{t}=\beta_{2}v_{t-1}+(1-\beta_{2})g_{t}^{2}\)
7: if \(t^{\prime}=0\) then
8: restart_step = 0
9: end for
10: if restart_step == 0 then
11:\(\hat{m}_{t^{\prime}}^{\wedge}=m_{t}/\left(1-\beta_{1}^{t^{\prime}}\right)\)
12:\(\hat{v}_{t^{\prime}}^{\wedge}=v_{t}/\left(1-\beta_{2}^{t^{\prime}}\right)\)
13: for each \(\theta\) parameter in group[ params ] do
14:\(m_{t^{\prime}}=\beta_{1}m_{t^{\prime}-1}+(1-\beta_{1})g_{t}\)
15:\(v_{t^{\prime}}=\beta_{2}v_{t-1}+(1-\beta_{2})g_{t}^{2}\)
16:\(g_{t}^{\prime}=g_{t}/\text{max}(1,[\beta_{t}]_{2}/clip\_value)\)
17:\(denom=\sqrt{\hat{v}_{t}+\varepsilon}\)
18:\(step\_size=lr/\left(1-\beta_{1}^{t^{\prime}}\right)\)
19:\(\theta_{new}=\theta-\frac{\hat{m}_{t^{\prime}}}{denom}step\_size\)
20: end for
21: update \(t^{\prime}\) to the value of the next cycle, \(t^{\prime}=t\text{mod}T\)
22: restart_step = (restart_step + 1) mod T end for Return: Returns the final optimized parameter \(\theta_{t}\). ```  
**Algorithm 1** CG-Adam  
```
1:Input: initial point \(\theta_{t}\), first moment decay \(\beta_{1}\), second moment decay \(\beta_{2}\), Gradient clipping threshold clip_value, Restart period T.
2:Initialize: set the initial value of momentum \(m_{0}\) and \(v_{0}\) to 0. Initialize \(t^{\prime}\) to 0 for cycle counting. Initialize restart_step to 0 to control the restart strategy. Set the initial step of the optimizer to 0. i.e., \(m_{0}=0,v_{0}=0\), \(t^{\prime}=0\



## 3 Experimental Design and Analysis of Results

### Experimental Environment Configuration

This experiment is based on the deep learning framework of Pytorch. The CG-Adam algorithm is jointly improved by the control restart strategy and Gradient norm joint clipping. The specific versions of the software mainly used in the experiment are shown in Table 1.

In order to test the effectiveness of the CG-Adam algorithm, image classification experiments were conducted on three commonly used datasets: the MNIST dataset, the CIFAR10 dataset, and the Stomach dataset.