# An Improved Adam's Algorithm for Stomach Image Classification
## 2 Design of the CG-Adam Algorithm  
### The Adam Optimization Algorithm  
The Adam algorithm is an adaptive learning rate optimizer that combines the advantages of the gradient descent and momentum strategies. Its core mechanism is based on the computation of first-order and second-order moment estimates of the gradient, which are used to dynamically adjust the learning rate. The first-order moments represent the mean of the gradient, and the second-order moments represent the mean of the squared gradient. As a frequently used optimization tool in neural network training, the Adam algorithm is notable for its ability to adaptively adjust the learning rate, allowing for personalized learning rate adjustments for each parameter to better fit the unique needs of each parameter. This approach avoids the complexity of manually setting the learning rate and shows better adaptability for handling sparse gradients, real-time data streams, and non-static datasets. Although Adam performs well in most cases, it may fall into local optimal solutions on some complex lossy surfaces, thus limiting the effectiveness of the optimization. In the implementation of the Adam algorithm, the parameters of the first-order moments and second-order moments, which are responsible for balancing the contribution of first-order and second-order moments, respectively, further optimize the learning process. In the Adam algorithm, the parameters used for both are defined as first-order moments and second-order moments, respectively. The moment estimation vectors for the gradient are shown in Equations (1) and (2):  
\[m_{t}=\beta_{1}m_{t-1}+(1-\beta_{1})g_{t} \tag{1}\]  
\[v_{t}=\beta_{2}v_{t-1}+(1-\beta_{2})g_{t}^{2} \tag{2}\]  
In Equations (1) and (2), \(\beta_{1}\) and \(\beta_{2}\) represent the exponential decay rates for the first-order moment and second-order moment, respectively; \(g_{t}\) is the gradient at step \(t\) and is calculated as shown in Equation (3):  
\[g_{t}=\
abla_{\theta}f_{t}(\theta_{t}) \tag{3}\]  
In Equation (3), \(\theta_{t}\) is the parameter vector updated at step \(t\); \(f_{t}(\theta_{t})\) is the loss function of the \(t\)-th iteration in the neural network; and \(g_{t}\) is the gradient of the loss function of the \(t\)-th repetition of the neural network with respect to the parameters. When the decay rate is very small in the initial stage, the first-order moment and second-order moment estimatesmay deviate from zero, leading to some bias. In order to eliminate this bias, [PERSON] applies a bias correction to the first-order moments and second-order moments during the decay process, as shown in Equations (4) and (5):  
\[\overset{\wedge}{m}=m/\left(1-\beta_{1}^{t}\right) \tag{4}\]  
\[\overset{\wedge}{v}=v/\left(1-\beta_{2}^{t}\right) \tag{5}\]  
After each iteration, the parameter \(\theta\) will be updated, as shown in Equation (6):  
\[\theta_{t+1}=\theta_{t}-\frac{\alpha}{\sqrt{\overset{\wedge}{v}+\varepsilon} }\overset{\wedge}{m}_{t} \tag{6}\]  
In Equation (6), \(\theta\) represents the learning rate; \(\varepsilon\) is a small value greater than 0, primarily used to prevent division by zero in the denominator. Despite the good performance of the Adam optimization algorithm, it is sensitive to the choice of learning rate and requires a careful adjustment of hyperparameters. In some non-convex optimization problems, the hyperparameters may not converge to the global optimum.