# Theory of AdaDelSPGD Algorithm in Fiber Laser-Phased Array Multiplex Communication Systems
## 2 Theoretical Analysis and Simulation of the Improved SPGD Algorithm
### AdaDelSPGD Algorithm and Simulation  
The above algorithm is a conventional SPGD algorithm, and, based on the deep learning algorithm; this paper studies in detail the adaptive optimization of the SPGD iterative step size. After the loss function is defined in deep learning, various methods are adopted to reduce the loss function. However, the value of the loss function is only a reference to optimize the parameters in the model. We optimized the loss function indirectly to optimize the model parameters and improve the measurement index of the model. Assuming that the objective function to be optimized is \(J(\theta)\):  
\[J(\theta)=E_{(x,y)\hat{p}dara}L(f(x,\theta),y) \tag{2}\]  
Figure 2: The relationship of system performance index with iteration times under different gain coefficients. (**a**) three-channel; (**b**) seven-channel. where \(L\) is the loss function of every sample, \((f(x,\theta),y)\) is the input, \(x\) is the predicted output, \(y\) is the objective function, and \(\hat{p}\)_data_ is the empirical distribution on the training set. However, theoretically, we hope that \(\hat{p}\)_data_ is the real data distribution. In machine learning, empirical distribution is used to replace real distribution. Since not all sample data can be collected, only training sets are used for training. However, since the distribution of the data is not known, this machine learning problem is transformed into an optimization problem by finding the expectation and minimizing the expected loss. The purpose of optimization is to minimize the empirical risk:  
\[E_{(x,y)\hat{p}\hat{data}}L(f(x,\theta),y)=\frac{1}{m}\sum_{i=1}^{m}L\left( \left(f\left(x^{i}\right),\theta\right),y^{\theta}\right) \tag{3}\]  
where \(m\) is the number of training samples. The training process based on this minimization of the mean training error is called experiential risk minimization. In the training of deep learning, the number of samples in the training set is often tens of thousands, and the amount of computation required to traverse each sample to calculate the expectation is huge. In practice, a small amount of data is collected in the training set for calculation, and then the average value of these samples is calculated. Stochastic gradient descent (SGD) is a classic optimization algorithm among small batch algorithms, which is often used in machine learning, especially in deep learning. Firstly, a small batch is generated by randomly selecting m samples from the training set, and then their gradient mean is calculated. One disadvantage of the SGD method is that the update direction is completely dependent on the gradient calculated by the current subset, so it is very unstable. The momentum SGD algorithm borrows the concept of momentum in physics, which simulates the inertia of object motion; that is, the direction of the previous update is retained to some extent when updating, while the gradient of the current subset is used to fine-tune the final update direction. In this way, one can increase the stability to a certain extent, so that one can learn faster, and there is a certain ability to eliminate local optimality. The previous series of optimization algorithms have a common feature; that is, each parameter is updated with the same learning rate. However, in practical application, the importance of each parameter is certainly different, so it is necessary to dynamically adopt different learning rates for different parameters, so that the objective function can converge faster. The Adagrad algorithm is used to take the square of the gradient of each iteration of each parameter, add it and then square it, and divide the basic learning rate by this number, in order to perform the dynamic update of the learning rate. In this way, the learning rate of each parameter is related to its gradient, so the learning rate of each parameter is different, which is the so-called adaptive learning rate  
The AdaDelta algorithm is an extension of the Adagrad algorithm.