# Linear Optics Calibration in a Storage Ring Based on Machine Learning
## 2 Materials and Methods
### Experimental Methods on Machine Learning  
#### 2.2.1 Building the CNN Model  
Our machine learning algorithm is programmed using Python. Keras is used as a deep learning module, and it is a high-level API based on TensorFlow [33, 34]. TensorFlow, which is based on dataflow programming, is widely used in programming various machine learning algorithms. After selecting the machine learning module, we started to build the neural network model. We choose the root mean square error as the loss function (RMSE) and the mean absolute error (MAE) as the metric function. The metrics function is optional. It is not directly involved in model optimization but can be used to describe model performance. Using MAE as the metrics function can well characterize the deviation between the predicted value and the label. The model is trained using the error backpropagation method with the Amsgard optimizer. Amsgard is a variant of the Adam optimizer [35] proposed by [PERSON], [PERSON], and [PERSON]. By analyzing the convergence proof of the Adam optimizer, they find that there are errors in the updated rules that can cause the algorithm to converge to a suboptimal point. They designed theoretical experiments to show Adam's failure scenarios and proposed their Amsgard method as a solution [36]. Our network is used to obtain the corresponding quadrupole errors after the input response matrix, so the input data become the response matrix and the output data become the quadrupole errors. In SSRF, the response matrix has 276 rows and 160 columns, including four small 138 \(\times\) 80 matrices: XX, XY, YX, and YY. XY and YX are coupling terms  
Figure 3: Schematic illustration of the machine-learning-based method for a linear optics calibration. and are not valid in the quadrupole calibration. Therefore, we remove XY and YX from the response matrix, leaving the XX and YY matrices, and transform them into a new matrix with 276 rows and 80 columns. This becomes the input data we used. For the output data, there are 206 quadrupoles in SSRF, and they are classified into 45 families. Therefore, the output data are the errors of 45 families of quadrupoles, which form a vector whose elements correspond to the error of each set of quadrupoles. The topological structure of neural networks is particularly important. The deeper neural networks with more hidden layers and the wider neural networks with more kernels in each layer can generally fit the training data better. However, a large model is prone to overfitting and requires more computational resources with a longer training time. We test different neural network structures, considering both accuracy and convergence speed. Finally, we choose a 10-layer CNN with 8 kernels, and the size of the kernel is set to [3; 3]. As we need a vector in the output layer, the data must go through a flattening layer after the convolutional layers. The final output layer is a dense layer, meaning that all neurons in the previous layer are connected to all neurons in the next layer. We use the rectified linear unit function as the activation function to simulate the process of neurons responding according to signal strength [37]. The rectified linear unit function is defined as follows:  
\[f(x)=max(0,x), \tag{4}\]  
The rectified linear unit function has the characteristics of unilateral suppression and a relatively wide excitation boundary. Rectifying neurons can produce sparse representations with true zeros and have shown equal or better performance than hyperbolic tangent networks [38]. In our model, its learning speed is faster than other activation functions, and the effect is also better. The next step is to initialize the weights of the neural networks. The purpose of weight initialization is to prevent gradient explosion or gradient disappearance in the forward propagation process of deep neural networks. Gradient explosion or gradient disappearance leads to a very large or very small derivative of the neural networks in backpropagation training, which makes the weight very large or unable to be updated.