# Fault Diagnosis Algorithm of Beam Pumping Unit Based on Transfer Learning and DenseNet Model
## 3 Dynamometer Classification Model Based on the Transfer Learning Method
### Transfer Learning Model Based on DenseNet
#### 3.2.2 Model Training Mechanism  
To improve the learning effect, the hyperparameters in the transfer learning model based on the DenseNet model (hereinafter referred to as the dense connection model) are tested and adjusted. To train the dense connection model, we set the initial learning rate to be 0.1, 0.01, and 0.001 for testing, and ultimately obtain the best effect when the initial learning rate is 0.01. At this rate, the training speed of the dense connection model is better and the fluctuation is small.  
One epoch means that a data set passes a neural network, and the learning time of the test run is 100 epochs. According to the image size, the structural complexity of the dense connection model, and the conditions of computing equipment, the size of a single data set is set to 32, that is, when there are 32 images in a single data set, the dense connection model has a better convergence speed.  
Since the learning rate has a significant impact on the performance of the densely connected model, an adaptive learning rate optimization algorithm is adopted considering the possibility of other changes, except the initial learning rate setting and the learning rate changing with the number of trainings. The Adam optimization algorithm is used in the training of the dense connection model. The algorithm has the advantages of high computational efficiency and low memory usage, and is suitable for large-scale data when applied to nonconvex optimization problems. It is expressed as Equation (8):\[\begin{array}{c}m_{t}=\beta_{1}m_{t-1}+(1-\beta_{1})g_{t}\\ v_{t}=\beta_{2}v_{t-1}+(1-\beta_{2})g_{t}^{2}\\ \hat{m}_{t}=\frac{m_{t}}{1-\beta_{t}^{2}}\\ \hat{\sigma}_{t}=\frac{v_{t}}{1-\beta_{2}^{2}}\\ W_{t+1}=W_{t}-\frac{\eta}{\sqrt{\eta}+e}\hat{m}_{t}\end{array} \tag{8}\]  
where:  
\(m_{t}\) and \(v_{t}\)--the first-order momentum and second-order momentum;  
\(\beta_{1}\) and \(\beta_{2}\)--usually take 0.9 and 0.999 for the power value;  
\(W_{t}\)--the parameters of the model in the \(t\)-th iteration;  
\(g_{t}\)--the gradient value of the loss function with respect to the parameter \(W\) in the \(t\)-th iteration;  
\(e\)--The minimal constant (usually takes its value to the \(10^{-8}\) to avoid a zero denominator);  
\(\eta\)--the initial learning rate.  
Before training, the ImageNet dataset is used to load the structure to obtain a pre-trained model, and then the training set for training and fine-tuning is loaded. The specific process is to freeze the saved basic model parameters and train the classifier. After the training results have stabilized, the frozen parameters are activated and fine-tuned with a lower learning rate.  
For the recognition model based on the DenseNet model using the global pooling classifier, this paper uses two types of classifiers, the fully connected layer and the global pooling layer, respectively. Due to the limited number of training samples and the characteristics of the indicator diagrams, it is not necessary to use color adjustment or image inversion for data augmentation to increase training samples  
After testing in the fully connected layer, the dropout layer parameter is 0.2, the L2 regularization is 0.0001, and the effect is better. At this time, the model can effectively avoid overfitting caused by too many parameters and ensure the training speed.