# Fault Diagnosis Algorithm of Beam Pumping Unit Based on Transfer Learning and DenseNet Model
## 3 Dynamometer Classification Model Based on the Transfer Learning Method
### Transfer Learning Model Based on DenseNet  
#### 3.2.1 DenseNet Model Structure  
Machine learning is a method to analyze data by generating models based on known data sets. The analysis of data features is essential in the process of building recognition models. When the feature information is insufficient, the recognition model cannot reach the ideal state. Too much feature information will lead to a high cost of calculation of the recognition model, which is not convenient for practical use. Therefore, whether the features extracted from the target image can be effectively utilized is one of the key factors affecting the classification effect of the model. The DenseNet model's densely connected approach transmits signals from each layer to the bottom layer to maximize the use of effective features. As shown in Figure 5, compared with the traditional \(L\)-layer convolutional network with \(L\) connections, DenseNet has \(\frac{L(L+1)}{2}\) connections in the \(L\)-layer convolutional network [17].  
The network input at layer \(L\) is expressed as Equation (4):  
\[x_{l}=H_{l}([x_{0},x_{1},\ldots,x_{l-1}]) \tag{4}\]  
where:  
\(x_{i},\;\;i=0,1,2,\ldots l-1\)--Input parameters for each layer;  
\(H_{l}\)--is a compound function composed of Batch Normalization (BN), activation function (ReLU), pooling function, and convolution layer (Conv).  
In the dense connection mode, each layer in the DenseNet model can receive all previous layer information, that is, feature propagation is strengthened, parameter reuse is realized, the number of parameters is reduced, and the calculation process is simplified.  
This paper establishes the basic model of the DenseNet structure and retains the convolutional layer as the feature extraction structure after pretraining on ImageNet. The structure of each layer of the convolutional layer maintains the BN-ReLU-Conv in order to avoid gradient vanishing. Taking into account the characteristics of the indicator diagram image, the pooling layer structure adopts the average pooling function to maximize the retention of the overall data characteristics, such as in Equation (5):  
\[L_{1}(A)=\sum_{x=1}^{f}\sum_{y=1}^{f}A_{k}^{l}(s_{0}i+x,s_{0}j+y) \tag{5}\]  
To avoid overfitting, add the dropout function Equation (6) to the model:  
Figure 5: Dense connection mode of DenseNet model.  
\[\begin{split} r^{(l)}\ \sim Bernoulli(p),\\ \widetilde{y}^{(l)}&=r^{(l)}\times y^{(l)},\\ f(x)&=max(0,x),\\ y^{(l+1)}&=f\bigg{(}w^{(l+1)}\times\widetilde{y}^{(l )}+b^{l+1}\bigg{)}\end{split} \tag{6}\]  
where:  
_Bernoulli_ function--\(r\) generates a vector \(r\) with Bernoulli distribution;  
\(y^{(l)}\)--the output value of the \(l\)-th layer;  
\(w^{(l+1)}\)--the weighted value of the neural network of the \(l+1\) th layer;  
\(b^{l+1}\)--the offset value;  
\(p\)--determined by the number of neurons in the fully connected layer.  
In the transfer process, this function can remove some weights in the layer, and the deleted weights will not disappear, but will also be reactivated by the use of the model. The main purpose of using this function is to simplify the calculation process and prevent overfitting caused by the overly complex model structure.  
The model uses the loss function to evaluate the gap between the predicted model and the actual results and uses the cross-entropy loss to represent the loss. The function is as follows in Equation (7):  
\[l=-\sum_{i=1}^{K}y_{i}\times\ln(p(x\ _{j})) \tag{7}\]  
where:  
\(l\)--the amount of loss;  
\(K\)--the number of classifications;  
\(y\)--the actual label, \(y=[y_{1},\dots,y_{i},\dots y_{k}]\);  
\(p(x)\)--the predicted label, \(p(x)=\big{[}p(x_{1}),\dots,p(x_{j}),\dots,p(x_{k})\big{]}\).  
Because cross-entropy can measure the difference between two probability distributions, this paper uses the cross-entropy function as the loss function to measure the difference between the true probability distribution and the predicted probability distribution. The smaller the cross-entropy value, the better the prediction effect of the obtained model.