# Application of CNN-LSTM in Gradual Changing Fault Diagnosis of Rod Pumping System

Authors: Yanfeng He; Yali Liu; Shuai Shao; Xuhang Zhao; Guojun Liu; Xiangji Kong; Lu Liu
Journal: Mathematical Problems in Engineering
 (2019)

---

## Abstract

Owing to the importance of rod pumping system fault detection using an indicator diagram, indicator diagram identification has been a challenging task in the computer-vision field. The gradual changing fault is a special type of fault because it is not clearly indicated in the indicator diagram at the onset of its occurrence and can only be identified when an irreversible damage in the well has been caused. In this paper, we proposed a new method that combines the convolutional neural network (CNN) and long short-term memory (LSTM) network to perform a gradual changing fault classification. In particular, we employed CNN to extract the indicator diagram multilevel abstraction features based on its hierarchical structure. We considered the change in the time series of indicator diagrams as a sequence and employed LSTM to perform recognition. Compared with traditional mathematical model diagnosis methods, CNN-LSTM overcame the limitations of the traditional mathematical model theoretical analysis such as unclear assumption conditions and improved the diagnosis accuracy. Finally, 1.3 million sets of well production were set as a training dataset and used to evaluate CNN-LSTM. The results demonstrated the effectiveness of utilizing CNN and LSTM to recognize a gradual changing fault using the indicator diagram and characteristic parameters. The accuracy reached 98.4%, and the loss was less than 0.9%.


---

## Body

## 1. Introduction

Rod pumping systems are used in approximately 94% of artificially lifted wells. Owing to their extensive application, detailed understanding of the diagnosis and analysis of a rod-pump working state using the indicator diagram is important. Indicator diagram, which is a closed curve, reflects the variation pattern of a suspended point load in a pumping unit with its displacement, which performs vital functions for identifying the rod-pump production state. Ideally, the ground theoretical indicator diagram of wells is an approximate parallelogram, as shown in Figure1, whose vertical and horizontal coordinates represent the load and displacement, respectively. Section AB represents the loading section in the upward direction, and section BC represents the upward movement of the sucker rod. Section CD represents the unloading section in the downward direction, and section DA represents the downward movement of the rod pump.Figure 1
Ideal indicator diagram.Currently, three methods are available to analyze the indicator diagram, namely, statistical method [1], knowledge-based fault diagnosis method [2], and artificial neural network (ANN) method. The traditional indicator diagram recognition method suffers from serious insufficiency in that its recognition accuracy is poor and its identification efficiency is low. ANN is the most suitable method for identifying the indicator diagram owing to its self-studying and fault-tolerance qualities. In the field of computer-vision research, the most classic and widely used structure of ANN is the convolutional neural network (CNN), which is a deep feed-forward neural network [3]. With its typical features consisting of concatenating convolution, local connection, weight sharing, and pooling, CNN can effectively reduce the network complexity and the number of training parameters. Owing to its unique strong robustness and fault-tolerance ability, CNN can be easily trained and optimized. CNN suffers from extreme difficulty in identifying an image that changes over time because of 2D convolution kernel. In other words, the CNN channel output after filter calculation only contains the height and width dimensions but no time dimension.In identifying the indicator diagram using CNN, we find that CNN can be positively applied in abrupt changing fault identification in which the fault presents obvious mutation results in short time, such as sticking problem, rod parting, and other problems. However, the CNN performance in identifying gradual changing faults [4] is poor. The gradual changing fault is a persistent fault whose features are only obvious to a certain extent until they accumulate for some time, such as casing corrosion, tubing leak, and other faults. We need to note that a gradual changing fault is not clearly indicated in the indicator diagram at the beginning of its occurrence and is identified only when a serious damage is caused to the well at a later stage. In this process, the production capacity of an oil well has already been damaged, and an even irreversible damage might have been caused. To identify a gradual changing fault, we combine the long short-term memory (LSTM) network with CNN.LSTM is a type of recurrent neural network (RNN) with a more complex computational unit. LSTM is suitable for handling a variable-length input and easily learns highly nontrivial long-distance dependence because of the forget gate installed in it [5]. Therefore, the CNN-LSTM system can be applied to identify the gradual changing fault by learning the characteristic of the indicator diagram. The specific identification process requires CNN to extract multiple features from an indicator diagram and input them to LSTM.LSTM can capture sequential patterns by learning to store or ignore certain information from the sequence inputs [6]. When LSTM takes the CNN output as an input, all previous inputs in the sequence are considered. Therefore, if we treat an indicator diagram as feature sequences, we can intuitively use LSTM to perform fault classification. Furthermore, because CNN can produce features with different levels of abstraction from an image [7, 8] and because the features of the different levels of abstraction can provide complementary information, gradual changing fault identification can be realized.


## 2. Parameters

### 2.1. Definition of Gradual Changing Fault

The gradual changing fault in a rod pumping system mainly refers to the loss of circulation, casing corrosion, sand production, scale deposition, and pump barrel wear. In the traditional determination method, gradual changing fault is recognized based on experience without employing any systematic or scientific method. The empirical method depends on human judgment, which may cause low accuracy in determining the result and similar-fault-identification error. A determination method called principal component analysis (PCA) is proposed, which can more accurately and reasonably divide a gradual changing fault by measuring the indicator diagram similarity between the normal production and failure processes [9]. PCA is a multivariate statistical technique based on the similarity factor (SPCA) of the principal components, which can be used to calculate the principal directions of variability in the data [10].The process of determining the gradual changing fault using PCA is described as follows: We assume standard working indicator diagram datasetH and new working indicator diagram dataset S that contain the same n variables. PCA analysis is carried out on the S and H datasets, and we assume that the datasets can be constructed using the K principal components contained in the respective PCA models. The corresponding (n × k) subspaces are denoted by L and M, respectively. The PCA similarity factor (SPCA) compares these reduced subspaces [11].S
PCA is defined as(1)SPCAλ=traceRTTTTR∑i=1kλilλim,where(2)R=LΛl,T=MΛm,Λl=λ1l0000λ2l0000⋱0000λil,Λm=λ1m0000λ2m0000⋱0000λim.The more similar subspacesL and M are corresponding to datasets H and M, the larger is the value of SPCAλ. The PCA similarity factor (SPCA) is used to distinguish the fault type. According to SPCAλ of 15 typical faults calculated by the SPSS software, the threshold value is set to 0.5 in this study. In other words, when SPCA between working indicator diagram dataset S and standard working condition indicator diagram dataset H is greater than 0.5, the new working condition is defined as a gradual changing fault; otherwise, it is defined as an abrupt changing fault.According to the PCA method, common failures of the rod pumping system are classified into gradual or abrupt changing faults, as listed in Table1.Table 1
Rod pumping system failure classification.
Type
Failure
SPCA
Gradual changing fault
Lost circulation
>0.5
Casing corrosion
>0.5
Sand production
>0.5
Scale deposition
>0.5
Pump barrel wear
>0.5
Insufficient for liquid
>0.5
Gas impact
>0.5
Abrupt changing fault
Sudden touch pump
<0.5
Burst travel valve closed
<0.5
Rod resistance increases
<0.5
Break off
<0.5
Serious leakage of valve
<0.5
Oscillation production
<0.5
Foreign body inside the pump
<0.5
Changes in stroke
<0.5After the classification, the type of gradual changing fault is determined, and the characteristic parameters are selected to describe the fault-changing process.


### 2.2. Characteristic Parameters

Because large variations exist in a single indicator diagram, dealing with a single image in gradual changing fault identification is difficult. If multiple views of successive indicator diagrams can be effectively fused, we can retain information that is useful for the gradual changing fault while searching for variations in images.The suspended point load of the rod pumping system is the key to ensure normal operation of the rod pump [12]. As an example of scale deposition, the change in the suspended point load during a fault is studied using the method of multiple continuous indicator diagrams, as shown in Figure 2.Figure 2
Indicator diagram dataset of scale deposition. (a) View. (b) Side view.X-axis represents time, the Y-axis represents load, and the Z-axis represents displacement.
(a)
(b)Figure2 shows that section AB represents the slow scale deposition occurrence with time, point C denotes the period of fault, and the load sharply increases. Following the occurrence of scale deposition, the load and indicator diagram area gradually increase because the occurrence of scaling creates an additional resistance to the whole stroke or a certain area in the piston. In the upward stroke, the additional resistance increases the rod-pump suspended point load. In the downward stroke, the additional resistance reduces the load, and a vibrational load appears. In summary, the load increases during the upward stroke and decreases during the downward stroke. The phenomenon of scale deposition reflected in the indicator diagram indicates that the load lines in the upward and downward strokes show obvious variation rules.Hence, load is an indicator of the scale deposition. The development pattern of the gradual changing fault can be identified through the load. The load data are collected from the oilfield and inputted into CNN-LSTM. The load variation in the occurrence process of gradual changing fault identified by CNN-LSTM is shown in Figure3. The figure shows that the rod pumping system suspended point load exhibits an obvious low periodic change, which reflects the change process of the fault within one cycle. Thus, identifying the gradual changing fault based on the load is feasible.Figure 3
Load variation identified by CNN-LSTM.


## 2.1. Definition of Gradual Changing Fault

The gradual changing fault in a rod pumping system mainly refers to the loss of circulation, casing corrosion, sand production, scale deposition, and pump barrel wear. In the traditional determination method, gradual changing fault is recognized based on experience without employing any systematic or scientific method. The empirical method depends on human judgment, which may cause low accuracy in determining the result and similar-fault-identification error. A determination method called principal component analysis (PCA) is proposed, which can more accurately and reasonably divide a gradual changing fault by measuring the indicator diagram similarity between the normal production and failure processes [9]. PCA is a multivariate statistical technique based on the similarity factor (SPCA) of the principal components, which can be used to calculate the principal directions of variability in the data [10].The process of determining the gradual changing fault using PCA is described as follows: We assume standard working indicator diagram datasetH and new working indicator diagram dataset S that contain the same n variables. PCA analysis is carried out on the S and H datasets, and we assume that the datasets can be constructed using the K principal components contained in the respective PCA models. The corresponding (n × k) subspaces are denoted by L and M, respectively. The PCA similarity factor (SPCA) compares these reduced subspaces [11].S
PCA is defined as(1)SPCAλ=traceRTTTTR∑i=1kλilλim,where(2)R=LΛl,T=MΛm,Λl=λ1l0000λ2l0000⋱0000λil,Λm=λ1m0000λ2m0000⋱0000λim.The more similar subspacesL and M are corresponding to datasets H and M, the larger is the value of SPCAλ. The PCA similarity factor (SPCA) is used to distinguish the fault type. According to SPCAλ of 15 typical faults calculated by the SPSS software, the threshold value is set to 0.5 in this study. In other words, when SPCA between working indicator diagram dataset S and standard working condition indicator diagram dataset H is greater than 0.5, the new working condition is defined as a gradual changing fault; otherwise, it is defined as an abrupt changing fault.According to the PCA method, common failures of the rod pumping system are classified into gradual or abrupt changing faults, as listed in Table1.Table 1
Rod pumping system failure classification.
Type
Failure
SPCA
Gradual changing fault
Lost circulation
>0.5
Casing corrosion
>0.5
Sand production
>0.5
Scale deposition
>0.5
Pump barrel wear
>0.5
Insufficient for liquid
>0.5
Gas impact
>0.5
Abrupt changing fault
Sudden touch pump
<0.5
Burst travel valve closed
<0.5
Rod resistance increases
<0.5
Break off
<0.5
Serious leakage of valve
<0.5
Oscillation production
<0.5
Foreign body inside the pump
<0.5
Changes in stroke
<0.5After the classification, the type of gradual changing fault is determined, and the characteristic parameters are selected to describe the fault-changing process.


## 2.2. Characteristic Parameters

Because large variations exist in a single indicator diagram, dealing with a single image in gradual changing fault identification is difficult. If multiple views of successive indicator diagrams can be effectively fused, we can retain information that is useful for the gradual changing fault while searching for variations in images.The suspended point load of the rod pumping system is the key to ensure normal operation of the rod pump [12]. As an example of scale deposition, the change in the suspended point load during a fault is studied using the method of multiple continuous indicator diagrams, as shown in Figure 2.Figure 2
Indicator diagram dataset of scale deposition. (a) View. (b) Side view.X-axis represents time, the Y-axis represents load, and the Z-axis represents displacement.
(a)
(b)Figure2 shows that section AB represents the slow scale deposition occurrence with time, point C denotes the period of fault, and the load sharply increases. Following the occurrence of scale deposition, the load and indicator diagram area gradually increase because the occurrence of scaling creates an additional resistance to the whole stroke or a certain area in the piston. In the upward stroke, the additional resistance increases the rod-pump suspended point load. In the downward stroke, the additional resistance reduces the load, and a vibrational load appears. In summary, the load increases during the upward stroke and decreases during the downward stroke. The phenomenon of scale deposition reflected in the indicator diagram indicates that the load lines in the upward and downward strokes show obvious variation rules.Hence, load is an indicator of the scale deposition. The development pattern of the gradual changing fault can be identified through the load. The load data are collected from the oilfield and inputted into CNN-LSTM. The load variation in the occurrence process of gradual changing fault identified by CNN-LSTM is shown in Figure3. The figure shows that the rod pumping system suspended point load exhibits an obvious low periodic change, which reflects the change process of the fault within one cycle. Thus, identifying the gradual changing fault based on the load is feasible.Figure 3
Load variation identified by CNN-LSTM.


## 3. Approach

### 3.1. Framework Overview

In this paper, we propose a scheme to combine CNN and LSTM to perform gradual changing fault identification. The overview of the framework of the proposed method is shown in Figure4.Figure 4
Framework overview of the proposed method.Figure4 shows that the pipeline of the proposed method mainly consists of three components: indicator diagram classification, CNN-LSTM training, and system application. In the indicator diagram classification stage, from a given dataset, we prepare a set of multiple continuous indicator diagrams for the complete period. Subsequently, in the CNN-LSTM training stage, the image views are fed to CNN for feature extraction, and multiple features from the indicator diagram are extracted using CNN and inputted to LSTM. To acquire the continuous features of the abstraction, we concatenate the output layer of CNN and the input layer of LSTM in each image view. Consequently, feature sequences are obtained using CNN, and each feature sequence is processed using the corresponding LSTM. In the system application stage, trained CNN-LSTM is used for gradual changing fault identification, and the CNN-LSTM outputs are classified using the corresponding softmax classifiers, which are combined to determine the category of the input indicator diagram [13].


### 3.2. Dataset Sources

The dataset is collected from 160 wells in M oilfield, which is characterized by complex lithology, deep reservoir burial, and poor physical properties. The pipeline control system with a large-scale data management collects the production data every half hour and save them as a CSV file for each month. The data represent the production records of oil wells collected from January 1, 2017 to November 30, 2017. The CSV files are used to generate indicator diagrams. The selected data are processed using MATLAB software, which include the threshold segmentation, imdilate function expansion operations, imerode function corrosion operations, thin function refinement operations, and shrink function shrinkage operations. The CNN-LSTM training dataset is constructed using the processed indicator diagram, and the loads are selected as feature parameters.


### 3.3. CNN-LSTM Model

Inspired by the application of neural networks to image classification [14] and speech recognition [15], we combine CNN with the Tree-LSTM network, which can not only effectively extract the local features in an indicator diagram but also prevent the gradient disappearance phenomenon to occur in the time axis.


### 3.4. CNN Structure

CNN contains three types of layers: input, convolution and pooling, and fully connected layers. In the input layer, information received consists of multiple image sequences {A1, A2, …, An}, which are composed of the indicator diagram dataset. In the dataset-collection process, the different collection points are synchronized using the difference-fitting method. The feature parameters are obtained by CNN, which mainly include the suspended point load of the rod pumping system. By analyzing the similarity factor (SPCA) of the indicator diagram between the unknown and different typical working conditions, we can identify the types of unknown working conditions and realize working-condition diagnosis.In the convolution and pooling layer, we usen convolution filters in each dataset to extract n feature vectors from the indicator diagram and obtain n feature maps, which are also known as n channels. All the feature vectors are stacked in regional matrix XAnmap as follows:(3)XAndataset=ReLUdataset,An.The structure of the CNN hidden layer is composed of several convolutional and pooling layers. Rectified linear unit (ReLu) is a nonlinear activation function that runs on neurons. The convolution layer contains filterWC ϵ Rd×h, where h is the step size of the filter and d is the dimension of the n feature vectors. The feature vector generated by the filter is expressed as follows [16]:(4)C=fconvXAndataset∗WC+b,where b is a bigoted vector, that is, the intercept of the function that is used to complete an accurate linear classification, as shown in Figure 5.Figure 5
“Bigoted vector” performing linear classification.The pooling layer subsamples the output of the convolutional layer. The most common method to perform pooling is to apply a max operation to the result of each filter and obtain the output value as follows:(5)XAnmap=maxC.Two reasons justify the use of the max-pooling operation. First, by eliminating the nonmaximal values, it reduces the computation of the upper layers. Second, it provides a form of translation invariance. The obtained feature vectors are then fed to the sequential layer. To capture the long-distance dependence, LSTM is introduced into the sequential layer for vector composition [3].


### 3.5. LSTM Structure

LSTM [17] adds a gate to the architecture. Through the combined action of the input, output, and forget gates on the input data, it can solve the problem of vanishing and exploding gradients that exist in RNN [18]. However, ordinary LSTM suffers from many obvious limitations, e.g., it only solves linear problems, it is over dependent on the training sequence, and it is not predictive [19]. Thus, the tree-structure is proposed. Figure 6 shows the LSTM structure.Figure 6
Added tree-structure to LSTM (X is the input feature data from CNN and Y is the predicted output of LSTM).For each parent node, two child nodes exist, namely,hn and cn, which represent the node hidden state and memory unit, respectively. The transfer equation of node n is expressed as follows:(6)cnfnlfnr=tanhσσTa,bXjhnlhnr,where Xj stands for the input vector, superscripts l and r represent the left and right child nodes, respectively, σ is the sigmoid function, and Ta,b denotes the linear change, where a and b are the network parameters to be learned.Then, we connect the output end of LSTM to the softmax classifier and obtain the classification result of the working-condition recognition [20]. Figure 7 shows the entire proposed CNN-LSTM model.Figure 7
CNN-LSTM model.


## 3.1. Framework Overview

In this paper, we propose a scheme to combine CNN and LSTM to perform gradual changing fault identification. The overview of the framework of the proposed method is shown in Figure4.Figure 4
Framework overview of the proposed method.Figure4 shows that the pipeline of the proposed method mainly consists of three components: indicator diagram classification, CNN-LSTM training, and system application. In the indicator diagram classification stage, from a given dataset, we prepare a set of multiple continuous indicator diagrams for the complete period. Subsequently, in the CNN-LSTM training stage, the image views are fed to CNN for feature extraction, and multiple features from the indicator diagram are extracted using CNN and inputted to LSTM. To acquire the continuous features of the abstraction, we concatenate the output layer of CNN and the input layer of LSTM in each image view. Consequently, feature sequences are obtained using CNN, and each feature sequence is processed using the corresponding LSTM. In the system application stage, trained CNN-LSTM is used for gradual changing fault identification, and the CNN-LSTM outputs are classified using the corresponding softmax classifiers, which are combined to determine the category of the input indicator diagram [13].


## 3.2. Dataset Sources

The dataset is collected from 160 wells in M oilfield, which is characterized by complex lithology, deep reservoir burial, and poor physical properties. The pipeline control system with a large-scale data management collects the production data every half hour and save them as a CSV file for each month. The data represent the production records of oil wells collected from January 1, 2017 to November 30, 2017. The CSV files are used to generate indicator diagrams. The selected data are processed using MATLAB software, which include the threshold segmentation, imdilate function expansion operations, imerode function corrosion operations, thin function refinement operations, and shrink function shrinkage operations. The CNN-LSTM training dataset is constructed using the processed indicator diagram, and the loads are selected as feature parameters.


## 3.3. CNN-LSTM Model

Inspired by the application of neural networks to image classification [14] and speech recognition [15], we combine CNN with the Tree-LSTM network, which can not only effectively extract the local features in an indicator diagram but also prevent the gradient disappearance phenomenon to occur in the time axis.


## 3.4. CNN Structure

CNN contains three types of layers: input, convolution and pooling, and fully connected layers. In the input layer, information received consists of multiple image sequences {A1, A2, …, An}, which are composed of the indicator diagram dataset. In the dataset-collection process, the different collection points are synchronized using the difference-fitting method. The feature parameters are obtained by CNN, which mainly include the suspended point load of the rod pumping system. By analyzing the similarity factor (SPCA) of the indicator diagram between the unknown and different typical working conditions, we can identify the types of unknown working conditions and realize working-condition diagnosis.In the convolution and pooling layer, we usen convolution filters in each dataset to extract n feature vectors from the indicator diagram and obtain n feature maps, which are also known as n channels. All the feature vectors are stacked in regional matrix XAnmap as follows:(3)XAndataset=ReLUdataset,An.The structure of the CNN hidden layer is composed of several convolutional and pooling layers. Rectified linear unit (ReLu) is a nonlinear activation function that runs on neurons. The convolution layer contains filterWC ϵ Rd×h, where h is the step size of the filter and d is the dimension of the n feature vectors. The feature vector generated by the filter is expressed as follows [16]:(4)C=fconvXAndataset∗WC+b,where b is a bigoted vector, that is, the intercept of the function that is used to complete an accurate linear classification, as shown in Figure 5.Figure 5
“Bigoted vector” performing linear classification.The pooling layer subsamples the output of the convolutional layer. The most common method to perform pooling is to apply a max operation to the result of each filter and obtain the output value as follows:(5)XAnmap=maxC.Two reasons justify the use of the max-pooling operation. First, by eliminating the nonmaximal values, it reduces the computation of the upper layers. Second, it provides a form of translation invariance. The obtained feature vectors are then fed to the sequential layer. To capture the long-distance dependence, LSTM is introduced into the sequential layer for vector composition [3].


## 3.5. LSTM Structure

LSTM [17] adds a gate to the architecture. Through the combined action of the input, output, and forget gates on the input data, it can solve the problem of vanishing and exploding gradients that exist in RNN [18]. However, ordinary LSTM suffers from many obvious limitations, e.g., it only solves linear problems, it is over dependent on the training sequence, and it is not predictive [19]. Thus, the tree-structure is proposed. Figure 6 shows the LSTM structure.Figure 6
Added tree-structure to LSTM (X is the input feature data from CNN and Y is the predicted output of LSTM).For each parent node, two child nodes exist, namely,hn and cn, which represent the node hidden state and memory unit, respectively. The transfer equation of node n is expressed as follows:(6)cnfnlfnr=tanhσσTa,bXjhnlhnr,where Xj stands for the input vector, superscripts l and r represent the left and right child nodes, respectively, σ is the sigmoid function, and Ta,b denotes the linear change, where a and b are the network parameters to be learned.Then, we connect the output end of LSTM to the softmax classifier and obtain the classification result of the working-condition recognition [20]. Figure 7 shows the entire proposed CNN-LSTM model.Figure 7
CNN-LSTM model.


## 4. Experiments

In this section, we present our conducted orthogonal experiments to optimize the CNN-LSTM parameters. The CNN-LSTM structure was defined in Section3. Subsequently, the combination of the neural network parameters with different scales is tested. Finally, the optimal parameter setting mode is determined.


### 4.1. Setting

CNN-LSTM experiments are conducted on the training dataset consisting of 1.3 million sets of well-production test data. The parameter settings are listed in Table2.Table 2
Parameter setting.
No.
Convolution layers
Cove
Pooling layers
Sampling rate
Hidden layers
Dropout
Train time (h)
Accuracy (%)
1
2
16 : 32
2
2 × 2
1/136
0
183
92.1
2
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.25
276
92.5
3
2
16 : 32
2
2 × 2
1/136
0
159
92.6
4
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.25
264
92.9
5
2
16 : 32
2
2 × 2
1/136
0
76
93.8
6
2
16 : 32
2
2 × 2
2/2048 : 512
0.25
83
94.2
7
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.25
135
96.1
8
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.5
168
95.7The neural network recognition accuracy of the seventh architecture is shown to be as high as 96.1%. The selected CNN convolution kernel parameters are 16, 32, 64, and 128.


### 4.2. Evaluation of CNN-LSTM

For the datasets, 80% of the indicator diagrams are used for training and 20% are used for testing. The proposed method is evaluated based on the categorization accuracy, which is defined as follows:(7)accuracy=number of correctly categorized indicator diagramstotal number of indicator diagrams.The training result of CNN-LSTM is shown in Figures8(a) and 8(b). The accuracy and loss results are shown in Figure 8(c).Figure 8
CNN-LSTM training results. (a) Training and testing dataset recognition. (b) Training and testing accuracy. (c) Accuracy and loss of CNN-LSTM.
(a)
(b)
(c)The blue part in Figure8(a) shows the recognition of the training dataset, and the purple part shows the test dataset recognition. Figure 8(b) shows that the recognition results both in the training and test datasets are superior. The training accuracy reaches 96.7%, whereas the testing accuracy reaches 94.1%. From the recognition results, we can observe that the proposed CNN-LSTM is quite robust against changes in the training data. By choosing a sufficient training time, the epoch can yield reasonable and accurate identification results.Figure8(c) shows that as the training epoch becomes large, the accuracy of the proposed CNN-LSTM identification method accordingly increases, and finally, the accuracy reaches 98.4%. The loss is less than 0.9%. As this epoch increases, CNN-LSTM becomes more powerful, which can process complex gradual changing fault recognition. Here, overfitting is not a problem because it can be easily overcome by early training stop.


### 4.3. Comparison with Other Methods

At present, the main method of fault recognition is to design experiments to simulate the working environment of a rod pumping system and then employ the theoretical mathematical model based on the experimental data. Next, different algorithms are adopted to calculate the established model. Fault diagnosis is influenced by the data. According to the literature [21], five analysis algorithms, namely, random forest, linear regression, boosting, partial least square, and K-nearest neighbor algorithms are adopted by the SPSS Model tools. To identify the fault, the accuracy is considered as an evaluation index. The comparison of the traditional and CNN-LSTM methods is shown in Figure 9.Figure 9
Accuracy of the algorithm and CNN-LSTM method.Compared with the other methods, the CNN-LSTM method overcomes the limitations of the theoretical analysis in the traditional models and improves the diagnosis accuracy. The traditional model contains many assumptions, which lead to a large deviation between the actual production environment and theoretical value. Thus, the theoretical model cannot achieve high-precision identification. In addition, the oilfields are widely equipped with automatic data-acquisition equipment to realize real-time monitoring of oil-well status and collect and save production data. The neural network performs better in dealing with a large number of data processing. In addition, the neural network has shown absolute advantages in data mining of hidden values and improves data utilization.


## 4.1. Setting

CNN-LSTM experiments are conducted on the training dataset consisting of 1.3 million sets of well-production test data. The parameter settings are listed in Table2.Table 2
Parameter setting.
No.
Convolution layers
Cove
Pooling layers
Sampling rate
Hidden layers
Dropout
Train time (h)
Accuracy (%)
1
2
16 : 32
2
2 × 2
1/136
0
183
92.1
2
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.25
276
92.5
3
2
16 : 32
2
2 × 2
1/136
0
159
92.6
4
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.25
264
92.9
5
2
16 : 32
2
2 × 2
1/136
0
76
93.8
6
2
16 : 32
2
2 × 2
2/2048 : 512
0.25
83
94.2
7
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.25
135
96.1
8
4
16 : 32 : 64 : 128
4
2 × 2
2/2048 : 512
0.5
168
95.7The neural network recognition accuracy of the seventh architecture is shown to be as high as 96.1%. The selected CNN convolution kernel parameters are 16, 32, 64, and 128.


## 4.2. Evaluation of CNN-LSTM

For the datasets, 80% of the indicator diagrams are used for training and 20% are used for testing. The proposed method is evaluated based on the categorization accuracy, which is defined as follows:(7)accuracy=number of correctly categorized indicator diagramstotal number of indicator diagrams.The training result of CNN-LSTM is shown in Figures8(a) and 8(b). The accuracy and loss results are shown in Figure 8(c).Figure 8
CNN-LSTM training results. (a) Training and testing dataset recognition. (b) Training and testing accuracy. (c) Accuracy and loss of CNN-LSTM.
(a)
(b)
(c)The blue part in Figure8(a) shows the recognition of the training dataset, and the purple part shows the test dataset recognition. Figure 8(b) shows that the recognition results both in the training and test datasets are superior. The training accuracy reaches 96.7%, whereas the testing accuracy reaches 94.1%. From the recognition results, we can observe that the proposed CNN-LSTM is quite robust against changes in the training data. By choosing a sufficient training time, the epoch can yield reasonable and accurate identification results.Figure8(c) shows that as the training epoch becomes large, the accuracy of the proposed CNN-LSTM identification method accordingly increases, and finally, the accuracy reaches 98.4%. The loss is less than 0.9%. As this epoch increases, CNN-LSTM becomes more powerful, which can process complex gradual changing fault recognition. Here, overfitting is not a problem because it can be easily overcome by early training stop.


## 4.3. Comparison with Other Methods

At present, the main method of fault recognition is to design experiments to simulate the working environment of a rod pumping system and then employ the theoretical mathematical model based on the experimental data. Next, different algorithms are adopted to calculate the established model. Fault diagnosis is influenced by the data. According to the literature [21], five analysis algorithms, namely, random forest, linear regression, boosting, partial least square, and K-nearest neighbor algorithms are adopted by the SPSS Model tools. To identify the fault, the accuracy is considered as an evaluation index. The comparison of the traditional and CNN-LSTM methods is shown in Figure 9.Figure 9
Accuracy of the algorithm and CNN-LSTM method.Compared with the other methods, the CNN-LSTM method overcomes the limitations of the theoretical analysis in the traditional models and improves the diagnosis accuracy. The traditional model contains many assumptions, which lead to a large deviation between the actual production environment and theoretical value. Thus, the theoretical model cannot achieve high-precision identification. In addition, the oilfields are widely equipped with automatic data-acquisition equipment to realize real-time monitoring of oil-well status and collect and save production data. The neural network performs better in dealing with a large number of data processing. In addition, the neural network has shown absolute advantages in data mining of hidden values and improves data utilization.


## 5. Conclusion

In this paper, we have proposed the combination of CNN and LSTM to perform gradual changing fault identification using the indicator diagram. We utilize the hierarchical structure of CNN to create image features from the indicator diagram and employ the recurrent structure of LSTM to integrate the features. We determine the periodic change in a gradual changing fault load. Corresponding to each indicator diagram, a softmax classifier is trained based on the outputs of LSTM, whose outputs are combined to identify the type of input fault. We evaluate the proposed CNN-LSTM recognition method using 1.3 million datasets. The experiments demonstrate the effectiveness of utilizing CNN and LSTM to recognize the gradual changing fault using the indicator diagram and characteristic parameters. Experiments on the proposed CNN-LSTM recognition method demonstrate that the accuracy reaches 98.4%, and the loss is less than 0.9%.



---

*Source: 4203821-2019-11-03.xml*