# Fault Early Warning Based on Improved Deep Neural Network of Auto-Encoder

Authors: Huijuan Hao; Huimiao Yuan; Yongwei Tang; Yu Zhang; Yuanyuan Zhao; Qingxuan Wei
Journal: Security and Communication Networks
 (2022)

---

## Abstract

In order to realize rapid fault detection and early warning, a fault detection method based on normal operation data is proposed. Firstly, the fault detection model is constructed based on the improved deep neural network of the auto-encoder. Secondly, the unsupervised pretraining and supervised fine-tuning of the network are finished through the operation data in a normal state to solve the contradiction between the small fault sample and the large training sample required by the deep network model. The adaptive threshold of reconstruction error is used as the evaluation index of the fault state to reduce the influence of environmental factors. Experimental results show that the proposed method can detect faults effectively.


---

## Body

## 1. Introduction

Major accidents caused by mechanical equipment failures always alert people to ensure the safe and reliable operation of equipment. Especially, the equipment failure of the key production line will bring significant shutdown losses. It will not only cause huge economic losses but also endanger personal safety in serious cases. The fault diagnosis and prediction of mechanical equipment play an important role in improving equipment operation reliability and optimizing operation [1]. It is the premise to ensure the safe and stable operation of mechanical equipment, and it is very important for the maintenance of mechanical equipment.Due to the large number of measuring points, monitoring parameters, and high sampling frequency of mechanical equipment, the complex condition of monitoring big data has been formed, which makes the fault diagnosis of mechanical equipment enter the era of big data [2–6]. The big data bring new opportunities and challenges to the health monitoring and fault diagnosis of mechanical equipment [7]. In recent years, deep learning has made a breakthrough in big data analysis in the fields of speech recognition and image recognition, and deep learning theory has been applied in the fields of mechanical fault diagnosis and health monitoring [8–11], but the diagnosis and identification of faults are still in the preliminary exploration stage [12–29]. A deep neural network of auto-encoder (DAE) can extract fault features from noise signals and can be well combined with a sample enhancement method to deal with small sample problems [30–37]. Fault diagnosis methods based on deep learning are often realized through supervised learning of a large number of fault data, which requires a large number of fault marking data. However, in practical industrial applications, the monitoring data are a large amount of normal operation data, and the fault data under abnormal conditions are difficult to collect from the mechanical system. The lack of tag training data is a common problem. It is difficult to reconcile the contradiction between small fault samples and large sample training for deep learning. How to solve the contradiction between small sample fault state data in actual fault diagnosis and prediction and large training samples required by the deep network model is a difficult problem. In order to realize the rapid detection and early warning of faults, this study constructs a fault detection model based on the improved deep self-coding network based on the data in the normal operation state, which provides a solution for the lack of complete samples in the actual diagnosis.


## 2. Fault Early Warning Model Based on Improved DAE

### 2.1. DAE Model

The DAE network is formed by stacking multiple automatic encoders (AE) [38, 39] (shown in Figure 1). The input data in high-dimensional space are converted into the coding vector in low-dimensional space through the coding network, and the coding vector in low-dimensional space is reconstructed back to the original input data through the decoding network.Figure 1
DAE model structure.


### 2.2. Fault Early Warning Model

When the equipment is in a normal state, the input and output of DAE fit very well, the original structure and stable relationship between variables have been maintained, and the reconstruction error of corresponding DAE is in dynamic and stable change. When the equipment is abnormal, it will cause the change of relevant variables, and the internal correlation of feature space will be destroyed which will be reflected in the error trend of variables. The reconstruction error of the DAE model will deviate from the original stable state and increase accordingly. It means that the fault occurs. Therefore, the abnormal change of reconstruction error on monitoring parameters is used as the index to detect the fault in this study.The reconstruction error of the variable is obtained from the difference between the reconstructed value and the actual value, as shown in the following formula:(1)Re=x′−x2,where Re is the reconstruction error of sample, x is the actual value, and x′ is the reconstruction value. The threshold is set according to the distribution of reconstruction error of training samples, and the fault detection criteria are shown in the following formula:(2)Re>Rth,fault warning.Re≤Rth,normal state.


### 2.3. The Improved Model

CNN was originally used to realize image classification and played an important role in the field of image classification. The vibration signal is one-dimensional data. Some scholars choose to process a one-dimensional vibration signal into a time-frequency map and input it into CNN as a two-dimensional image to realize fault diagnosis, which has achieved certain success. However, the above operations make the data processing inconvenient. Therefore, the one-dimensional CNN (1d-CNN) [40, 41] is used to directly process the original vibration signal. With the increase in network depth, the training becomes more and more difficult. Some special weight initialization strategies and batch normalization (BN) methods have greatly improved this problem. However, when the model converges, the methods bring another problem: with the increase of network depth, the training error does not decrease, but increases. Residual network (ResNet) solves the problem of training difficulty caused by network depth, and the network performance (accuracy and accuracy) is far better than the traditional network model.The original vibration signal is used as the input of 1d-CNN, and a self-coding network model is constructed, as shown in Figure2.Figure 2
Improved AE.


## 2.1. DAE Model

The DAE network is formed by stacking multiple automatic encoders (AE) [38, 39] (shown in Figure 1). The input data in high-dimensional space are converted into the coding vector in low-dimensional space through the coding network, and the coding vector in low-dimensional space is reconstructed back to the original input data through the decoding network.Figure 1
DAE model structure.


## 2.2. Fault Early Warning Model

When the equipment is in a normal state, the input and output of DAE fit very well, the original structure and stable relationship between variables have been maintained, and the reconstruction error of corresponding DAE is in dynamic and stable change. When the equipment is abnormal, it will cause the change of relevant variables, and the internal correlation of feature space will be destroyed which will be reflected in the error trend of variables. The reconstruction error of the DAE model will deviate from the original stable state and increase accordingly. It means that the fault occurs. Therefore, the abnormal change of reconstruction error on monitoring parameters is used as the index to detect the fault in this study.The reconstruction error of the variable is obtained from the difference between the reconstructed value and the actual value, as shown in the following formula:(1)Re=x′−x2,where Re is the reconstruction error of sample, x is the actual value, and x′ is the reconstruction value. The threshold is set according to the distribution of reconstruction error of training samples, and the fault detection criteria are shown in the following formula:(2)Re>Rth,fault warning.Re≤Rth,normal state.


## 2.3. The Improved Model

CNN was originally used to realize image classification and played an important role in the field of image classification. The vibration signal is one-dimensional data. Some scholars choose to process a one-dimensional vibration signal into a time-frequency map and input it into CNN as a two-dimensional image to realize fault diagnosis, which has achieved certain success. However, the above operations make the data processing inconvenient. Therefore, the one-dimensional CNN (1d-CNN) [40, 41] is used to directly process the original vibration signal. With the increase in network depth, the training becomes more and more difficult. Some special weight initialization strategies and batch normalization (BN) methods have greatly improved this problem. However, when the model converges, the methods bring another problem: with the increase of network depth, the training error does not decrease, but increases. Residual network (ResNet) solves the problem of training difficulty caused by network depth, and the network performance (accuracy and accuracy) is far better than the traditional network model.The original vibration signal is used as the input of 1d-CNN, and a self-coding network model is constructed, as shown in Figure2.Figure 2
Improved AE.


## 3. The Process of Fault Early Warning Method

Re fluctuates within the threshold range under normal conditions. When Re exceeds the threshold and remains above the threshold, it can be determined that the equipment has failed. The fault prediction method based on DAE mainly includes offline model training, reconstruction error construction, and online fault detection, as shown in Figure 3.Figure 3
Fault early warning model based on DAE.


### 3.1. Offline Model Training

The vibration signal in normal conditions collected by the sensor is normalized as samplesxii=1,2,…,n. The samples are divided into training set and test set. The training of the DAE model is shown in Figure 4.Figure 4
Training flow chart of DAE.The calculation between hidden layers is shown in the following formula:(3)xi,jl=F∑i=1L∑j=1mxi,jl−1×wi,jl−1+b,where xi,jl is the i-th characteristic value of the j-th feature vector value in the L-th layer of DAE, wi,jl−1 is weight coefficient, L is the convolution kernel size, F is the activation function, and b is the deviation value.Add an output layer after the last hidden layer, the output of DAE is shown in the following formula:(4)xi′=F∑i=1L∑j=1mxi,j×wi,j+b,where xi′ is the i-th output value of DAE, xi,j is the i-th characteristic value of the j-th feature vector in the last hidden layer of DAE, wi,j is weight coefficient, L is the convolution kernel size, F is the activation function, and b is the deviation value.The error between output and expected value is calculated by the following formula:(5)R=1m∑i=1nxi−xi′,where xi is the i-th true value of the sample, and xi′ is i-th output value of the DAE.θ updates with the following formula:(6)θ=θ−α∂R∂θ,where θ=W1,W2,…,Wn−1,W,b1,b2,…,bm−1,b, and α is the parameter learning rate.


### 3.2. Online Fault Detection

The vibration signalf of the equipment is collected in real time by the acquisition device, and the samples are normalized to obtain the data xi.Inputxi into the trained DAE model to obtain the corresponding reconstruction error Rei, and Rei is compared with evaluating the equipment status.


## 3.1. Offline Model Training

The vibration signal in normal conditions collected by the sensor is normalized as samplesxii=1,2,…,n. The samples are divided into training set and test set. The training of the DAE model is shown in Figure 4.Figure 4
Training flow chart of DAE.The calculation between hidden layers is shown in the following formula:(3)xi,jl=F∑i=1L∑j=1mxi,jl−1×wi,jl−1+b,where xi,jl is the i-th characteristic value of the j-th feature vector value in the L-th layer of DAE, wi,jl−1 is weight coefficient, L is the convolution kernel size, F is the activation function, and b is the deviation value.Add an output layer after the last hidden layer, the output of DAE is shown in the following formula:(4)xi′=F∑i=1L∑j=1mxi,j×wi,j+b,where xi′ is the i-th output value of DAE, xi,j is the i-th characteristic value of the j-th feature vector in the last hidden layer of DAE, wi,j is weight coefficient, L is the convolution kernel size, F is the activation function, and b is the deviation value.The error between output and expected value is calculated by the following formula:(5)R=1m∑i=1nxi−xi′,where xi is the i-th true value of the sample, and xi′ is i-th output value of the DAE.θ updates with the following formula:(6)θ=θ−α∂R∂θ,where θ=W1,W2,…,Wn−1,W,b1,b2,…,bm−1,b, and α is the parameter learning rate.


## 3.2. Online Fault Detection

The vibration signalf of the equipment is collected in real time by the acquisition device, and the samples are normalized to obtain the data xi.Inputxi into the trained DAE model to obtain the corresponding reconstruction error Rei, and Rei is compared with evaluating the equipment status.


## 4. Determination of Adaptive Threshold

In the fault prediction, it is commonly used to set the fixed threshold of the reconstruction error. However, due to the influence of environment, modeling error, and other factors, the reconstruction error in a normal state may fluctuate. So, if a fixed threshold is set for state evaluation, it is easy to cause a false alarm. Therefore, the adaptive reconstruction error is set for fault detection and early warning in this study.According to the principle of statistics, the mean and variance of residuals are calculated as follows:(7)μRj,tk=1n∑i=1nritk|Rj,σ2Rj,tk=1n−1∑i=1nritk−μRj,tk2|μj,where Rj is the reconstruction error corresponding to different times.The confidence interval of the mean value (the confidence is1−α) can be expressed as the following formula:(8)Pμ¯−zα<μ<μ¯+zα=1−α,where α is the confidence level, and z is the correlation coefficient of the confidence level. In practical application, the confidence is usually set to be 95%–99%.If the confidence is 95%, thenz is 1.96, and the threshold can be obtained from the following formula:(9)Rthl=μRj,tk±1.96σ2Rj,tk.TakeRthl as the threshold. When the residual exceeds this threshold and remains above the threshold, it indicates that the system is abnormal and in the early stage of fault.


## 5. Experiment and Analysis

### 5.1. Experimental Equipment and Data

As shown in Figure5, the data are collected from the two-stage gearbox.Figure 5
Gearbox experimental system.The gear speed change is controlled by a motor. The torque is provided by the magnetic brake and can be adjusted by changing the input voltage.The pinion with 32 teeth and 80 teeth is installed on the first-stage input shaft. The second stage consists of a 48 teeth pinion and a 64 teeth pinion. The input shaft speed is measured by the tachometer, and the gear vibration signal is measured by the accelerometer. The vibration signals of normal and fault samples used in this study are shown in Figure6 and Figure 7.Figure 6
Normal sample.Figure 7
Fault sample.


### 5.2. Model Parameter of Improved DAE

A seven-layer DAE model is constructed in this study. The network structure parameter is shown in Table1, and the parameter of the model is shown in Table 2.Table 1
Network structure parameter.
Network layerCombination form of node73600-1200-400-134-400-1200-3600Table 2
Parameter of the model.
LayerStructure nameStructure parameterNumber of channelsOutput size1Input(3600,1)1(3600,1)2Convolution layer(1,1)5(3600,1)3Convolution layer(3,1)4(1200,1)4Convolution layer(3,1)8(400,1)5Convolution layer(3,1)4(134,1)6Convolution layer(3,1)8(400,1)7Convolution layer(3,1)4(1200,1)8Convolution layer(1,1)5(3600,1)9Convolution layer(3600,1)1(3600,1)


### 5.3. DAE Model Test of Gearbox under Normal Condition

After the model is trained, the data of the gearbox under normal state are tested. The adaptive threshold is calculated by (9). The change trend of Re obtained from DAE is shown in Figure 8. It can be seen that Re of the gearbox is always within the adaptive threshold range under normal conditions.Figure 8
Re within the adaptive threshold.


### 5.4. DAE Model Test of Gearbox under Normal Condition

The data before and after the fault are used to verify the fault detection effect of the model. When the model is used in the gearbox fault state, calculateRe., The trend of Re is shown in Figure 9.Figure 9
Re exceeded the adaptive threshold.It can be seen thatRe exceeds the threshold at 2200, and the value is basically above the threshold. The gearbox fault can be determined, and the early warning of fault can be carried out.


## 5.1. Experimental Equipment and Data

As shown in Figure5, the data are collected from the two-stage gearbox.Figure 5
Gearbox experimental system.The gear speed change is controlled by a motor. The torque is provided by the magnetic brake and can be adjusted by changing the input voltage.The pinion with 32 teeth and 80 teeth is installed on the first-stage input shaft. The second stage consists of a 48 teeth pinion and a 64 teeth pinion. The input shaft speed is measured by the tachometer, and the gear vibration signal is measured by the accelerometer. The vibration signals of normal and fault samples used in this study are shown in Figure6 and Figure 7.Figure 6
Normal sample.Figure 7
Fault sample.


## 5.2. Model Parameter of Improved DAE

A seven-layer DAE model is constructed in this study. The network structure parameter is shown in Table1, and the parameter of the model is shown in Table 2.Table 1
Network structure parameter.
Network layerCombination form of node73600-1200-400-134-400-1200-3600Table 2
Parameter of the model.
LayerStructure nameStructure parameterNumber of channelsOutput size1Input(3600,1)1(3600,1)2Convolution layer(1,1)5(3600,1)3Convolution layer(3,1)4(1200,1)4Convolution layer(3,1)8(400,1)5Convolution layer(3,1)4(134,1)6Convolution layer(3,1)8(400,1)7Convolution layer(3,1)4(1200,1)8Convolution layer(1,1)5(3600,1)9Convolution layer(3600,1)1(3600,1)


## 5.3. DAE Model Test of Gearbox under Normal Condition

After the model is trained, the data of the gearbox under normal state are tested. The adaptive threshold is calculated by (9). The change trend of Re obtained from DAE is shown in Figure 8. It can be seen that Re of the gearbox is always within the adaptive threshold range under normal conditions.Figure 8
Re within the adaptive threshold.


## 5.4. DAE Model Test of Gearbox under Normal Condition

The data before and after the fault are used to verify the fault detection effect of the model. When the model is used in the gearbox fault state, calculateRe., The trend of Re is shown in Figure 9.Figure 9
Re exceeded the adaptive threshold.It can be seen thatRe exceeds the threshold at 2200, and the value is basically above the threshold. The gearbox fault can be determined, and the early warning of fault can be carried out.


## 6. Conclusion

This study presents a fault detection method based on normal operation data, which solves the contradiction between small sample fault data and large training samples required by the deep network model. The main conclusions are as follows:(1)
An improved DAE network is designed, which directly uses the original vibration data for fault detection, and the data in the normal state are used for unsupervised pretraining and supervised fine-tuning of the network. In this method, the reconstruction error is selected as the fault state parameter. In order to reduce the misjudgment caused by environmental factors, the adaptive threshold of reconstruction error is introduced as the decision-making basis of fault early warning.(2)
The experimental results show that under the premise of small data sets, this model has a very high accuracy and can quickly realize fault detection and early warning.(3)
This study constructs an improved self-encoder prediction model mainly for small data sets of gearboxes, but in practice, the factors affecting the high-precision prediction of gears are extremely complex. With the continuous development of deep learning algorithm research, combined with gearbox monitoring data, advanced deep learning algorithm and comprehensive high-precision prediction model of multiple environmental factors outside the gearbox will be the next focus of this study.



---

*Source: 5767642-2022-09-09.xml*