## 3 Fault Diagnosis Method Based on LSTM and Multi-Sensor Data Fusion  
### The Network Architecture of LSTM Based on Multi-Sensor Data Fusion  
The architecture of a LSTM network is one kind of Recurrent Neural Network (RNN) network. Compared with basic version of RNN, LSTM can address the problem of exploding gradients and gradient vanishing and deal with relatively long-term data. The network architecture of LSTM is more complicated than of RNN, and the internal framework of LSTM adds three gated activation functions, i.e., input gate \(i_{t}\), forgetting gate \(f_{t}\) and output gate \(o_{t}\). The gated mechanism alleviates the problems of vanishing gradient and exploding gradient of RNN and solves the problem of long-term dependence relationship. The essential updating of LSTM can be described as follows:  
\[i_{t}=\sigma(W_{i}\cdot[h_{t-1},\ x_{t}]+b_{i}) \tag{1}\]  
\[f_{t}=\sigma\Big{(}W_{f}\cdot[h_{t-1},\ x_{t}]+b_{f}\Big{)} \tag{2}\]  
\[o_{t}=\sigma(W_{o}\cdot[h_{t-1},\ x_{t}]+b_{o}) \tag{3}\]  
where, \(i_{t},f_{t}\) and \(o_{t}\) denote input gate, forget gate and output gate, respectively, \(h_{t-1}\) denotes last time output, \(W_{s}\) denotes the weight coefficient matrix, and \(b_{s}\) represents the offset vector. The network architecture of a basic LSTM block is displayed in Figure 1. According to characteristics of time sequence data and multi-factor correlation, a fault diagnosis method of an A-class thermal insulation panel production line is proposed based on LSTM and multi-sensor feature fusion. The detailed network architecture is shown in Figure 2. The outputs of LSTM block are concatenated along channels, which satisfy the requirementsfor subsequent attention and dilated convolution modules. Because of the implementation of a ResNet block, and attention and dilated Convolution modules, the proposed model can merge the outputs of the LSTM block and predict the state of the production line.  
The basic LSTM block has three main stages:  
* Forget stage. This phase is mainly about selectively forgetting the input passed in by the previous node. That is, the network determines whether the intermediate information should be forgotten or remembered. Specifically, it controls information flow from the previous state, which needs to be retained or forgotten, by calculating the value of the forgetting gate.
* Memory stage. This stage is to decide what new information should be kept in the cell state. A sigmoid layer as the input gate layer decides which values should be updated. A tanh layer creates a vector of new candidate values, then combines above values to create an update to the state;
* Output stage. This is the final stage to decide which information is outputted. This output of the proposed CNN-LSTM contains prediction of each sensor deployed on equipment. Then, the network output is fed into the k-NN algorithm to obtain the final prediction.  
The network output shown in Figure 2 is a ten-dimension tensor in which each dimension corresponds to the state of each sensor deployed in the A-class thermal insulation panel production line. Therefore, the output of the proposed CNN-LSTM model is a prediction of the current state of the production line. Finally, the fault diagnosis result is predicted by the k-NN algorithm, as further discussed in Section 4.  
The proposed network structure of LSTM based on hierarchical residual connection is divided into three parts: the LSTM block, the ResNet module and the attention module. To enhance the feature expression ability of the current hidden layer state, the output of LSTM block is stacked by four residual connections and a LeakyReLU activation function. Then, the feature extraction capability of the network is improved by the attention module  
Figure 1: The schematic diagram of basic LSTM block.  
Figure 2: The network architecture of the proposed CNN-LSTM based on multi-sensor fusion.  
and the dilated convolution module. Finally, the predicted results are obtained by the Softmax function.