## 2 LSTM Neural Network Modeling
### LSTM Basic Algorithm  
A traditional RNN network can retain the effective information of the previous sequence samples and impact the current output. However, traditional RNN network training often has the phenomenon of gradient disappearance or gradient explosion when processing long input sequence samples, resulting in a poor network training effect. To solve this problem and improve the performance of an RNN network, some scholars have introduced gate operations on the hidden layer nodes of a traditional RNN network, resulting in some RNN variants, such as an LSTM network and a gated recurrent unit (GRU) network [21; 22; 23]. In practice, LSTM networks have achieved good results. An LSTM network can selectively memorize sequence information, with advantages such as the satisfactory applicability to measurement data, high accuracy, and short response time. The research of [PERSON] et al. [24] proved that the recognition rate of LSTM neural networks in diagnosis of different faults of a single or multiple sensors is over 96% and that the response time is less than 1 millisecond. [PERSON] et al. [25] applied an LSTM to the fault detection and recovery of UAV flight data, and they found that satisfactory performance was achieved. Three types of gates in LSTMnetworks are as follows: input gate, forget gate, and output gate. It is possible to control the flow of information between the hidden layers of an LSTM network by adjusting the state of these gates. The hidden layer of the original RNN has only one state and is very sensitive to short-term inputs. An LSTM adds one cell state to the RNN to keep the long-term state. An LSTM neural network structure is shown in Figure 1.  
The forget gate \(f_{t}\) determines the degree to which the long-term characteristics of preorder information are preserved or discarded in the cell state, which can be expressed as:  
\[\mu=\frac{1}{m}\sum_{i=1}^{m}x_{i}\]  
where \(f_{t}\) is the activation value of output \(h_{t-1}\) at \(t-1\), the input value at \(t\), and the bias term of \(b_{f}\). The function is sigmoid, and \(f_{t}\) is scaled down between zero (complete forgetting) and one (complete memory). \(W_{f}\) is the weight term and includes \(W_{f_{X}}\) and \(W_{fh}\).  
The input gate determines how to update the cell state, namely, the new information is selectively memorized to the cell state \(C_{t}\). The formula is as follows:  
\[\left\{\begin{aligned} & i_{t}=\sigma\ (W_{i}\cdot[h_{t-1},x_{t}]+b_{i})\\ &\widetilde{C}_{t}=\text{tanh}\ (W_{C}\cdot[h_{t-1},x_{t}]+b_{C})\\ & C_{t}={f_{t}}^{*}C_{t-1}+{i_{t}}^{*}\widetilde{C}_{t}\end{aligned}\right.\]  
where \(h_{t-1}\) and \(x_{t}\) are passed to the sigmoid function to obtain its value; \(h_{t-1}\) and \(x_{t}\) are passed to the tanh activation function to obtain the candidate value \(\widetilde{C}_{t}\) of the cell state, and it and \(\widetilde{C}_{t}\) jointly determine the new cell state; \(W_{i}\), and \(W_{C}\) are weight terms, including \(W_{ix}\) and \(W_{ih}\) and \(W_{Cx}\) and \(W_{Ch}\), respectively; \(b_{i}\) and \(b_{C}\) are bias terms; and \"*\" refers to multiplication with matrix elements.  
The output gate \(O_{t}\) determines the input at the present moment. Similar to the input gate, the information carried by \(h_{t}\) is determined by \(O_{t}\) and \(C_{t}\), and the formula is as follows:  
\[\left\{\begin{aligned} & O_{t}=\sigma(W_{O}\cdot[h_{t-1},x_{t}]+b_{ O})\\ & h_{t}=O_{t}*\text{tanh}(C_{t})\end{aligned}\right.\]  
where \(W_{O}\) is the weight term, which includes \(W_{ox}\) and \(W_{ho}\), and \(b_{O}\) is the bias term.  
The LSTM method diagram is shown in Figure 2.  
Figure 1: LSTM neural network structure.