# Research on Fault Diagnosis Algorithm of Ship Electric Propulsion Motor
## 2 Theoretical Background
### Long Short-Term Memory Neural Network  
The neurons of the LSTM model have a cell state and three gate mechanisms. As shown in Figure 3, the LSTM model has three gates: the forgetting gate, the updating gate, and the output gate, through which the cell state is protected and controlled.  
Forgetting door: The sigmoid function determines which information message is discarded, and it can be expressed as:  
\[f(t)=\sigma\Big{(}W_{f}[a(t-1),x(t)]+b_{f}\Big{)} \tag{20}\]  
where \(f(t)\) is the quantity outgoing to each cell in state \(c(t-1)\); \(a(t-1)\) is the outgoing at time \(t-1\); \(x(t)\) is the intromission at time \(t\); \(\sigma\) is a sigmoid function, \(\sigma(x)=\left(1+e^{-x}\right)^{-1}\); \(W_{f}\) is the weight of each variable, and \(b_{f}\) is the bias term.  
Update door: Update storage information message. First, the sigmoid function calculates the result \(u(t)\) to determine which information message to update. Then, a new candidate value vector \(\hat{c}(t)\) is generated according to the Tanh function and added to the cell state. By multiplying the old cell state with the forgetting gate \(f(t)\), part of the old information message is forgotten, \(u(t)\ast\hat{c}(t)\) is added, and the current cell state is updated. It can be expressed as:  
\[u(t)=\sigma(W_{u}[a(t-1),x(t)]+b_{u}) \tag{21}\]  
\[\hat{c}(t)=tanh(W_{c}[a(t-1),x(t)]+b_{c}) \tag{22}\]  
\[c(t)=u(t)\odot\hat{c}(t)+f(t)\odot c(t-1) \tag{23}\]  
where \(u(t)\in[0,1]\); \(tanh\) outputs a value between \(-1\) and \(1\); \(c(t-1)\) is the cell status value at time \(t-1\); \(\hat{c}(t)\) indicates the information message to be extracted from the intromission information at time \(t\), and \(c(t)\) indicates the updated cell status value.  
Output gate: Output information message based on the cell status. The sigmoid function determines the quantity of output information. The \(tanh\) function processes \(c(t)\) to obtain a value between \(-1\) and \(1\). \(o(t)\) and \(c(t)\) are multiplied to obtain the output value at time \(t\), which can be expressed as:  
\[o(t)=\sigma(W_{o}[a(t-1),x(t)]+b_{o}) \tag{24}\]  
\[a(t)=o(t)\odot tanh(c(t)) \tag{25}\]  
Structurally, compared with the one-way LSTM network, the Bi-LSTM network adopts a two-way loop structure of forward and backward propagation. Viewing the problem from the flow of time, Bi-LSTM adds the data flow from the future into the past based on the one-way flow of LSTM data from the past into the future, and the hidden layer for the past and the hidden layers for the future are independent of each other, so LSTM can better explore the time series feature of data.  
The structure of the Bi-LSTM network is shown in Figure 4.  
Figure 3: The structure of the LSTM model.  
Forward propagation has two steps: (1) move from left to right and calculate from the original time step to the ultimate time step; move from right to left and calculate from the last time step to the ultimate time step. At time step \(t\), the forward layer and the backward layer in the Bi-LSTM use the same input \(X_{t}\), and they jointly generate the output and state. However, the two LSTM network layers do not share the bias vector parameters and weight matrix. The formula can be expressed as:  
\[h_{t}^{1}=f\Big{(}W^{1}\cdot X_{t}+U^{1}\cdot h_{t-1}\Big{)} \tag{26}\]  
\[h_{t}^{2}=f\Big{(}W^{2}\cdot X_{t}+U^{2}\cdot h_{t+1}\Big{)} \tag{27}\]  
\[Y_{t}=soft\;max\Big{(}V^{1}h_{t}^{1}+V^{2}h_{t}^{2}\Big{)} \tag{28}\]  
where \(f\) is the hidden-layer activation function; \(W^{1}\) and \(W^{2}\) are weight matrix parameters from the intromission layer to the hidden layer; \(U^{1}\) and \(U^{2}\) are weight parameters from the hidden layer to the hidden layer, \(V^{1}\) and \(V^{2}\) are weight parameters from the hidden layer to the outgoing layer. The weight matrix and bias vector parameters of Bi-LSTM are adjusted by backpropagation according to the error function output by \(Y_{t}\) theory.