# Feature Mining and Sensitivity Analysis with Adaptive Sparse Attention for Bearing Fault Diagnosis
## 2 Introduction of Relevant Basic Models
### BiLSTM Network  
Bearing vibration acceleration time series contains rich state information. Recursive analysis of the time series can help extract the periodic response characteristics [31; 32]. The recursive neural network (RNN) includes feedback connections between its hidden layer and the layer before it [33], and it has the ability to process sequential information. RNN can be trained using back propagation with target output and sequence input data. For the input sequence \(x=[x_{1},x_{2},\ldots,x_{N}]\), RNN hidden state vector calculation sequence \(h=[h_{1},h_{2},\ldots,h_{N}]\), the output sequence \(y=[y_{1},y_{2},\ldots,y_{N}]\), through the iterative equation below, from \(t=1\) to \(N\). \[h_{t}=f(w_{xh}x_{t}+w_{hh}h_{t-1}+b_{h}) \tag{3}\]  
\[y_{t}=w_{hy}h_{t}+b_{y} \tag{4}\]  
The \(w\) is the weight matrix and the \(b\) is the offset term. Where \(w_{xh}\) represents the weight matrix for transformation between the hidden layer and the input layer, and \(b_{h}\) is the offset vector of the hidden layer. The \(f\) is the nonlinear activation function of the hidden layer. In general, \(\gamma\) can be a sigmoid function in RNN. However, RNN's performance suffers significantly as a result of the gradient disappearance issue during back propagation, which indicates that RNN may not be able to effectively capture long sequence data features. Therefore, a long- and short-term memory architecture (LSTM) was proposed to model the dependency between long sequences and prevent the problem of gradient disappearance in back propagation [19]. Figure 1 shows the LSTM operation process. The core idea of LSTM lies in its memory unit, which is an accumulator of state information. The LSTM has the ability to add or remove information from the cellular state, and this fine-tuning is achieved through the structure of the \"gate\". Among them, the input gate \(i_{t}\) controls how much of the current network input \(x_{t}\) is saved to the unit state \(c_{t}\), the forgetting gate \(f_{t}\) controls how much of the last moment's unit state \(c_{t-1}\) is reserved to the current moment \(c_{t}\), and the output  
gate \(o_{t}\) controls how much of the unit state \(c_{t}\) is sent to the current output value \(h_{t}\) of LSTM. The mainstream LSTM algorithm in [34] is adopted in this study, as shown in the following function:  
\[i_{t}=\sigma(w_{xi}x_{t}+w_{hi}h_{t-1}+w_{ci}c_{t-1}+b_{i}) \tag{5}\]  
\[f_{t}=\sigma\Big{(}w_{xf}x_{t}+w_{hf}h_{t-1}+w_{cf}c_{t-1}+b_{f}\Big{)} \tag{6}\]  
\[c_{t}=f_{t}c_{t-1}+i_{t}\ tanh(w_{xc}x_{t}+w_{hc}h_{t-1}+b_{c}) \tag{7}\]  
\[o_{t}=\sigma(w_{xo}x_{t}+w_{ho}h_{t-1}+w_{co}c_{t}+b_{o}) \tag{8}\]  
\[h_{t}=o_{t}\ tanh(c_{t}) \tag{9}\]  
where \(\sigma\) is the sigmoid activation function. Accordingly, the w matrix subscript in the formula is easy to understand. For example, \(w_{hi}\) represents the matrix of transformation between the input gate and the hidden layer, \(w_{xc}\) represents the matrix of transformation between the input state and the output gate, etc. With LSTM, the back propagation gradient can be captured in memory cells to prevent rapid loss. LSTM assumes that the current time step is determined by the sequence of previous earlier time steps; thus, it passes information forward and backward through hidden states. Sometimes, the current time step may also be determined by subsequent time steps. If LSTM can keep an eye out for both front and back information, the network will better mine the characteristic information of vibration signals. Bidirectional RNN realizes bidirectional information transmission by setting two parts: forward hidden information \(\overset{\rightarrow}{h}\) and backward hidden information \(\overset{\leftarrow}{h}\).