# Fault Detection Method for Wind Turbine Generators Based on Attention-Based Modeling
## 2 Attention-Weighted Long Short-Term Memory Network (AW-LSTM)
### Attention-Weighted LSTM Fault Detection Model  
For bearing signal analysis, there is a significant temporal correlation between adjacent data frames. Moreover, the impact of previous data frames on the current frame is often different. In addition, different frames theoretically contribute differently to fault prediction in the entire time series signal. For example, signals at peaks and valleys often accompany data anomalies, making them more important for fault prediction. On the other hand, the importance of unrelated, noisy signals is lower. Therefore, if an algorithm can distinguish the importance of different data frames, it can improve the final bearing fault detection rate. Attention mechanism is a good probability weight allocation mechanism that calculates the attention probability weights at different times. It can allow nodes that are highly relevant to the fault prediction target to receive more attention and be assigned larger probability weights, thereby improving the quality of the hidden layer feature vector and helping to improve fault prediction accuracy. The model structure of the attention-weighted long short-term memory network proposed in this paper is shown in Figure 2.  
In the attention operation combined with LSTM, for the original data signal \(\mathbf{x}(t)\), with \(i\in(1,n)\), \(\mathbf{x}_{i}\) is input to the LSTM network, and the output hidden state is denoted as \((\mathbf{h}_{1},...,\mathbf{h}_{t-1},\mathbf{h}_{t},...,\mathbf{h}_{n})\). Then, after attention re-weighting, the new hidden state \(\mathbf{s}\) is obtained by jointly operating on the initially hidden state vectors \(\mathbf{h}_{i}\) from each time step. The calculation steps include  
\[ \begin{split}&\mathbf{e}_{i}=\mathbf{v}\tanh(\mathbf{w}\mathbf{h}_{i}+\mathbf{b})\\ &\alpha_{i}=\frac{\exp(\mathbf{e}_{i})}{\sum_{i=1}^{n}\exp(\mathbf{e}_{i })}\\ &\mathbf{s}=\sum_{i=1}^{n}\alpha_{i}\mathbf{h}_{i}\end{split}  \tag{5}\]  
Figure 2: Attention-weighted LSTM for fault detection. where \(\mathbf{e}_{i}\) is the intermediate energy value of the fourth hidden state, obtained from a full connection to the network, \(\mathbf{w}\) and \(\mathbf{v}\) represent the trainable weight matrices, \(\mathbf{b}\) denotes the bias vector, and \(\exp\) represents the natural exponential function. The resulting \(\alpha_{i}\) is the attention weight, which reflects other moments' hidden states for the present impact of the moment: if it is larger, it indicates that the \(i\)-th frame is more important and should be focused, and vice versa, by calculating the correlation between LSTM states and the resulting attention matrix and the final re-weighted feature representation. As shown in Figure 3, the AW-LSTM model proposed in this paper mainly consists of three parts: a data processing module, a feature extraction layer, and a data output layer. The data processing module decomposes the original data signal into \(D\) IMF components \(\{\mathbf{IMF}_{d}\}_{d=1}^{D}\) through EMD. The feature extraction layer is composed of four layers of LSTM and attention operations. The data output layer is implemented through an additional fully connected network. The implementation details are as follows. Data processing module. The original bearing vibration signals usually contain anomalies generated during the data acquisition process, so the isolated forest algorithm is used to eliminate the high anomalies in the data. In addition, in order to help the downstream neural network layer focus more explicitly on the high-frequency response in the data and ignore irrelevant terms, such as noise interference, this paper uses EMD to decompose the signal into IMF components and residuals at different frequencies, which contains a total of \(D=7(\mathbf{IMF}_{d})_{d=1}^{D}\) components. Feature extraction layer. The goal of the feature extraction layer is to efficiently analyze the IMF components of the bearing signal and to extract effective information, so as to achieve high-precision fault detection.