# Anomaly Identification of Wind Turbine Yaw System Based on Two-Stage Attention-Informer Algorithm
## 3 Yaw Position Anomaly Recognition Model
### Model TSA-Informer
The TSA mechanism introduces an intermediate routing mechanism: at each time step \(\theta\), the model uses a learnable vector \(\lambda_{\theta,:}\) of fixed dimension as the intermediate routing vector. This vector is responsible for summarizing the information of each feature dimension. In this way, the model can focus on those features that are most critical for the current prediction. Subsequently, the model separates the information of each feature sequence from the intermediate routing and further analyzes and mines the mutual influences between features, thus achieving the cross and fusion of information at the feature level, specifically expressed as follows:  
\[\lambda_{\theta,:}=M_{\text{PSA1}}^{d}(R_{\theta,:},Z_{\theta},Z_{\theta,:}), 1\leq\theta\leq\frac{T}{L} \tag{7}\]  
\[\overline{Z}_{\theta,:}^{d}=M_{\text{PSA2}}^{d}(Z_{\theta,:},\lambda_{\theta, },\lambda_{\theta,:}),1\leq\theta\leq\frac{T}{L} \tag{8}\]  
In the formula, \(R_{\theta,:}\in R^{(T/L)\times c\times d}\) is the intermediate routing scientific vector matrix; \(Z_{\theta,:}\) is the two-dimensional vector at time step \(\theta\); \(\overline{Z}_{\theta,:}^{d}\) is the output of the intermediate routing mechanism at time step \(\theta\); \(T\) is the time sliding window; \(L\) is the sequence length of feature latitude \(d\). Through the interaction of the above two stages, the TSA mechanism not only improves the model's ability to capture time dependencies in time series data, but also enhances the recognition of complex relationships between features, thereby achieving more accurate and comprehensive predictions in multi-dimensional time series analysis tasks. This hierarchical attention allocation strategy enables the model to process high-dimensional time series data more flexibly and efficiently, providing strong support for improving the performance of diagnostic models. To enhance the Informer model's ability to capture the interrelations between feature dimensions in time series prediction tasks, an Informer model based on the Adam optimization algorithm and the TSA mechanism (hereinafter referred to as Adam-TSA-Informer) is proposed. The TSA mechanism improves the model's sensitivity to complex data by capturing short-term dependencies within time series and aggregating long-term relationships between different features. The adaptive learning rate adjustment strategy of the Adam optimizer makes model training more efficient and shows good stability, especially when dealing with long-term dependencies. This method enhances the generalization ability of the Informer model, enabling it to provide more accurate prediction results when facing variable time series data. Figure 1 shows the structure of the TSA-Informer model. The algorithm flow of the TSA-Informer model based on Adam optimization can be summarized as the following steps:  
(1) Initialization: for each parameter \(w\) in the model, set the first matrix \(m_{0}\) and the second matrix \(v_{0}\) to 0, and configure the initial learning rate \(\alpha\) as well as the decay rates \(\beta_{1}\) and \(\beta_{2}\). (2) Forward propagation: perform a forward propagation to obtain the model output and calculate the loss function \(L\). (3) Gradient calculation: determine the gradient \(\
abla_{w}L\) of each parameter through backpropagation. (4) First matrix update: update the first matrix \(m_{t}\) to be the exponential moving average of the gradient. \[m_{t}=\beta_{1}\cdot m_{t-1}+(1-\beta_{1})\cdot\
abla_{w}L \tag{9}\]  
(5) Second matrix update: update the second matrix \(v_{t}\) to be the exponential moving average of the squared gradient. \[v_{t}=\beta_{2}\cdot v_{t-1}+(1-\beta_{2})\cdot\left(\
abla_{w}L\right)^{2} \tag{10}\]  
(6) Bias correction: calculate the corrected first and second moments. \[\hat{m}_{t}=\frac{m_{t}}{1-\beta_{1}^{t}} \tag{11}\]  
\[\hat{v}_{t}=\frac{v_{t}}{1-\beta_{2}^{t}} \tag{12}\]  
(7) Adaptive learning rate: based on the corrected first and second moments, calculate the adaptive learning rate for the parameters.