# Wind Turbine Predictive Fault Diagnostics Based on a Novel Long Short-Term Memory Model
## 3 Methods
### Time-LSTM-3 and T2V-LSTM Models  
For Time-LSTM-1, the single time gate \(T_{j}\) is mainly controlled by \(\Delta t_{j}\). [PERSON] et al. [39] developed two alternative Time-LSTM models, Time-LSTM-2 and Time-LSTM-3, both containing double time gates, \(T1_{j}\) and \(T2_{j}\). \(T1_{j}\) controls the influence of the last consumed item, \(x_{j}\), on current recommendations, while \(T2_{j}\) stores \(\Delta t_{j}\) for later recommendations. Based on \(T_{j}\) in Equation (9), \(T1_{j}\) and \(T2_{j}\) can be expressed by:  
\[T1_{j}=\sigma\big{(}W_{t1}x_{j}+\sigma(U_{t1}\Delta t_{j})+b_{t1}\big{)} \tag{12}\]  
\[T2_{j}=\sigma\big{(}W_{t2}x_{j}+\sigma(U_{t2}\Delta t_{j})+b_{t2}\big{)} \tag{13}\]  
where \(W_{t1}\), \(W_{t2}\), \(U_{t1}\), and \(U_{t2}\) are given weights and \(b_{t1}\) and \(b_{t2}\) are given biases. Among three LSTM models with time gates, Time-LSTM-3 is validated with the best predictions by  
Figure 6: LSTM architecture. coupling input and forget gates, inspired by the LSTM variant from [PERSON] et al. (2018) and the cell state, \(C_{j}\), in Equation (7) under Time-LSTM-3 can be modified by:  
\[C_{j}=\big{(}1-i_{j}\big{)}\odot C_{j-1}+i_{j}\odot\overline{C}_{j} \tag{14}\]  
Hence, Time-LSTM-3 has a shorter processing time than Time-LSTM-2 due to its simpler architecture and fewer parameters to calculate. By removing the forget gate, Equation (14) can be modified by:  
\[\widetilde{C}_{j}=\big{(}1-i_{j}\odot T1_{j}\big{)}\odot C_{j-1}+i_{j}\odot T1 _{j}\odot\overline{C}_{j} \tag{15}\]  
\[C_{j}=\big{(}1-i_{j}\big{)}\odot C_{j-1}+i_{j}\odot T2_{j}\odot\overline{C}_{j} \tag{16}\]  
where \(\widetilde{C}_{j}\) is a new cell state to store the result. The output gate, \(O_{j}\), in Equation (5) and the output vector, \(h_{j}\), in Equation (8) can be replaced by:  
\[O_{j}=\sigma\big{(}W_{o}x_{j}+U_{o}h_{j-1}+V_{o}\Delta t_{j}+b_{o}\big{)} \tag{17}\]  
\[h_{j}=\sigma_{h}\bigg{(}\widetilde{C}_{j}\bigg{)}\odot O_{j} \tag{18}\]  
Here, both \(i_{j}\) and \(T1_{j}\) are filters for \(\overline{C}_{j}\), while \(T2_{j}\) stores \(\Delta t_{j}\), transferred to \(C_{j}\), \(C_{j+1}\), \(C_{j+2}\), \(\cdots\), for modelling the long-term interests for later recommendations. \(\widetilde{C}_{j}\) is implemented through an activation function, \(\sigma_{h}\), influencing the current recommendations. A model-agnostic vector representation for time, known as Time2 Vec, is used to rebuild the architectures of Time-LSTM with the consumption of time features under either stationary or non-stationary activation functions. For this reason, Time2 Vec replaces the time interval, \(\Delta t_{j}\), by a model-agnostic vector, \(T2V(\Delta t_{j})\), as follows:  
\[T2V(\Delta t_{j})[i]=\left\{ \begin{aligned} &\omega_{i}\cdot\Delta t_{j}+ \varphi_{i},\;if\;i=0\\ &\mathcal{F}\big{(}\omega_{i}\cdot\Delta t_{j}+\varphi_{i}\big{)}, \;if\;1\leq i\leq k\end{aligned} \right. \tag{19}\]  
where \(T2V(\Delta t_{j})[i]\) is the \(i\)th element of \(T2V(\Delta t_{j})\), \(\mathcal{F}\) can be any stationary or non-stationary activation functions, such as _Sine_ and _Tanh_, and \(\omega_{i}\) and \(\varphi_{i}\) are learnable parameters. Then, in a T2V-LSTM model, all time vectors, \(\Delta t_{j}\), should be replaced by Time2 Vec elements, \(T2V(\Delta t_{j})\), so the time gates, \(T1_{j}\) and \(T2_{j}\), in Equations (12) and (13), respectively, and the output gate, \(O_{j}\), in Equation (17) are modified as follows:  
\[T1_{j}=\sigma\Big{(}\,W_{t1}x_{j}+\sigma(U_{t1}\cdot T2V(\Delta t_{j})\big{)} +b_{t1}\Big{)} \tag{20}\]  
\[T2_{j}=\sigma\Big{(}\,W_{t2}x_{j}+\sigma(U_{t2}\cdot T2V(\Delta t_{j})\big{)} +b_{t2}\Big{)} \tag{21}\]  
\[O_{j}=\sigma\Big{(}\,W_{o}x_{j}+U_{o}h_{j-1}+V_{o}\cdot T2V(\Delta t_{j})+b_{o }\Big{)} \tag{22}\]  
For T2V-LSTM, the output vector, \(h_{j}\), is still controlled according to Equation (18).