# Wind Turbine Predictive Fault Diagnostics Based on a Novel Long Short-Term Memory Model
## 1 Introduction
[22] for WT mechanical faults with a median accuracy between 93.5% and 98% in fault detection. For various classification tasks, SVM [18; 19; 23], \(k\)-NN [20; 21], ANN [22; 24; 25], and RF [25; 26] are commonly used with SCADA, simulation, or experimental data. Most importantly, accurate fault diagnosis is the prerequisite for developing any prediction model. ANN has been widely applied to the ML approach for supervised classification learning [27]. The typical ANN architecture, multi-layer perception (MLP), is a feed-forward multi-layered neural network consisting of an input layer, several hidden layers, and an output layer. The ANN prediction results are determined by data size, data pre-processing, selected neural network structures under their optimum activation functions, etc. [16]. Due to its robustness towards poor-quality data with noise and system disturbances, a well-trained ANN model can still make wise predictions, which cannot be achieved by other ML classifiers [6]. With the escalation of quantitative data sizes and complexity, ANN is a model with ideal predictive results but a slow convergence speed. Compared to ANN, the Recurrent Neural Network (RNN) is a more promising neural network model for time- and sequence-based tasks because its recurrent structure captures the temporal dependency among inputs with sequential characteristics to predict the next scenario [28]. RNN is a class of deep-learning neural networks designed for variable-length sequence inputs by remembering important events and allowing the previous values as inputs to predict future outputs with recurrent connections in hidden layers. RNN overcomes the over- and under-fitting issues and reduces the convergence time compared to ANN. However, vanishing gradients, caused by error information flowing backward, are large barriers to the success of vanilla RNNs because of the resultant oscillating weights or loss of long-term dependencies [29]. To address vanishing gradients, Long-Short-Term Memory (LSTM), proposed by [PERSON] and [PERSON] [29], is a remarkable RNN model to control the information flow with additional interacting layers. Based on SCADAdata, [PERSON] et al. [30] verified the outperformance of LSTM over ANN and autoencoder (AE) for anomaly detection. The integrated LSTM-AE model further improved the detection accuracy due to the raw input processed by AE and the time feature managed by LSTM. Based on both single-sensor and multi-sensor signals, LSTM outperformed RNN and ANN on the classification of 11 faults on the wind wheel, bearing, bearing support, and rotor [31]. For a case study of fault classification on inner, outer, and ball faults from rolling bearings [32], LSTM demonstrated higher accuracy than ANN and SVM, and stacked LSTM further enhanced the prediction accuracy. The advantages of LSTM have been validated according to multiple time-series fault diagnosis tasks [30; 31; 32]. The time-series events can occur either synchronously or asynchronously. However, most of the RNN or LSTM models fail to make use of time as a feature by considering all inputs to be synchronous. [PERSON] et al. [33] proposed a model-agnostic vector representation for time, known as Time2 Vec (T2V), to be integrated with the LSTM model to refurbish the architecture with the consumption of time features. The key contributions in this paper can be generalised as follows:  
* A feature selection method, Recursive Feature Elimination (RFE) [34], is conducted along with an RF classifier for WT fault prediction. The weights of each feature are computed under the RF classifier, and the RFE application reserves the optimal number of features in order of their significance levels to maintain a balance between prediction accuracy and computational costs. * By integrating Time2 Vec into LSTM, this approach, T2V-LSTM, has been validated to outperform LSTM with a stationary Time2 Vec activation function based on several synchronous datasets [33].