## 4 PdM Structure Development Methodology
### Predictive Analytics  
The aim of this phase, as per the process shown in Figure 3, is to identify the appropriate predictive machine learning model for queried application by processing the raw historical data from sensors that have been kept for a certain period up to the chosen fitting model. Data processing involves the understanding of data, cleaning data from outliers and other artifacts. This might lead to the easier extraction and selection of valuable information related to the parameter being monitored. The feature extraction focuses on the interesting signals from the preprocessed data that can be the useful indicators for a fault detection and failure point assumption. Different machine learning models can be trained in order to classify data and provide prediction before the monitored component is likely to oblige repair or replacement. From the literature, traditional time-series techniques, such as Hidden Markov model [90] and Kalman filtering [91], perform the prediction by detecting variations in data distribution based on predefined sequences, interval of time or distance, as well as thresholds. The main shortcoming of those algorithms is the ability to learn long term dependence for detecting the relation between features in different time series. Our real time monitoring requires a model with the capability to keep the long time dependence performance data without predefined time steps ahead of time. Thus, the Seasonal Autoregressive Integrated Moving Average (SARIMA) and LSTM models are candidates, where LSTM performed well on our experimental data with a low root mean square error (RMSE) when compared to SARIMA. The Long Short Term Memory (LSTM) predictive model that was introduced and modified in [92; 93] is proposed to this application with the aim of keeping the memories of long past time point to predict the tendency of future behaviors of the equipment. LSTM is a special type of Recurrent Neural Network (RNN) with a hidden unit that consists of four gates, namely input (i), output (o), forget (f), and update (u), as illustrated in Figure 4. The LSTM unit performs the mathematical Equation (1) to compute Equations (2)-(8), for four different gates with same size, in order to update the next hidden state \(h_{t}\) and cell state and compute an output \(Y_{t}\) at a specific time step \(t\) by taking the previous hidden state \(h_{t-1}\) and current input \(x_{t}\), stacking them, and then multiplying them with weight matrix, \(W\), all being triggered by activation functions. Gates are used in order to update memory cell \(C_{t}\) at time \(t\), with new time step information and forgetting the previous historical data, and then expose part of cell as the hidden state at the next time step. Figure 3: Predictive analytics and modeling process. LSTM Cell function:  
\[h_{t} \begin{pmatrix}i\\ f\\ o\\ u\end{pmatrix} = \begin{pmatrix}\delta\\ \delta\\ \phi\end{pmatrix} W \begin{pmatrix}h_{(t-1)}\\ x_{t}\end{pmatrix}  \tag{1}\]  
\[i_{t}=\delta(W_{x_{t}}x_{t}+W_{h_{t}}h_{t-1}+b_{i}) \tag{2}\]  
\[f_{t}=\delta(W_{x_{t}}x_{t}+W_{h_{t}}h_{t-1}+b_{f}) \tag{3}\]  
\[o_{t}=\delta(W_{x_{t}}x_{t}+W_{h_{t}}h_{t-1}+b_{o}) \tag{4}\]  
\[u_{t}=\phi(W_{x_{u}}x_{t}+W_{h_{u}}h_{t-1}+b_{u}) \tag{5}\]  
\[C_{t}=f_{t}\odot C_{t-1}+i\odot u \tag{6}\]  
\[h_{t}=o_{t}\odot tanh(C_{t-1}) \tag{7}\]  
\[Y_{t}=W_{h_{t}}h_{t}+b_{o} \tag{8}\]  
\(\delta\) denotes an application of the sigmoid logistic activation function, \(\phi\) denotes an elementwise application of hyperbolic tangent (tanh) activation function, \(x_{t}\) represents an input vector to model, \(h_{t-1}\) is the activation delivered in the previous sequence step, and \(C_{t}\) denotes the value of LSTM memory cell; all at the \(t^{t}h\) time step and \(h_{t}\) is the probability distribution result function at a given step, which is also the next hidden state. \(W_{x}\) and \(W_{h}\) are the input and hidden connection weight matrices to gates and \(b\) is the bias vectors employed to create connection between input layer, output layer, and memory block.