## 3 Methodology
### Deep Neural Networks
#### 3.2.2 Recurrent Neural Network and LSTM
Each row in the data is a snapshot of data taken during a single running time cycle and includes 26 columns: the 1 st column indicates the engine ID, the 2 nd column indicates the current running cycle number, the 3 rd-5 th columns indicate three operational settings, the 6 th-26 th column indicate 21 sensor measurement data that have a substantial effect on engine performance. A detailed description of the dataset can be found in [44]. ### Results

In order to empirically evaluate the availability of the proposed scheme for RUL prognostics, the proposed scheme was tested with the testing set. As shown in Figure 9, four testing trajectories selected from four sub-datasets are taken as examples. Though some obvious errors exist between the prediction values and the true RUL values, the prognostic performance is good, especially when the equipment is close to failure. Preferably, accurate RUL prognostics of the equipment in the late phase would be able to improve management availability and reduce maintenance costs because it is extremely necessary to maintain or exchange equipment in the last phase of its lifetime. To further demonstrate the performance improvement of the proposed scheme, we compared four existing models with the same dataset:

**Model 1: Multilayer perceptron (MLP). ** MLP is a multilayer neural network to address the regression problem. In this paper, the MLP is constructed by using two hidden layers, where each layer consists of 50 neurons. The Relu function is the activation function in each hidden layer. **Model 2: Support vector regression (SVR). ** SVR is a machine learning method for time series prognostics. The aim of SVR is based on the computation of a linear regression function in a high-dimensional feature space where the input data are mapped via a nonlinear function [55]. In this paper, the Gaussian radical basis function (RBF) is considered as the kernel function. **Model 3: Convolutional neural network**. The standard convolutional neural network is introduced in Section 3.2.1. In this paper, we adopt CNN to extract spatial features and the fully connected neural network to obtain a regression model [33]. **Model 4: LSTM neural network**. The standard LSTM neural network is introduced in Section 3.2.2. In this paper, we adopt two LSTM layers to extract long-term temporary dependencies and the fully connected neural network to achieve RUL prognostics [8]. All of these methods were simulated and applied to the C-MAPSS dataset. Tables 2 and 3 show the results of the performance comparison using four sub-datasets. Figure 10 shows the model comparison in terms of the RMSE, MAP, MAE, and the average penalty score for the entire testing set. The average penalty score is defined as follows:

\[sc\bar{\sigma}re=\frac{1}{N}\sum_{i=1}^{N}score(i), \tag{25}\]

where \(score(i)\) is penalty score function, which is defined in Section 3.5, and \(N\) is the sample number of the testing set. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Metrics**} & \multicolumn{4}{c}{**FD001**} & \multicolumn{4}{c}{**FD002**} \\ \cline{2-10}  & **RMSE** & **MAPE** & **MAE** & **Score** & **RMSE** & **MAPE** & **MAE** & **Score** \\ \hline Model 1 & 29.152 & 25.404\% & 22.991 & \(2.71\times 10^{4}\) & 33.659 & 29.492\% & 27.999 & \(4.34\times 10^{5}\) \\ Model 2 & 25.123 & 20.364\% & 17.768 & \(4.63\times 10^{3}\) & 27.943 & 23.778\% & 19.447 & \(4.14\times 10^{5}\) \\ Model 3 & 21.512 & 18.002\% & 17.300 & \(3.98\times 10^{3}\) & 26.545 & 23.325\% & 16.772 & \(2.87\times 10^{4}\) \\ Model 4 & 19.402 & 16.421\% & 12.508 & \(1.96\times 10^{3}\) & 20.812 & 21.436\% & 13.161 & \(1.19\times 10^{4}\) \\ Proposed & 16.127 & 15.340\% & 10.957 & \(\textbf{3.03}\times 10^{2}\) & 20.455 & 21.306\% & 13.267 & \(3.44\times 10^{3}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of performance on sub-dataset (FD001) and sub-dataset (FD002). Figure 9: Model comparison.