# An Attention-Based Method for Remaining Useful Life Prediction of Rotating Machinery
## 3 Model Verification and Comparison
### Prediction Experiments Using the Attention-GRU Model
#### 3.3.2 Analysis of Experimental Results  
For comparison, the attention-GRU model proposed here and a GRU model without the attention mechanism are trained separately. To this end, 80% of the XJTU-SY dataset under working condition 2 is used for model training, and the remaining 20% is used for testing; the mean squared error (MSE) is used as the loss function, the learning rate is set to 0.001, and the number of training epochs is set to 30. Figure 10 shows the RUL prediction results achieved by the two models in comparison with the actual RUL.  
The blue curve in Figure 10 indicates the prediction result achieved by our attention-GRU model, and the red curve shows the result from the attention-free GRU model. As the figure shows, the GRU model with an attention module achieves better prediction performance than the attention-free model, with its curve better-fitted to the actual RUL curve and showing a smaller prediction error. Using RMSE as the evaluation index, the prediction error of the two models on the dataset was calculated separately (Table 5).  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Parameter**} & \multirow{2}{*}{**Optimizer**} & \multicolumn{3}{c}{**Attention-GRU**} & \multicolumn{3}{c}{**Dense**} \\ \cline{2-9}  & & **Input Size** & **Hidden Size** & **Num Layers** & **Attention Batch** & **Dense1** & **Dense2** & **Batch Size** & **Epoch** \\ \hline Parameter/class & Adam & 64 & 64 & 2 & 10 & 32 & 1 & 32 & 30 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Parameter configuration of the attention-GRU model.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Bearing Group** & **B2\_1** & **B2\_2** & **B2\_3** & **B2\_4** & **B2\_5** \\ \hline Attention-GRU & 0.092 & 0.167 & 0.316 & 0.174 & 0.253 \\ GRU & 0.156 & 0.305 & 0.524 & 0.279 & 0.348 \\ \hline \hline \end{tabular}
\end{table}
Table 5: RUL prediction error of attention-GRU and GRU.  
Figure 9: Attention-GRU model for RUL prediction experiments.  
Comparative experiments are also performed between the LSTM model proposed in [25] and our model; Table 6 compares their prediction error, and Figure 11 shows a comparison of the two models on the B2_1 dataset.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Bearing Group** & **Attention-GRU** & **LSTM** \\ \hline B2\_1 & 0.092 & 0.286 \\ B2\_2 & 0.167 & 0.227 \\ B2\_3 & 0.316 & 0.374 \\ B2\_4 & 0.174 & 0.216 \\ B2\_5 & 0.253 & 0.342 \\ \hline \hline \end{tabular}
\end{table}
Table 6: RUL prediction error of LSTM and our model.  
Figure 10: Comparison of RUL prediction results of bearings under working condition 2.  
According to Figures 10 and 11 and Tables 5 and 6, the attention-GRU model achieves a smaller prediction error and more stable prediction performance than other models compared in this study.  
As revealed in the experiments, in the training process of the attention-free GRU model, the weight assigned to healthy sample data, which present little fluctuation and little relevance to RUL, is not diminished, demonstrating the model's convergence and reducing the model's prediction accuracy. On the contrary, the model with an attention module (our model) adjusts the weight assigned to data as per the performance state corresponding to the data, which improves prediction accuracy and reduces errors (as shown in Figure 10 and Table 5). Meanwhile, the LSTM and the attention-free GRU models show larger fluctuations in the curves than our model because the gating mechanism works only on the current input and one previous state, and if the previous state or the current input is an anomaly, the subsequent predicted state will follow this anomalous trend, and the stability of prediction is poor.