## 5 Results and Analysis
### RUL Prediction Accuracy  
In Table 1, a comparison of the RUL prediction accuracy calculated for the proposed neural network performance metrics and the two described datasets is presented. Root Mean Square Error (RMSE) is a common neural network performance metric. The RMSE for the CMAPPS engine dataset ranged from 5.057 for the Unhealthy engine state to 12.947 for the Middle engine state. This was up to an order of magnitude lower than the RMSE achieved for the real-life engine operational dataset, which ranged from 62.618 for the Healthy engines to 130.201 for the Unhealthy engines. Comparing the R-squared performance metric of the two datasets, it may be concluded that the difference was not remarkably high and ranged from 2.1 to 12.8%. For the CMAPPS data, the achieved R-squared was close to 1, reaching 0.986 for the Unhealthy engine condition. Such a situation confirms the quality of the proposed model, which defines how well the engine model reflects real engine RUL. One of the crucial neural network performance metrics assessing engine RUL prediction is the Score function. This metric penalizes so-called late predictions, which means that the predicted engine RUL is higher than the real engine RUL. For the CMAPPS engine data, the achieved Score was close to zero. Unfortunately, the Score function was negative, which means that the predicted RUL is greater than the real one. However, looking at the real values, it was found that the difference is within the assumed tolerance range. In this case, it was five cycles, which is about 0.45% (on average) of the real engine's remaining useful life. Figure 6 presents the selected test engines' CMAPPS and real-life RUL prediction errors. Figure 6 illustrates the RUL prediction error for selected test engines from both the CMAPSS and real-life engine datasets. The figure provides a visual representation of the deviation between the predicted RUL and the actual RUL for individual engines. RUL prediction error is a crucial performance metric. Its significance lies in its specific consideration of situations where the predicted RUL value is lower than the real value. This is paramount for aviation safety, as a late RUL prediction (i.e., predicting a higher RUL than the actual remaining life) could lead to an in-flight engine failure, potentially resulting in an aircraft incident or accident. The RUL for all selected real-life test engines was positive, which indicates an early prediction (the model predicts failure before it happens). For 3 out of four selected CMAPPS test engines, the RUL prediction error is negative, which indicates a late or on-time prediction (the model predicts failure at or after it happens). Regarding the loss value (Table 1), results achieved for the CMAPPS dataset ranged from 0.079 for the Middle-state engines to 0.207 for Unhealthy engines. What was surprising was that despite the real-life engine dataset being full of noisy signals and plenty of outliers resulting from sensors' nuisance readings, the final loss results were lower, to the order of \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Engine Health State**} & \multicolumn{2}{c}{**RMSE**} & \multicolumn{2}{c}{**R\({}^{2}\)**} & \multicolumn{2}{c}{**Score**} & \multicolumn{2}{c}{**Loss**} \\ \cline{2-9}  & **CMAPPS** & **Real** & **CMAPPS** & **Real** & **CMAPPS** & **Real** & **CMAPPS** & **Real** \\ \hline Healthy & 9.847 & 62.618 & 0.971 & 0.951 & \(-\)0.138 & 1.899 \(\times\) 10\({}^{6}\) & 0.151 & 0.038 \\ Middle & 12.947 & 127.813 & 0.982 & 0.856 & \(-\)0.071 & 243.8 \(\times\) 10\({}^{6}\) & 0.079 & 0.031 \\ Unhealthy & 5.057 & 130.201 & 0.986 & 0.873 & \(-\)0.040 & 3.096 \(\times\) 10\({}^{4}\) & 0.207 & 0.028 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: RUL prediction accuracy comparison. magnitude for Healthy and Unhealthy engines, with comparable results for the Middle ones. This suggests that while the real-life data is more challenging, the model's training process was able to converge effectively, indicating its robustness.