# Multi-Head Self-Attention-Based Fully Convolutional Network for RUL Prediction of Turbofan Engines
## 5 Performance Comparison and Discussion
### Ablation Experiments for the Multi-Head Structure
It can be observed that the three modules using fewer heads obtain higher mean values in terms of RMSE and Score, while the three modules using more heads only slightly improve the corresponding performance metrics, or even make them worse. Based on these observations, it can be concluded that insufficient learning of the degradation information, due to the use of fewer heads and the over-parameterization of the neural network, due to the use of more heads, may lead to poor performance in RUL prediction. Therefore, it is crucial that a balance between performance and computation cost is found for RUL prediction. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Variant**} & \multicolumn{2}{c}{**MSA-FCN**} & \multicolumn{2}{c}{**SSD-SHF-SSF**} & \multicolumn{2}{c}{**SSD-SHF-MSF**} & \multicolumn{2}{c}{**SSD-MHF-SSF**} \\ \cline{2-9}  & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** \\ \hline RMSE & \multirow{2}{*}{**17.165**} & \multirow{2}{*}{**0.213**} & \multirow{2}{*}{18.714} & \multirow{2}{*}{0.613} & \multirow{2}{*}{19.419} & \multirow{2}{*}{1.406} & \multirow{2}{*}{18.652} & \multirow{2}{*}{0.721} \\ FD002 & & & & & & & & \\ Score & \multirow{2}{*}{1395.750} & \multirow{2}{*}{**93.024**} & \multirow{2}{*}{3371.237} & \multirow{2}{*}{564.548} & \multirow{2}{*}{3424.214} & \multirow{2}{*}{1780.444} & \multirow{2}{*}{3021.862} & \multirow{2}{*}{837.848} \\ FD002 & & & & & & & & \\ \hline RMSE & \multirow{2}{*}{**19.401**} & \multirow{2}{*}{**0.349**} & \multirow{2}{*}{22.906} & \multirow{2}{*}{1.489} & \multirow{2}{*}{22.942} & \multirow{2}{*}{2.413} & \multirow{2}{*}{22.793} & \multirow{2}{*}{1.551} \\ FD004 & & & & & & & & \\ \hline
**Variant** & \multicolumn{2}{c}{**MSD-SHF-SSF**} & \multicolumn{2}{c}{**MSD-MHF-SSF**} & \multicolumn{2}{c}{**MSD-SHF-MSF**} & \multicolumn{2}{c}{**SSD-MHF-MSF**} \\ \cline{2-9}
**Model** & \multicolumn{2}{c}{**Mean**} & \multicolumn{2}{c}{**STD**} & \multicolumn{2}{c}{**Mean**} & \multicolumn{2}{c}{**STD**} & \multicolumn{2}{c}{**Mean**} & \multicolumn{2}{c}{**STD**} \\ \hline RMSE & \multirow{2}{*}{17.530} & \multirow{2}{*}{0.448} & \multirow{2}{*}{17.352} & \multirow{2}{*}{0.315} & \multirow{2}{*}{17.299} & \multirow{2}{*}{0.246} & \multirow{2}{*}{19.508} & \multirow{2}{*}{1.653} \\ FD002 & & & & & & & & \\ Score & \multirow{2}{*}{1635.987} & \multirow{2}{*}{260.051} & \multirow{2}{*}{1635.596} & \multirow{2}{*}{195.693} & \multirow{2}{*}{**1310.653**} & \multirow{2}{*}{102.014} & \multirow{2}{*}{3002.581} & \multirow{2}{*}{1587.702} \\ FD002 & & & & & & & & \\ \hline RMSE & \multirow{2}{*}{20.189} & \multirow{2}{*}{0.481} & \multirow{2}{*}{20.563} & \multirow{2}{*}{0.611} & \multirow{2}{*}{20.091} & \multirow{2}{*}{0.410} & \multirow{2}{*}{22.747} & \multirow{2}{*}{2.363} \\ FD004 & & & & & & & & \\ Score & \multirow{2}{*}{3696.677} & \multirow{2}{*}{764.451} & \multirow{2}{*}{4535.615} & \multirow{2}{*}{1475.615} & \multirow{2}{*}{2955.290} & \multirow{2}{*}{747.684} & \multirow{2}{*}{6736.500} & \multirow{2}{*}{3618.271} \\ FD004 & & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 12: The influence of the corresponding neural network established by adopting the multi-head structure on the prediction performance. Figure 10: Effects of the multi-head structure’s number of heads on RUL prediction: (**a**) DMSAM_Data; (**b**) MHFCN; and (**c**) DMSAM_Features.