# Multi-Head Self-Attention-Based Fully Convolutional Network for RUL Prediction of Turbofan Engines
## 6 Conclusions  
In this paper, a novel deep neural network, the multi-head self-attention-based fully convolutional network (MSA-FCN), is proposed to predict the RUL of turbofan engines. MSA-FCN not only focuses on the degradation correlation among various components within the engine but also considers the differences in importance of various sensor data points and degradation representations for RUL prediction. Based on these features, MSA-FCN can extract valuable and abundant degradation representations under multiple operating conditions and failure modes, thus achieving RUL prediction for the corresponding turbofan engines. The experimental results using the C-MAPSS dataset demonstrate that MSA-FCN possesses competitive advantages in terms of accuracy and timeliness for RUL prediction when compared with 11 existing deep neural networks.  
In the future, the authors will focus on a knowledge-based and data-driven RUL prediction method to achieve better prediction performance with fewer training samples while further enhancing the generalization capability and transferability of our RUL prediction method.  
**Author Contributions:** Methodology, Z.L.; writing--original draft, Z.L.; software, Z.L.; data curation, Z.L.; conceptualization, X.Z., A.X., M.G. and A.J.; writing--review and editing, X.Z., A.X., M.G. and A.J. All authors have read and agreed to the published version of this manuscript.  
**Funding:** This work was supported by the National Natural Science Foundation of China (Grant No. U22A047), the Basic Public Welfare Research Project of Zhejiang Province (Grant No. LGG22F030005), and the \"Ling-Yan\" Research and Development Project of Zhejiang Province of China (Grant No. 2023C01157).  
**Data Availability Statement:** The datasets are available from the corresponding author upon reasonable request.  
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Variant** & \multicolumn{2}{c}{**Two Heads**} & \multicolumn{2}{c}{**Three Heads**} & \multicolumn{2}{c}{**Five Heads**} & \multicolumn{2}{c}{**Six Heads**} \\ \cline{2-9}
**Model** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** \\ \hline RMSE (FD004) & 19.790 & 0.627 & **19.401** & 0.349 & 19.556 & **0.308** & 19.900 & 0.653 \\ Score (FD004) & 2428.334 & 443.077 & **2184.660** & **158.393** & 2473.780 & 405.598 & 2576.746 & 750.688 \\ \hline \hline \end{tabular}
\end{table}
Table 14: Effect of the number of heads in MHFCN on RUL prediction performance.  
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Variant** & \multicolumn{2}{c}{**2 Heads**} & \multicolumn{2}{c}{**6 Heads**} & \multicolumn{2}{c}{**12 Heads**} & \multicolumn{2}{c}{**18 Heads**} \\ \cline{2-9}
**Model** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** \\ \hline RMSE (FD004) & 20.409 & 0.443 & 19.847 & 0.388 & 19.931 & 0.445 & 19.784 & 0.579 \\ Score (FD004) & 4350.241 & 553.995 & 3207.318 & 762.989 & 2799.949 & 795.470 & 2432.146 & 546.375 \\ \hline \hline \multirow{2}{*}{**Variant Model**} & \multicolumn{2}{c}{**23 Heads**} & \multicolumn{2}{c}{**24 Heads**} & \multicolumn{2}{c}{**25 Heads**} & \multicolumn{2}{c}{**30 Heads**} \\ \cline{2-9}
**Model** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** & **Mean** & **STD** \\ \hline RMSE (FD004) & 19.707 & 0.478 & **19.401** & **0.349** & 19.683 & 0.478 & 19.794 & 0.367 \\ Score (FD004) & 2217.502 & 415.572 & **2184.660** & **158.393** & 2476.285 & 388.739 & 2459.290 & 306.164 \\ \hline \hline \end{tabular}
\end{table}
Table 15: Effect of the number of heads in DMSAM_Features on RUL prediction.