# PEMFC RUL Prediction for Non-Stationary Time Series Based on Crossformer Model
## 4 Results and Discussion
### Results
#### 4.2.2 Network Model Ablation Studies  
To validate the effectiveness of various components within the model architecture, an experiment was conducted that included four ablation models: Crossformer + features + MSE (Designed_1), Crossformer + features + DAIN + MSE (Designed_2), Crossformer + features + DAIN + De-stationary + MSE (Designed_3), and Crossformer + features + DAIN + De-stationary + TILDE-Q (Designed_4). The bolded results indicate the best performance. In the experiments, the input time series step length was fixed at 48, and ablation experiments were conducted with six different prediction step lengths in the range of 8 to 64. The Adam optimizer was used, with 100 training epochs, and early stopping was applied with a patience of 5. Table 3 presents the results of the ablation studies. According to the analysis of the MSE metric, the Designed_2 model demonstrated a significant improvement over the Designed_1 model at longer time steps (32-64), enhancing performance by 3.31%. This improvement significantly surpasses the performance change observed at shorter time steps (8-24), which was only 0.04%. This indicates that DAIN has a more pronounced positive impact on prediction tasks at longer time steps. DAIN helps to mitigate internal covariate shift, enhancing the model's generalization capability, and thereby provides more effective performance gains for tasks involving long-term dependencies. Following the introduction of the non-stationary attention mechanism and the TILDE-Q loss function, the MSE improvements for Designed_3 and Designed_4, compared to \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Models** & **LSTnet** & **TCN** & **LSTM-ATTN** & **Transformer** & **Crossformer** & **De-Stationary-Crossformer** \\ \hline
**MSE** & 3.598 \(\pm\) 1.621 & 2.401 \(\pm\) 1.019 & 2.328 \(\pm\) 1.470 & 2.425 \(\pm\) 0.845 & 1.156 \(\pm\) 0.357 & **1.004 \(\pm\) 0.209** \\
**DTW** & 3.329 \(\pm\) 2.233 & 2.950 \(\pm\) 1.750 & 2.592 \(\pm\) 1.337 & 2.138 \(\pm\) 0.767 & 2.020 \(\pm\) 0.600 & **1.602 \(\pm\) 0.367** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Prediction Results on fuel cell vehicle datasets. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Models** & **Designed\_1** & **Designed\_2** & **Designed\_3** & **Designed\_4** \\
**Steps** & **MSE** & **DTW** & **MSE** & **DTW** & **MSE** & **DTW** & **MSE** & **DTW** \\ \hline
**8** & 0.725 & 0.373 & 0.710 & 0.364 & 0.625 & 0.337 & **0.556** & **0.324** \\
**16** & 0.809 & 0.597 & 0.821 & 0.585 & 0.740 & 0.583 & **0.717** & **0.551** \\
**24** & 0.888 & 0.783 & 0.892 & 0.807 & 0.812 & 0.759 & **0.778** & **0.743** \\
**32** & 1.003 & 0.928 & 0.884 & 0.912 & 0.820 & 0.885 & **0.791** & **0.862** \\
**48** & 1.022 & 1.230 & 1.053 & 1.222 & 0.952 & 1.186 & **0.886** & **1.136** \\
**64** & 1.058 & 1.425 & 1.044 & 1.411 & 0.964 & 1.371 & **0.795** & **1.237** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of ablation study results at different time steps. Designed_2 and Designed_3, were 9.09% and 7.94%, respectively. These results suggest that incorporating these two structural components significantly enhances the model's ability to extract non-stationary information from data and to evaluate shape similarity, thereby reducing the error between predicted and actual values. This contributes to an improvement in the overall performance of the model, particularly when dealing with time series data that exhibit shape variability and non-stationarity, offering substantial practical value. According to the analysis of the DTW metric, the introduction of the DAIN layer in Designed_2 resulted in a relatively minor improvement in overall performance, with only a 0.66% increase compared to Designed_1. However, the introduction of the non-stationary attention mechanism played a positive role in the prediction of PEMFC RUL. This mechanism enables the model to more effectively capture non-stationary features within the time series, making it more flexible and accurate in RUL prediction tasks.