# STFEformer: Spatial-Temporal Fusion Embedding Transformer for Traffic Flow Prediction
## 4 Experimental Results and Discussion
### Ablation Study Results  
To assess the efficacy of each component in STFEformer, ablation experiments with five variants of our model were conducted:  
* w/o MaskSSA. This removes the mask spatial self-attention. * w/o mask. This removes the binary mask matrix. * w/o \(E_{sp}\). This removes the adaptive spatial embedding, \(E_{sp}\). * w/o \(E_{f}\). This removes the native feature embedding, \(E_{f}\). * w/o \(E_{l}\). This removes the temporal embedding, \(E_{l}\). The experiments were carried out on the PeMS04, PeMS07, and PeMS08 datasets. The results are as shown in Tables 5-8 and Figures 2-7 illustrating the visualization of the experimental results. The outcomes of the ablation experiments provide a comprehensive perspective on the performance of STFEformer following the removal of different components. Overall, considering 12-time-step prediction, STFEformer performed the best, while w/o \(E_{t}\) ranked the lowest. Compared to the STFEformer model, the performance sequentially decreased for w/o mask, w/o \(E_{f}\), w/o MaskSSA, and w/o \(E_{sp}\). Compared to w/o \(E_{t}\), STFEformer demonstrated a significant improvement in next overall 12-time-step prediction and showed enhancements in MAE, RMSE, and MAPE. For example, STFEformer improved by 13.2%, 11.6%, and 8.0% on the PeMS04 dataset; on the PeMS07 dataset, the improvements were 11.3%, 10.4%, and 7.4%; and on the PeMS08 dataset, the improvements were 7.2%, 8.8%, and 5.7%. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline  & **3** & \multicolumn{4}{c}{**6**} & \multicolumn{4}{c}{**9**} & \multicolumn{4}{c}{**12**} \\
**Model** & **MAE** & **RMSE** & **MAPE** & **MAE** & **RMSE** & **MAPE** & **MAE** & **RMSE** & **MAPE** & **RMSE** & **MAPE** \\ \hline w/o MaskSSA & 19.82 & 31.62 & 12.64\% & 20.66 & 33.13 & 13.31\% & 21.42 & 34.33 & 13.89\% & 22.40 & 35.71 & 14.54\% \\ w/o mask & 18.28 & 29.55 & 12.20\% & 19.27 & 31.08 & 12.89\% & 19.97 & 32.33 & 13.41\% & 20.96 & 33.60 & 14.04\% \\ w/o \(E_{sp}\) & 19.33 & 31.06 & 12.78\% & 21.28 & 33.76 & 13.51\% & 22.15 & 36.25 & 14.84\% & 24.28 & 38.44 & 15.97\% \\ w/o \(E_{f}\) & 19.17 & 30.83 & 12.47\% & 20.03 & 32.36 & 13.05\% & 21.07 & 33.69 & 13.68\% & 21.41 & 34.32 & 14.26\% \\ w/o \(E_{t}\) & 19.92 & 31.85 & 12.71\% & 21.48 & 34.18 & 13.66\% & 22.64 & 36.81 & 15.18\% & 24.72 & 38.67 & 16.15\% \\ STFEformer & 18.00 & 29.30 & 11.95\% & 18.98 & 30.96 & 12.63\% & 19.62 & 32.02 & 13.07\% & 20.61 & 33.36 & 13.69\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The next 3-, 6-, 9-, and 12-time-step prediction results of the ablation study on PeMS04. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & **3** & \multicolumn{4}{c}{**6**} & \multicolumn{4}{c}{**9**} & \multicolumn{4}{c}{**12**} \\
**Model** & **MAE** & **RMSE** & **MAPE** & **MAE** & **RMSE** & **MAPE** & **MAE** & **RMSE** & **MAPE** & **MAE** & **RMSE** & **MAPE** \\ \hline w/o MaskSSA & 21.02 & 33.39 & 8.62\% & 22.17 & 35.71 & 9.09\% & 23.14 & 37.41 & 9.54\% & 24.34 & 39.18 & 10.16\% \\ w/o mask & 19.55 & 31.37 & 8.42\% & 20.68 & 33.66 & 8.85\% & 21.58 & 35.28 & 9.23\% & 22.89 & 37.15 & 9.95\% \\ w/o \(E_{sp}\) & 20.39 & 33.06 & 8.80\% & 22.61 & 36.04 & 9.28\% & 25.18 & 39.96 & 10.03\% & 27.47 & 42.47 & 11.00\% \\ w/o \(E_{f}\) & 20.16 & 32.42 & 8.61\% & 21.46 & 35.47 & 8.95\% & 22.27 & 36.49 & 9.29\% & 23.54 & 38.16 & 10.00\% \\ w/o \(E_{t}\) & 21.00 & 33.74 & 8.75\% & 22.66 & 37.12 & 9.33\% & 25.49 & 40.86 & 10.24\% & 27.76 & 43.36 & 11.13\% \\ STFEformer & 19.29 & 30.98 & 8.21\% & 20.40 & 33.17 & 8.63\% & 21.37 & 34.85 & 9.08\% & 22.67 & 36.68 & 9.84\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The next 3-, 6-, 9-, and 12-time-step prediction results of the ablation study on PeMS07.