# The Quest for the Best Explanation: Comparing Models and XAI Methods in Air Quality Modeling Tasks
## 3 Results
### Station-Level Results with Feature Engineering and SHAP  
To better understand the station-specific effects of SHAP-based feature reduction after applying lag-based feature engineering, we report the R\({}^{2}\) values before and after the dimensionality reduction for each monitoring station, along with the number of features removed. The lag-related features numbered 55, as there were 11 lags for a chosen set of 5 features (PM\({}_{10}\), TEMP, PRES, DEWP, and WSPM). Despite the expanded feature space due to lagged variables, SHAP was able to consistently reduce the number of input features while maintaining or slightly improving performance in most stations in terms of R\({}^{2}\) and managing to reduce the RMSE (root mean square error) in the majority of stations (Table 2). Overall, the SHAP-based dimensionality reduction technique we applied was quite effective across all locations, with several stations (such as Dingling, Guanyuan, and \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Split Type** & **Initial Mean R\({}^{2}\)** & **Mean R\({}^{2}\)** & **Mean Change (\%)** & **Avg Features Before** & **Avg Features After** & **Feature Reduction (\%)** \\ \hline Train \textless{} 2017/Test \textgreater{} 2017 & 0.935 & 0.934 & \(-\)0.08\% & 27 & 22.7 & 15.93\% \\ Random 80/20 Stratified Split & 0.946 & 0.942 & \(-\)0.35\% & 27 & 22.7 & 15.93\% \\ Lagged Features on 80/20 Split & 0.963 & 0.963 & +0.06\% & 84 & 64.3 & 23.45\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Overall evaluation metrics and selected features of dimensionality reduction. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Station** & **All Features R\({}^{2}\)** & **SHAP Reduced R\({}^{2}\)** & **Features Removed (Initial 84)** & **RMSE (\%) Change** \\ \hline Aotizhongxin & 0.957 & 0.956 & 22 & +0.6\% \\ Changping & 0.956 & 0.955 & 19 & +0.75\% \\ Dingling & 0.973 & 0.974 & 23 & \(-\)1.83\% \\ Dongsi & 0.964 & 0.966 & 20 & \(-\)2.36\% \\ Guanyuan & 0.965 & 0.968 & 20 & \(-\)4.66\% \\ Gucheng & 0.951 & 0.953 & 21 & \(-\)2.2\% \\ Huairou & 0.959 & 0.960 & 20 & \(-\)1.83\% \\ Nongzhanguan & 0.966 & 0.967 & 22 & \(-\)0.84\% \\ Shunyi & 0.965 & 0.965 & 22 & +0.95\% \\ Tiantan & 0.965 & 0.965 & 20 & +0.52\% \\ Wanliu & 0.966 & 0.965 & 20 & \(-\)1.56\% \\ Wanshouxigong & 0.958 & 0.957 & 20 & +1.91\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Station-wise results for dimensionality reduction with respect to R\({}^{2}\), feature elimination, and RMSE. Wanliu) experiencing a slight improvement in predictive accuracy after simplifying the model. On average, around 22 features were removed per station, indicating that many of the lagged and original features were either redundant or had a minor impact on the model. These findings demonstrate the value of explainability-informed reduction strategies across diverse spatial monitoring contexts. The slight performance improvements observed in stations such as Guanyuan and Dingling highlight how feature simplification can support generalization. Figure 2 illustrates the SHAP dependence plots across all stations for the three most important features (PM\({}_{10}\), CO, and O\({}_{3}\)) as well as the most prominent interactions among them. These features were selected based on their highest [PERSON]'s correlation with the ground truth (PM\({}_{2.5}\)) values, highlighting their strong influence on the model's behavior. Each subplot displays the relationship between the raw feature value (x-axis) and its corresponding SHAP value (y-axis), which reflects the feature's contribution to the model's prediction for each instance. Positive SHAP values indicate that the feature drives the prediction toward higher PM\({}_{2.5}\) concentrations, while negative values suggest a lowering effect on the predicted PM\({}_{2.5}\) levels.