# VDMS: An Improved Vision Transformer-Based Model for PM\({}_{2.5}\) Concentration Prediction
## 4 Results
### Model Estimation  
Figure 4 presents a density scatter plot evaluating the performance of the VDMS model in air quality prediction, comparing scenarios with and without the incorporation of meteorological variables. Two CV techniques were used: Sam-CV and Sta-CV. When meteorological factors were integrated, the model exhibited superior predictive accuracy. In the Sam-CV setting (Figure 4a), the R\({}^{2}\) reached 0.93, with an RMSE of 4.05 \(\upmu\)g/m\({}^{3}\) and a MAE of 3.23 \(\upmu\)g/m\({}^{3}\). In the Sta-CV setting (Figure 4b), the corresponding values were an R\({}^{2}\) of 0.88, an RMSE of 5.57 \(\upmu\)g/m\({}^{3}\), and a MAE of 4.21 \(\upmu\)g/m\({}^{3}\). In contrast, excluding meteorological factors resulted in decreased performance. Under the Sam-CV approach (Figure 4c), the R\({}^{2}\) declined to 0.73, with an RMSE of 5.79 \(\upmu\)g/m\({}^{3}\) and a MAE of 4.61 \(\upmu\)g/m\({}^{3}\), whereas the Sta-CV approach (Figure 4d) yielded an R\({}^{2}\) of 0.69, an RMSE of 7.96 \(\upmu\)g/m\({}^{3}\), and a MAE of 6.01 \(\upmu\)g/m\({}^{3}\). These results highlight the substantial improvement attributable to the incorporation of meteorological factors, particularly under the Sam-CV configuration where the R\({}^{2}\) increased by 27.4%, and the RMSE and MAE decreased by 30.1% and 29.9%, respectively. These findings demonstrate the critical role of meteorological data in refining the VDMS model's air quality predictions. We note that prediction errors rise for PM\({}_{2.5}\) above 40 \(\upmu\)g/m\({}^{3}\), largely because our training set under-represents high-concentration cases. With few examples under heavy-pollution conditions--such as temperature inversions, high humidity, and low wind--the model cannot fully learn the complex nonlinear relationships. Table 1 shows that, compared with CNN, SimCLR, and the ViT model, the VDMS model exhibits superior performance as evidenced by a lower RMSE and higher R\({}^{2}\). We also calculate the adjusted R\({}^{2}\) since the adjusted R\({}^{2}\) is very close to R\({}^{2}\), we use R\({}^{2}\) as the standard. Specifically, VDMS achieves an R\({}^{2}\) of 0.88, an RMSE of 4.05 \(\upmu\)g/m\({}^{3}\), and a MAE of 3.23 \(\upmu\)g/m\({}^{3}\). Table 2 further indicates that incorporating an LSTM module into the conventional ViT model increases the R\({}^{2}\), RMSE, and MAE to 0.58, 7.64 \(\upmu\)g/m\({}^{3}\), and 5.23 \(\upmu\)g/m\({}^{3}\), respectively. Although this incorporation provided some improvement, the enhancement was relatively limited. The subsequent introduction of a double-layered LSTM significantly improved performance, with R\({}^{2}\), RMSE, and MAE values rising to 0.73, 6.14 \(\upmu\)g/m\({}^{3}\), and 4.77 \(\upmu\)g/m\({}^{3}\), respectively, demonstrating a marked improvement over the single-layer LSTM. Finally, after the integration of a multi-head self-attention mechanism and a SimCLR module, the performance of the model was further optimized, with R\({}^{2}\), RMSE, and MAE improving from 0.85, 4.54 \(\upmu\)g/m\({}^{3}\), and 3.31 \(\upmu\)g/m\({}^{3}\) to 0.93, 4.05 \(\upmu\)g/m\({}^{3}\), and 3.23 \(\upmu\)g/m\({}^{3}\), respectively, achieving the best overall performance. These enhancements can be attributed to the improved integration and extraction of feature information, particularly through the inclusion of the multi-head self-attention mechanism and SimCLR module, which enable the more precise modeling of complex input features. The VDMS model demonstrates robust performance on the ACAG data, achieving metrics statistically consistent with those derived from our primary dataset. This cross-dataset validation confirms the model's predictive reliability across heterogeneous data sources.