# Improving PM2.5 Concentration Forecast with the Identification of Temperature Inversion
## 4 Experimental Results and Comparative Performances
### Pm\({}_{2.5}\) Concentration Estimation  
After training the classifiers, namely, CART and CNN, each test day could be firstly labeled as the one with or without temperature inversion. Secondly, the hourly PM\({}_{2.5}\) concentration of the test day was estimated by the regression with the training set according to the test day's label. If the test day was in the temperature inversion class, the training set including all days with temperature inversion in 2016 was used. Otherwise, a short period of training days immediately prior to the test day was employed as the training set. To evaluate the accuracy of the PM\({}_{2.5}\) estimation, the performance measures, the rooted mean square error (RMSE) and the mean absolute error (MAE), which have been widely used in the literature, were adopted in this paper and they are defined as follows. \[\text{RMSE}=\sqrt{\sum_{t}(y(t)-\bar{y}(t))^{2}/\sum_{t}1}, \tag{7}\] \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **2017** & \multicolumn{3}{c}{**2018**} & \multicolumn{3}{c}{**2019**} \\ \hline  & **CART** & **CNN** & **CART** & **CNN** & **CART** & **CNN** \\ \hline
**TP** & 10 & 11 & 10 & 6 & 9 & 7 \\
**FP** & 0 & 2 & 2 & 1 & 0 & 1 \\
**TN** & 9 & 7 & 9 & 10 & 12 & 12 \\
**FN** & 5 & 4 & 3 & 7 & 3 & 4 \\ \hline
**Precision** & 1.00 & 0.85 & 0.83 & 0.86 & 1.00 & 0.88 \\
**Recall** & 0.67 & 0.73 & 0.77 & 0.46 & 0.75 & 0.64 \\
**Accuracy** & 0.79 & 0.75 & 0.79 & 0.67 & 0.88 & 0.79 \\
**F1-Score** & 0.80 & 0.79 & 0.80 & 0.60 & 0.86 & 0.74 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparative performances between CART and CNN for temperature inversion classification. \[\text{MAE}=\sum_{t}\lvert y(t)-\hat{y}(t)\rvert/\sum_{t}1, \tag{8}\]  
where \(t\) is the duration of test time in number of hours. Five different forecasting strategies were compared. As noted in Section 3.2, the days in 2016 were manually labeled by experts with two classes, the class with temperature inversion (referred to as _Temp_inv_) and the class without the phenomenon (referred to as _Normal_). We noted the results by an expert, which were then used to train CART and CNN, as described in Section 3.3. So we had three classification models (Expert, CART and CNN) which were able to classify a test day into a class (_Temp_inv_ or _Normal_). For comparison, we further built two dummy classifiers, AllTemp and AllNormal, which always label the test day as _Temp_inv_ or _Normal_, respectively. The PM\({}_{2.5}\) concentration forecasting was made by multivariate regression as described in Section 3.4. If the test day was classified as _Temp_inv_ by the deployed model, the regression trained on the days in _Temp_inv_ of 2016 was used to make the forecast. Otherwise, the regression trained on a short-term period of days preceding the test day was employed to predict the PM\({}_{2.5}\) concentration. To verify the best length of the short-term training dataset for forecasting the PM\({}_{2.5}\) of a test day recognized as _Normal_, we experimented with five, six, and seven days, respectively, to see which training length gives the best performance. The performance of the forecasting results with the test days in the three test years ranging from 2017 to 2019 is tabulated in Tables 2 to 4. Several implications can be drawn from the results. (1) It is noted that the experiments with various length of short-term training dataset only influenced Expert, CART, CNN, and AllNormal. The model AllTemp always refers to the long-term training dataset in 2016 and it will not be affected by varying short-term training length. It is seen that Expert and CART manifest the best performance on all cases when the length of short-term training dataset is set to seven days preceding the test day. Although CNN and AllNormal didn't always get better results with seven days on all cases, they had the best performance overall with seven-day training dataset. The exceptions were as follows.