# A Comparative Analysis for Air Quality Estimation from Traffic and Meteorological Data
## 2 Materials and Methods
### Data Preprocessing and Analysis
\begin{table}
\begin{tabular}{l r r r r} \hline \hline
**Feature** & **Total Missing** & **Percent.** & **Missing (agg.)** & **Percent.z (agg.)** \\ \hline \(BC\) & 7623/52558 & 14.50\% & 974/26279 & 3.70\% \\ \(C_{6}H_{6}\) & 12334/105116 & 11.73\% & 2/26279 & 0.01\% \\ CO & 2558/105116 & 2.43\% & 2/26279 & 0.01\% \\ \(NH_{3}\) & 3835/26279 & 14.59\% & 3835/26279 & 14.59\% \\ \(NO_{2}\) & 9959/210232 & 4.74\% & 2/26279 & 0.01\% \\ \(NO_{x}\) & 8675/183953 & 4.72\% & 17/26279 & 0.06\% \\ \(O_{3}\) & 4426/78837 & 5.61\% & 2/26279 & 0.01\% \\ \(SO_{2}\) & 1800/26279 & 6.85\% & 1800/26279 & 6.84\% \\ \(PM_{10}\) & 131/3288 & 3.98\% & 0/1096 & 0.00\% \\ \(PM_{2.5}\) & 95/2192 & 4.33\% & 0/1096 & 0.00\% \\ \hline humidity & 7329/131405 & 5.58\% & 0/26281 & 0.00\% \\ pressure & 171/26281 & 0.65\% & 171/26281 & 0.65\% \\ radiation & 26455/78843 & 33.55\% & 0/26281 & 0.00\% \\ rain & 42105/131405 & 32.04\% & 1/26281 & 0.01\% \\ temp & 7471/157686 & 4.74\% & 0/26281 & 0.00\% \\ wind dir. & 48433/131405 & 36.86\% & 6373/26281 & 24.25\% \\ wind speed & 35308/131405 & 26.87\% & 2/26281 & 0.01\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Total amount of missing values for weather and pollutant sensors. On the left, the count represents the amount w.r.t. total feature values, on the right, the result after mean aggregation among groups. \begin{table}
\begin{tabular}{l r r} \hline \hline
**From** & **To** & **Period Length** \\ \hline
2014-02-06 00:00:00 & 2014-02-06 23:00:00 & 23 h \\
2014-06-01 00:00:00 & 2014-06-01 23:00:00 & 23 h \\
2014-08-04 00:00:00 & 2014-08-17 23:00:00 & 13 d 23 h \\
2015-05-30 17:00:00 & 2015-05-30 23:00:00 & 6 h \\
20 the vehicle type. An analogous procedure was carried out on target variables. On the basis of the former analysis, we decided to exclude from the study both sulphur dioxide (\(SO_{2}\)) and ammonia (\(NH_{3}\)) measurements for three main reasons: first and foremost, both pollutants were represented by a single ground-truth sensor each which contained a large portion of missing data, as shown in Table 2. Second, the low correlation displayed by the two pollutants, visible in Figure 11, cannot justify a possible imputation phase over their respective time series, especially with an high percentage of data to be estimated. Third, our main objective is the estimation of an Air Quality Index where both categories are not among the required pollutants for its computation, thus not essential for our case study. The aggregation procedure reduces the amount of missing information, nevertheless many features still remain incomplete or even unchanged in the particular case of transits, where the spanned missing intervals were the same across every record, as shown in Figure 2 and Table 3. While this does not represent a problem in most cases, it is preferable to maintain a continuous time series, especially for experiments with sequence-based models such as deep Recurrent Neural Networks. For consistency, we therefore opted for a domain-based imputation phase, during which existing features from the same domain (namely weather, pollutants and transits) are exploited to estimate missing values in the others. This decision was once again motivated by the extreme trend similarities among sensors of the same category, as reflected by the correlation matrices displayed in Appendix A. In our work, we adopted an hybrid approach: considering the time series representing the evolution of a single feature and an empirically determined threshold of six hours, if a continuous sequence of missing values remains below the threshold, we simply impute by polynomial interpolation (with grade 2). Otherwise, we apply an iterative statistical imputation strategy, where the features with the least amount of missing values are estimated first, using all the others as input with a round-robin process. This allows for theoretically better results since at every step we maximise the amount of ground-truth information provided to the model.