# Improving PM2.5 Concentration Forecast with the Identification of Temperature Inversion
## 3 Proposed Methods
### Temperature Inversion Classification  
With the preliminary results of feature engineering, we selected the following features for designing the temperature inversion classification models: three categorical features (the labeling of hourly temperature, relative humidity, and wind speed, as high, middle, or low determined by k-means) and two numerical features (precipitation and PM\({}_{2.5}\) concentrations in the immediate previous day). Two classification models, CART and CNN, were chosen for comparative performance analyses. CART [35] is an advanced decision tree algorithm. It can be applied to accomplish classification (with categorical target value) or regression (with continuous target value) learning tasks. CART is widely used because it is provided in the Scikit-learn library. The classification by CART proceeds as follows. A binary tree is grown to a specified maximum depth. At each level of depth, the feature which minimizes the Gini impurity is chosen to branch the inner node with an appropriate threshold into dichotomies. The Gini impurity is an estimate of the probability that a randomly chosen sample is misclassified. After the classification tree is fully grown, a pruning step with cross-validation is applied to reduce  
Figure 4: The data labeling for two plausible examples of temperature inversion phenomena. Figure 5: The mean hourly PM\({}_{2.5}\) value observed at Puli EPA supersite for the days with or without the plausible meteorological phenomena causing temperature inversion. the tree complexity. Because the CART implementation provided by Scikit-learn currently does not support categorical variables, we converted our categorical features (temperature, relative humidity, and wind speed) to numerical ones by redefining them as the number of hours that each individual feature is labeled as each of the range classes (high, middle, and low). The previous day 24 h PM2.5 concentrations were replaced by their mean value to reduce the number of similar features. Figure 6 shows the learned decision tree from the collected data in 2016 where three days had missing values and were removed and the data of the remaining 363 days were used as the training data. It is seen that the most salient rule for labeling the days as temperature inversion is as follows. _No precipitation, the previous day mean hourly PM2.5 concentration is above 34, the number of low-wind-speed hours is less than 19, the number of high-temperature hours is greater than 4, and the number of high relative humidity hours is less than 14._  
The second most significant rule is as follows. _No precipitation, the previous day mean hourly PM2.5 concentration is between 18 and 34, and there is no high wind speed hours._  
The CNN [36] has emerged as one of the main research streams in deep learning. A CNN adds several convolution layers in front of traditional neural networks such as the fully-connected multilayer perceptron. The convolution layer applies local convolution operations with different kernels/filters such that the local features contained in the original input image can be automatically extracted. The salient responses of these local features are retained by pooling operations. By concatenating several convolution layers, the features of various details and resolutions are learned. These features are fed forward to the traditional fully-connected networks to learn the weights and bias for the classification. Due to the increasing computing capability of processors such as GPUs, large CNN architectures which are able to learn classification for millions of images have been proposed. Ever since AlexNet [37] won the first prize of ILSVRC 2012, the development of CNN-based applications has been overwhelming. AlexNet improves the error rate by 10 percent as compared to the champion in ILSVRC 2011. AlexNet consists of five convolution layers followed by a two-layer fully-connected neural network. More recently, sophisticated CNNs such as GoogLeNet [38], ResNet [39], and SENet [40], have prevailed the ILSVRC competitions.