# Machine Learning Approaches for Outdoor Air Quality Modelling: A Systematic Review
## 3 Results and Discussion
### General Description
Finally, Stacking is the method used to make a prediction from a battery of different possible algorithms. After applying these algorithms to the dataset, they are assessed by cross-validation, for instance, in order to give a weight to each method according to its performance. Random Forest is one of the most popular Ensemble Learning techniques and constitutes the majority of the papers classified under this label in our literature review. This method is based on an ensemble of Decision Tree predictors. The first step implements bagging on trees. Then, a second step adds stochasticity to split the tree by using the random subspace method or the random split selection, which consists of applying at each node the algorithm with a subset only of the features to split the node. Artificial Neural Networks (NN) are inspired by the working of the human being's nervous system. The simplest neural network architecture is called Perceptron. This method classifies inputs by using a linear combination of the observations of the dataset, as defined in Equation (10). Figure 4: Proportion of machine learning algorithms used. ’Ensemble’ stands for Ensemble Learning (mainly Random Forest). ’NN’ stands for any algorithm based on an Artificial Neural Networks approach. ’SVM’ stands for Support Vector Machine (it also includes Support Vector Regression). ’Regression’ stands for Multiple Linear Regression. ’Others’ includes Decision Rules and Lazy methods. \[x=\sum_{j=0}^{J}w_{j}\sigma_{j} \tag{10}\]  
where o is an observation characterized by several features (from 0 to J) and \(w_{j}\) is the weight applied to each feature. For instance, in the case of two classes, if x is greater than 0 the observation is classified in class 1, otherwise it is classified in class 2. In an NN, the weights are learned. The algorithm starts by setting all weights to 0. Then, a loop is applied until all observations in the training data are classified correctly. In this loop, for each observation o, if o is classified incorrectly and if it belongs to class 1, it is added to the weight vector, otherwise it is subtracted. No modification is applied if the observation is classified correctly. Nevertheless, the NN implemented in the selected papers consist of at least three types of layers (i.e., Multilayer Perceptron)--input layer, hidden layer, and output layer. The input is composed with the features (e.g., contaminant covariates, meteorology, etc.). The output is the variable to be predicted (i.e., concentration of a contaminant or value of an air quality index). And the hidden layer consists of additional nodes between the two previous layers that enables multiple kinds of connections. When the number of hidden layers is large the NN is called Deep Learning Neural Network. Each node (or neuron) performs a weighted sum of its input and thresholds the result. Instead of using the binary threshold of the basic Perceptron, the Multilayer Perceptron usually applies a smooth activation function based on a sigmoid. In such an implementation, as the inputs become more extreme, they approach the step function, which corresponds to the hard-edge threshold used in the basic Perceptron. Finally, the learning stage consists of getting the best weights to minimize the error between the predictive output and the actual output. This is an iterative process using a steepest descent method, in which the gradient is determined by a backpropagation algorithm that seeks to minimize the cost function \(J(\theta)\) defined in Equation (11). \[J(\theta)=\frac{1}{m}\ \sum_{i=1}^{m}\Bigl{(}h_{\theta}\Bigl{(}x^{(i)}\Bigr{)}- y^{(i)}\Bigr{)}^{2} \tag{11}\]  
where m is the number of observations, y is the actual output and h\({}_{\theta}\)(x) is the predicted output. Support Vector Machine (SVM) is another algorithm that draws a boundary through the widest channel between two classes, which is the maximum separation from each class.