# Predicting Road Traffic Collisions Using a Two-Layer Ensemble Machine Learning Algorithm
## 2 Materials and Methods
### Building the Two-Layer Ensemble Model
The following formula illustrates the k-NN approach:  
\[d(x,y)=\sqrt{\sum_{i=1}^{n}\left(yi-xi)^{2}\right)} \tag{2}\]  
where \(x,y,\) are the two points in n-space, \(n\) is the number of input samples, and \(yi,\ xi\) are the distance vectors starting from the original point. (iii) Decision Trees (DT): This methodology is a nonparametric supervised learning method for classification and regression. The goal is to build a model that predicts the target variable's value by learning simple decision rules based on data attributes [29]. This is shown by the mathematical formula below:  
\[Entropy:\sum i=1-p*log_{2}(pi) \tag{3}\]  
\[Entropy(S)=-p+log_{2}p+-p-log_{2}p- \tag{4}\]  
Given that \(S\) is the sample of training examples and \(p+\) is the proportion of the positive training examples, while \(p-\) is the proportion of the negative training examples. DT has an overfitting problem, and to overcome it, we used a pruning technique to remove splits with little information gained (DT). This simplifies the DT by reducing the time cost of training and testing; it also eliminates the problem of overfitting [30]. In our study, increasing the tree depth in the early stages resulted in a corresponding improved performance of the training dataset and reduced performance of the test dataset. As the tree depth grows, a corresponding improvement is noted on both the training and test datasets up to the depth of 4. Depth 5 reveals that the model overfits the training dataset at the expense of the test dataset. as shown in Figure 5. In our study, we set the maximum tree depth at 4. (iv) Adaptive Boosting (AdaBoost): AdaBoost is a classification method that repeatedly calls a given weak learner algorithm over a number of rounds. In the training dataset, each instance is weighed, and overall errors are calculated. More weight is given when it is difficult to predict, and less weight is given when it is simple to predict [31; 32]. The AdaBoost approach has a weight that is represented as a vector for each weak learner. The input samples are illustrated in the following equation:  
\[Weight,w_{i}=\frac{1}{n} \tag{5}\]  
where \(w_{i}\) is the \(i\)th training instance weight and \(n\) is the number of training instances. (v) Stacking ensemble method: Stacking is a method of integrating predictions from various machine learning models into the same dataset, such as bagging and boosting [33]. Figure 5: Line plot illustrating DT accuracy on training and test datasets at different tree depths. The stacking technique's architecture consists of two or more models, known as base models or level-0, and meta-models that combine the predictions of the base models, known as level-1 models [34]. For our study, stacking was selected because the employed models are often distinct and fit the same dataset. Then, a single model was trained to integrate the outputs of the base as best as possible [35]. In our study, we implemented logistic regression as a meta-model to provide a seamless interpretation of the base models' predictions.