# Predicting Road Traffic Collisions Using a Two-Layer Ensemble Machine Learning Algorithm
## 2 Materials and Methods
### Data Preprocessing  
The data with 15 features were loaded into the panda dataframe object to facilitate various preprocessing procedures. First, the data set was normalized using 15 features, after which missing values were discovered in some of the fields. Since the missing values would affect the performance of the model, we replaced the blank and null feature values by applying the mean value of the relevant feature column [18; 19]. The mean values that were used to fill the missing feature records presented no extreme values that could have affected the mean.  
Feature selection is a critical factor in obtaining an accurate prediction. Using all the features leads to an inefficient model because, as the number of features increases, models struggle for accuracy, and hence model performance is reduced [20]. In this study, we used Sklearn, a Python library, to select the features. To obtain the most important features for this study, we employed four algorithms: particle swarm optimization (PSO), univariate feature selection, recursive feature elimination, and feature importance.  
1. Particle swarm optimization (PSO) algorithm: This technique works by searching for the optimal subset of features. It locates the minimum of a function by creating several 'particles'. These particles store their best position, as well as the global position. It is this combination of local and global information that gives rise to'swarm intelligence' [21]. In our study, we implemented XGBoost and linear regression algorithms to select the best features.
2. Recursive feature elimination: This technique works by selecting the optimal subset of features for estimation by iteratively reducing 0 to N features [22]. The best subset is then chosen based on the model's accuracy, cross-validation score, or Roc-Auc curve.
3. Univariate feature selection: This approach works by selecting the optimal features using univariate statistical tests. It might be considered a stage in the estimator's preprocessing process [23]. In our study, we implemented the chi-squared statistical test using the SelectKBest method.
4. Feature importance: This works by classifying and evaluating each attribute to create splits. Decision tree models that are developed on ensembles; for example, extra trees and random forests can be used to rank the relevance of certain features [24]. In our study, we employed the extra trees classifier for feature selection.  
After performing the feature selection algorithms, we selected the top six features, as shown in Table 1, based on the selected features algorithms.  
Three techniques, univariate feature selection, recursive elimination method, and feature importance had the top six common features, while the PSO algorithm had four features in common with the other three techniques. For this study, we employed the PSO feature selection method because the performance of the model was not affected when evaluating the model using the features selected by the other three techniques.  
Figure 2: A participant driving on the simulated road scenario at Strathmore University.