## 5 Factor Identification and Predictive Models  
Predictive analytics is defined as finding patterns in data and predicting values in future trends and outcomes by using those patterns. Moreover, machine learning techniques have become more popular in accomplishing predictive analytics due to their exceptional usage in managing large scale data in recent years. In addition to the EDA, in this study, we built several machine learning models to rank the important factors associated with the severity of crashes and performed crash severity prediction using these identified factors from the EDA. The machine learning process was implemented using Python and the related Python data learning libraries, such as Pandas and SKlearn. The detailed steps are described as follows:  
Step 1: W Cleaned and filter raw crash data from the CRIS. \"No data\" and \"Unknown data\" of crash severity are filtered out of the dataset. Choose the features based on the previous EDA and remove the features with the large percentile of missing data. In the end, we obtained 8859 valid crash records during the period of year 2010 to 2018. Each crash record had 46 features about the characteristics of crash, teen driver, vehicle, road, and driving environment. Step 2: Since most of the features in the crash dataset are categorical, encode categorical data to numerical data for further construction of predictive models. Use two encoders that are available in the Python Sklearn library: label encoder and one hot encoder. Label encoder assigns a non-repeat numerical value to each category in the data type of object. However, one weakness of the label encoder is that, by assigning different values to different categories, it gives an order to all categories, even though all categories are weighted equally and have no relationships between each category. One hot encoder splits one categorical feature into a binary sparse matrix. For example, in the categorical feature \"Day of the week\", there are seven different values representing each day of the week. One hot encoder splits the feature into a binary matrix with seven columns, where 1 represents true and 0 represents false for the value in each column. The differences between these two coding methods are illustrated in Figure 6.  
Figure 5: Other exploratory analysis on teen drivers and crash severity. After encoding the data, all data should be numerical in all cases, but the values of some continuous features are in large numbers by nature (such as the average daily traffic), which means it may have a higher impact on prediction than other features. Standardize all continuous features so that all variability is measured at the same scale. We used the Standardscaler function in the SKlearn library to normalize all the features in the dataset. The standard scalar examples are also shown in Figure 6.  
Step 3: With the encoded and scaled data, start building predictive models for crash severity prediction. Here, approach the prediction as a multivariate regression problem and choose Random Forest (RF) and XGBoost as predictive methods. We also used the Python SKlearn library to implement the selected algorithms. Random Forest is an ensemble machine learning algorithm that is used for classification and regression problems. It employs the decision tree algorithm for parametrization, but it mixes a sampling procedure, a subspace technique, and an ensemble tactic to optimize the model building. Bootstrap is the name of the sampling method, which uses a random sampling tactic with replacement. The subspace technique also takes a random sampling method, but it assists in removing smaller subsets (i.e., subspaces) of variables. The overfitting problem in decision trees is corrected in random decision forests by providing manifold. Use trained decision tree algorithms for the testing stage [21]. This property makes the RF preferable over the regular decision trees algorithm, since it was the most popular machine learning algorithm amongst data scientist until XGBoost took over.