Collision severity prediction in Seattle City
Published: 2020-09-09 10:34:05

## Business understanding 
The main objective of the current data science project is to reduce the collision rate in a community. Specifically, this report will be targeted to drivers driving on bad road conditions along highways in Seattle, USA.

To achieve this goal, we will apply machine learning algorithms to predict the severity level of car accidents by training historical traffic collision data. In practice, the model receives real-time road condition data and alerts drivers with a predicted collision severity level so that the drivers can take precautions accordingly.

## Data understanding
The data consists of collision events taking place in Seattle from 2004/01/01 to 2020/05/20. Each row represents a collision event. In total there are 194673 events are recorded. Based on the definition of our problem, information that is needed is:
* road conditions when the collision happened
* data measuring the severity level of collision
As a result, the following attributes will be considered:
* The target field is 'SEVERITYCODE', as it is used to measure the severity of an accident. It has two possible values, as follows 1—prop damage, 2—injury.
* Attributes will be used to predict the severity level are 'WEATHER', 'ROADCOND', and 'LIGHTCOND', which are weather conditions, road conditions, and light conditions, respectively. These attributes are categorical, so we will convert these features to numerical values so that the machine learning models can handle it.

## Data cleaning
To work effectively with the data, it will be prepared in a way that addresses missing or invalid values and removes duplicates, toward ensuring that everything is properly formatted. We will mainly focus on 4 variables used for prediction, which are 'WEATHER', 'ROADCOND', 'LIGHTCOND' and 'SEVERITYCODE'. 

For road conditions, there are 10 distinct values, out of which 'nan' and 'Unknown' should be dropped out as they are missing values. Thus in total, there are 8 road conditions: 'Wet' 'Dry' 'Snow/Slush' 'Ice' 'Other' 'Sand/Mud/Dirt' 'Standing Water' 'Oil'
For weather conditions, there are 12 distinct values, out of which 'nan' and 'Unknown' should be dropped for the same reason. Thus in total, there are 10 weather conditions: 'Overcast' 'Raining' 'Clear' 'Other' 'Snowing' 'Fog/Smog/Smoke' 'Sleet/Hail/Freezing Rain' 'Blowing Sand/Dirt' 'Severe Crosswind' 'Partly Cloudy'

For light condition, there are 10 distinct values, out of which 'nan' and 'Unknown' should be dropped are for the same reason. 'Dark - No Street Lights' and 'Dark - Street Lights Off' should be merged as they represent the same condition. Thus in total, there are 7 light conditions: 'Daylight' 'Dark - Street Lights On' 'Dusk' 'Dawn' 'Dark - Street Lights Off' 'Other' 'Dark - Unknown Lighting'.

## Exploratory Data Analysis
After cleaning the data, the frequency of each condition is examined to understand their distribution. As can be seen that dry, wet, ice, and snow are the most frequent factors. However, this doesn't mean these factors lead to the collision. As for the weather conditions, clear, ranging and overcast are the top three most frequent conditions. And the light conditions are distributed more evenly. It seems that the collision is mostly like happen in good conditions when the road is dry, weather clear, and during day time. This does make sense as the collisions tend to happen when the roads are busy. And no conclusion can be reached so far.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/lusuvs1tde0fkot9g4yz.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wpm0xv10dtu15lmy5yzt.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/b8ffq1rouw3p45v5o1vq.png)

### Data Balance
By comparing the frequency of severity levels, it’s found that the quantity of severity 1 events is nearly double that of severity 2. To make an accurate prediction, the dataset needs to be balanced. In practice, the severity 1 events are sampled with the amount that equals to severity 2, which is 55851.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/yttrlgmoi7o1xsy4bm1i.png)
After data is balanced, the distribution of each variable is also balanced in terms of severity levels. We are now ready for data modeling.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/8d9jg6vt6omkazu6dhbk.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9fye7rwjlykumwf6wqpb.png)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/3816oock78yy2wt1oou9.png)
## Methodology
The problem at hand is a classification problem and I will apply 3 algorithms to solve it, which are K nearest neighbor (KNN), Decision Tree, and Logistic Regression. KNN predicts the severity level by finding the most similar data points that close to it. A decision tree model gives us a layout of all possible outcomes so we can fully analyze the consequences of a decision. As there are only two severity levels within the provided dataset, our model will only predict one of those two classes. This binary feature makes it perfect to apply logistic regression algorithm.

## Data preprocessing 
To use scikit-learn library, we have to convert the Pandas data frame to a Numpy array.  First Declare the following variables:
* X as the Feature Matrix ('WEATHER', 'ROADCOND', 'LIGHTCOND')
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/q5gkzurel0r1lv6z7mfk.png)
* y as the response vector ('SEVERITYCODE')
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/z4e53cau3nlvwke8onjz.png)
Then convert categorical variables to numerical values by using the preprocessing function in the scikit-learn library.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mwrn81vcegzncmpq12y0.png)

### Data normalization
Data Standardization gives data to zero mean and unit variance, it is good practice, especially for algorithms such as KNN which is based on the distance of cases:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/au6p22kzkgp3ja7bca84.png)

###T rain/Test Spit

Then we will apply Train/Test Split to our datasets, it involves splitting the dataset into training and testing sets respectively, which are mutually exclusive. This will provide a more accurate evaluation of out-of-sample accuracy because the testing dataset is not part of the dataset that has been used to train the data. In this case, we will use 70% of data for training and the rest 30% for testing.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/cye9zwzqs1twc5rlt54d.png)

## Prediction

### K nearest neighbor (KNN)

K-Nearest Neighbors is an algorithm for supervised learning. Where the data is 'trained' with data points corresponding to their classification. Once a point is to be predicted, it takes into account the 'K' nearest points to it to determine it's classification. 
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/lz0vwjo5f1jxj6tr6i8y.png)
The right value for K is determined by repeating the training process with different k values and see which k is the best for your model. The best accuracy was with 0.5072961117245084 with k= 8.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/04qlbwwzsm1ywtj0okih.png)

### Decision Tree

We will first create an instance of the DecisionTreeClassifier called decisionTree.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/zs55ftqzhlybz764q5ho.png)
Inside of the classifier, specify criterion="entropy" so we can see the information gain of each node. Next, we will fit the data with the training feature matrix X_train and training response vector y_train.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/vpixa5y7oi508j1yuavt.png)
Let's make some predictions on the testing dataset and store it into a variable called y_test_predTree.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/162b1jubnrgl7dck5gru.png)
The optimum max_depth is determined by using the same method as that for best k value determination for the KNN model. The best accuracy was with 0.5185163080779446 with max_depth = 9.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/nu850mnf1muevhplqbby.png)

### Logistic Regression
Let's build our model using LogisticRegression from Scikit-learn package. This function implements logistic regression and can use different numerical optimizers to find parameters, we choose ‘liblinear’ for the current study.
Regularization is a technique used to solve the overfitting problem in machine learning models. C parameter indicates the inverse of regularization strength which must be a positive float. Let's first try with 0.01. Smaller values specify stronger regularization. Now let's fit our model with the train set:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/q11s5modjjo9j5zg9wr7.png)

## Result evaluation

We can use classification_report to get a more detailed overview of the prediction, where
Precision is a measure of the accuracy provided that a class label has been predicted. It is defined by: precision = TP / (TP + FP)
Recall is a true positive rate. It is defined as: Recall = TP / (TP + FN)
The F1 score is the harmonic average of the precision and recall, where an F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0. It is a good way to show that a classifier has a good value for both recall and precision.
And finally, we can tell the average accuracy for this classifier is the average of the F1-score for both labels. And it can be concluded that KNN provides the highest accuracy.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/k8m542r8air4csdz4gxq.png)

## Discussion

In our study, we apply K-Nearest Neighbor, Decision Tree, and Logistic Regression to solve the classification problem. Choosing different k, max depth and parameter C values helped to improve their accuracy to be the best possible. 
The K-Nearest Neighbor model performs pretty well in predicting both the severity level 1 and 2 events, as the recall is 54% and 48%.On the contrary, the Decision Tree and Logistic Regression gave bad recall for severity level 1 event, meaning only around 30% of the severity 1 event was successfully detected. However, the accuracy of the prediction is around 50% for all the models, meaning we can only use our prediction result as a reference.

## Conclusion

By using the historical collision data consists of weather, road, and light conditions and their associated severity level, we created machine learning models based on the classification algorithms to predict collision severity.
The application of a specific model is mainly based on purposes. If the stakeholder cares more about avoiding severity level 2 events, then the decision Tree or logistic regression model is a good choice. If both severity level 1 & 2 events are to be alerted, then the KNN model is the best choice.