Distracted Driver Detection Using Machine and Deep Learning Techniques
By: Akhil Mahajan
Tags: Deep Learning, Robotics, Machine Learning, Automation, Artificial Intelligence

Introduction

Hello ML enthusiasts! We are a group of students from IIITD pursuing Masters in CSE. In this article, We have proposed a comparative analysis between Deep Leaning techniques and ML Algorithms to solve a computer vision Kaggle challenge.

Team Members:

Akanksha Shrimal, Akhil Mahajan , Prabal Jain

Driving a car is a complex task, and it requires complete attention. Distracted driving is any activity that takes away the driver’s attention from the road. Approximately 1.35 million people die each year as a result of road traffic crashes.

Road traffic crashes cost most countries 3% of their gross domestic product So, our aim/goal in this project is to detect if the car driver is driving safe or performing any activity that might result in an accident or any harm to others, by using various Machine Learning Models to classify the provided images into different categories of Distraction.

Dataset

The dataset used is State Farm Distracted Driver Detection taken from https://www.kaggle.com/c/state-farm-distracted-driver-detection/data.

The dataset has 22424 training images, 79727 testing images and has 10 classes. The images are coloured and are of size 640×480 pixels. The classes are labelled as follows:

c0 : safe driving

: safe driving c1 : texting — right

: texting — right c2 : talking on the phone — right

: talking on the phone — right c3 : texting — left

: texting — left c4 : talking on the phone — left

: talking on the phone — left c5 : operating the radio

: operating the radio c6 : drinking

: drinking c7 : reaching behind

: reaching behind c8 : hair and makeup

: hair and makeup c9: talking to a passenger

Dataset Visualization

Data Pre-processing

Images are resized to 64 × 64 × 3 using CV2 in order to improve the computing efficiency of the classifier.

Stratified splitting is used to split the dataset into 80:20 Training-Testing ratio. The training dataset is further split into 90:10 Training-Validation set.

Thus, the final training set has 16145 images; the final validation set has 1794 images and the final testing set has 4485 images.

Workflow

Detecting Distracted Driver — Flow Chart

Feature Extraction

Following feature extraction techniques were used for extracting features from the images:

HOG — The histogram of oriented gradients

(HOG) is a feature descriptor used in computer vision and image processing for the purpose of object detection. The technique counts occurrences of gradient orientation in localized portions of an image.

LBP — Local Binary Pattern (LBP)

LBP is a simple yet very efficient texture operator which labels the pixels of an image by thresholding the neighbourhood of each pixel and considers the result as a binary number.

SURF

The SURF method (Speeded Up Robust Features) is a fast and robust algorithm for local, similarity invariant representation and comparison of images. The main interest of the SURF approach lies in its fast computation of operators using box filters, thus enabling real-time applications such as tracking and object recognition.

Color Histogram

A color histogram is a representation of the distribution of colors in an image. For digital images, a color histogram represents the number of pixels that have colors in each of a fixed list of color ranges, that span the image’s color space.

KAZE

KAZE is a 2D feature detection and description method that operates completely in nonlinear scale space.

Features extracted using Feature Extraction Techniques

Normalization: Min-Max Normalization

In this technique of data normalization, a linear transformation is performed on the original data. Minimum and maximum value from data is fetched and each value is replaced according to the following formula:

where x is the original value and x’ is the normalized value.

Dimensionality Reduction

We have used three dimensionality reduction techniques which are stated below:

PCA

Principal component analysis (PCA) is a technique for reducing the dimensionality of datasets, increasing interpretability but at the same time minimizing information loss.

After applying feature extraction techniques, PCA with n_components = 100 components is applied to features extracted from each extraction technique individually and then later combined together to form a combined feature set of 700 features.

LDA

Linear discriminant analysis (LDA) is a generalization of Fisher’s linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events.

LDA over PCA

LDA is applied on combined features which are obtained after applying PCA to further reduce the features, get a better class separation and to increase computational efficiency.

Models

Following Models were used for classification of our dataset:

Traditional ML Models

Decision Tree

Decision Tree is an important non-parametric method for image mining, This method uses several simple decision rules of the image to provide a path for image classification.

Support Vector Machine

SVM’s are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis.

KNN

The KNN algorithm assumes that similar things exist in close proximity. In other words, similar things are near to each other KNN captures the idea of similarity (sometimes called distance, proximity, or closeness)

Ensembling Methods

XGBoost

XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework. In prediction problems involving unstructured data (images, text, etc.) artificial neural networks tend to outperform all other algorithms or frameworks.

Bagging

Bootstrap aggregating also called bagging (from bootstrap aggregating), is a machine learning ensemble meta-algorithm designed to improve the stability and accuracy of machine learning algorithms used in statistical classification and regression. It also reduces variance and helps to avoid overfitting.

ADABoost

AdaBoost algorithm, short for Adaptive Boosting, is a Boosting technique that is used as an Ensemble Method in Machine Learning. It is called Adaptive Boosting as the weights are re-assigned to each instance, with higher weights to incorrectly classified instances.

Deep Learning Techniques

CNN

Convolutional Neural Network (CNN) has proven to be very effective in areas such as image recognition and classification. The advantage of using CNN is that it can automatically learn a complex feature utilizing massive simple neurons and backpropagation.

CNN architecture that was implemented for classification is shown below:

CNN Architecture

Transfer Learning: ResNet-101

Transfer learning is a technique in deep learning where one model that is trained on a task is repurposed to fit another task. ResNet-101 was applied with 2 strategies:

Strategy-1(S1)

Retrain only the last classifier layer of the pre-trained model and freeze all other parameters.

Strategy-2(S2)

Retrain the last few layers of the model including the classifier layer.

Result and Analysis

Grid Search

Optimal parameters obtained after hyperparameter tuning using GridSearchCV were :

Hyper-parameters obtained after tuning

ROC curves

ROC curves for each model after applying PCA

ROC curves for each model after applying LDA

ROC curves for each model after applying LDA over PCA

Classification Report

After applying PCA

After applying LDA

After applying LDA over PCA

Plots of Training and Validation Loss

CNN

ResNet-101: Stratergy 1 (left) and Stratergy 2 (right)

Confusion Matrix

CNN

ResNet-101: Stratergy 1 (left) and Stratergy 2 (right)

CAM

Class activation maps (CAMs) is a way to highlight the regions within an image that a CNN uses to make a classification decision for that particular image.

Class Activation Map for ResNet-101 — S2

Conclusion

PCA gave better results than LDA and LDA over PCA. Combination of features from various feature extractions gave better accuracy than taking individual features. Adaboost performed poorly for classification on this dataset as boosting techniques usually perform better on high variance datasets which is contrary to our data set which has low variance. After plotting the ROC curves for various traditional models, the best AUC-ROC score was found for SVM. ResNet-101 S2 performed much better than ResNet-101 S1 which is also higher than the accuracy obtained using CNN. Data Augmentation and Data Extraction techniques helped the classifier to perform better.

Blog Authors and Their Contributions:

Akhil Mahajan: Extracted features using HOG. Combined features and applied feature reduction techniques like PCA, LDA and LDA over PCA. Found optimized parameters for Decision Tree and XGBoost. Plotted ROC curves. Appendix

Akanksha Shrimal: Extracted features using SURF and LDB. Found optimized parameters for AdaBoost and SVM. Implemented ResNet-101, plotted loss vs epoch and confusion matrices. Implemented CAM and CNN. Presentations.

Prabal Jain: Extracted features using KAZE and Color Histogram. Found optimized parameters for BaggingClassifier and KNN. Implemented CNN and plotted loss vs epoch for it. Made presentations.

Combined work: Literature Survey, Abstract, Inference, Report, Data Analysis, References, Blog Writing

Under the guidance of

Professor: linkedin.com/in/tanmoy-chakraborty-89553324 Prof. Website: faculty.iiitd.ac.in/~tanmoy/ Teaching Fellow: Ms Ishita Bajaj Teaching Assistants: Shiv Kumar Gehlot, Vivek Reddy, Pragya Srivastava, Chhavi Jain, Shikha Singh and Nirav Diwan.

References:

[1] : http://cs229.stanford.edu/proj2019spr/report/24.pdf

[2]:https://github.com/Raj1036/ML_Distracted_Driver_Detection/blob/master/CS539_DistractedDriverDetetion_FinalProject.pdf

[3] : H. Eraqi, Y. Abouelnaga, M. Saad, and M. Moustafa, “Driver Distraction Identification with an Ensemble of Convolutional Neural Networks”, Journal of Advanced Transportation 2019, 1–12; doi: 10.1155/2019/4125865

[4] : https://snappishproductions.com/blog/2018/01/03/class-activation-mapping-in-pytorch.html.html

[5] : https://medium.com/@sam.bell_43711/distracted-driver-detection- using-deep-learning-ecc7216ae8d0

[6] : https://kharshit.github.io/blog/2018/09/07/skip-connections-and-residual-blocks

[7] : http://cs229.stanford.edu/proj2016/report/SamCenLuo-ClassificationOfDriverDistraction-report.pdf