# An Intelligent Real-Time Driver Activity Recognition System Using Spatio-Temporal Features
## 1 Introduction
### Related Works
[28] designed the Drive-Net architecture, which combines a CNN and a random decision forest to classify ten various driver distraction behaviors. In another study, [PERSON] et al. [29] proposed an ensemble of various CNNs to classify ten distracted driver postures. The classifier model was trained using raw images as well as face and hand images to make the classification accuracy robust. Furthermore, [PERSON] et al. [30] built a mock-up car environment equipped with front and side cameras to obtain multi-stream video data. This paper introduced an interwoven convolutional neural network (InterCNN) to detect different behaviors, including the use of a mobile device, eating, and drinking. [PERSON] et al. [31] adopted VGG-16 [32] architecture to detect nine different manual distraction activities and safe driving. In this experiment, the VGG-16 architecture was modified by employing various regularization methods such as dropout and ridge regression (L2) regularization while maintaining the overall classification accuracy. Furthermore, [PERSON] et al. [33] designed a simulated driving testbed and utilized four different deep CNNs, including VGG-16 [32], AlexNet [34], GoogLeNet [35], and a residual network. Ten various driving activity datasets collected from the side view of the driver were considered, including normal and distracted driving postures. The authors reported that the GoogleNet approach achieved better detection performance compared to the other DCNNs implemented while testing on the designed driving simulator. Lastly, [PERSON] et al. [17] investigated the impact of pre-trained architecture, namely AlexNet [34], GoogLeNet [35], and ResNet50 [36], to detect seven driver activities. A Kinect camera mounted on the front side of the driver was used to collect the raw RGB images of the driver activities. Then, prior to feeding the images to the classifier network, the raw images were first segmented using the GMM to extract the driver region from the background. The experiment reveals that applying the described segmentation system can significantly enhance the recognition accuracy. Despite the substantial progress that has been achieved in the detection of driver behavior studies, there are still several problems that need to be addressed. In most existing works, particularly computer vision-based driver behavior studies have widely applied visible RGB images that limit the recognition model to only broad daylight or daytime. Thus, recognizing drivers' behavior during nighttime conditions remains one of the main issues in the current computer vision-based driver activity recognition studies. Furthermore, previous studies utilized static input image information and employed various CNN variants along with significant traditional preprocessing measures such as background subtraction and image segmentation. These techniques can be time-consuming and increase computational costs. On the other hand, it is worth noting that some of the studies focused on limited types of driver distraction actions. Thus, our proposed work addresses the problem discussed.