# A Multimodal Recurrent Model for Driver Distraction Detection  
[PERSON]  
1 Research Group Networks and Mobility, University of Applied Sciences Upper Austria, Softwareparak 11, 4232 Hagenberg, Austria; [EMAIL_ADDRESS]  
1 Correspondence: [EMAIL_ADDRESS] or [EMAIL_ADDRESS]  
2020 April 16  
performance, such as interacting with the infotainment system, talking to passengers, and eating or drinking while driving. Depending on the nature of the distraction, these activities can have different effects on the driver's perception and significantly increase reaction times, thereby increasing the risk of a collision. Various strategies can be used to reduce the prevalence of distracted driving. In addition to enforcing stricter penalties and raising awareness through public campaigns and driver training, implementing countermeasures directly in vehicles is also an effective approach [2]. Modern vehicles are equipped with advanced driver assistance systems (ADASs) designed to assist the driver in performing the driving task. Systems such as emergency braking assistants and lane departure warnings significantly reduce the risk of road accidents. The current trend is even toward the development of partially or fully automated vehicles, where control of the vehicle is completely transferred to the system. However, until fully automated vehicles are on our roads, the human driver still acts as a supervisor of the system [6]. In partially automated systems, even when the vehicle is in control, the driver must remain alert and ready to intervene quickly in dangerous situations that the vehicle cannot handle on its own. However, the increased comfort of self-driving situations may tempt drivers to engage in distracting activities, which may impair their ability to intervene in a timely manner. A promising solution to reduce the prevalence of distraction on the road is the implementation of driver distraction detection systems [2]. These systems, which use technologies such as camera-based monitoring, can detect and warn drivers when they are distracted. By reducing the frequency of distracting activities, drivers are encouraged to remain focused on their driving tasks. Consequently, in partially automated vehicles, this approach keeps the driver actively involved in the driving task, enabling them to safely take control in critical situations. To mitigate the risk and reduce the prevalence of distracted driving, the implementation of driver distraction detection systems has emerged as a promising solution. These systems are designed to detect distractions and provide timely warnings to the driver. Various AI-based approaches have been explored in research and industry to detect driver distraction, including vision-based methods that analyze driver camera images, sensor-based techniques that use sensor data, and multimodal approaches that combine both image and sensor data [7]. Vision-based methods, such as those used in the State Farm Distracted Driver Detection competition, rely on camera images to classify distractions [8]. Different CNN architectures such as _DenseNet-201_[9], _ResNet-50_[10], and _InceptionV3_[11] have been tested and have achieved high levels of accuracy on datasets such as the State Farm dataset and the AUC dataset [12]. Hybrid models combining CNNs with recurrent neural networks (RNNs), especially LSTMs, have shown improved performance, especially when considering temporal dependencies [12; 13; 14]. Sensor-based methods most often use data from non-intrusive sensors such as IMUs and vehicle dynamics data to detect distractions. Studies have shown that LSTM networks, especially those enhanced with attention mechanisms, can effectively classify distractions based on sensor data with high accuracy [15]. Intrusive sensors, while accurate, are less practical for real-world applications due to their impact on the driver [6]. Multimodal approaches that combine visual and sensor data have been shown to improve detection accuracy.