# A Multimodal Recurrent Model for Driver Distraction Detection
## 5 Conclusions  
This study focused on the development of a recurrent multimodal approach for driver distraction detection and investigated whether classical vision-based methods using CNNs could be improved by incorporating LSTM networks and sensor data. A custom dataset was created with six drivers performing different distracting activities. A _CNN_ model using a pre-trained _VGG16_ served as the baseline. This study introduced a _CNN-LSTM_ model to capture temporal dependencies and a multimodal _CNN-LSTM-S_ model that combined image sequences with sensor data. Comparisons were made using a _CNN Sequential_ approach, where _CNN_ outputs for each frame were averaged across sequences.  
The results showed that incorporating temporal dependencies improved classification performance. While the _CNN Sequential_ approach slightly outperformed LSTM-based models in accuracy, high variability was observed across training runs. Leave-one-driver-out cross-validation showed that all models except the _CNN-LSTM_ struggled with validation data from a driver with darker images, highlighting the importance of considering multiple images for reliable detection. The recurrent approaches, especially the LSTM-based models, showed significantly better performance in such scenarios.  
The inclusion of an LSTM network improved the accuracy of the vision-based approach, and the sensor data also showed a trend toward improvement. However, further research is needed to validate this trend. The primary objective of this work was to compare different technologies rather than to develop a model for production use. Consistency in the recordings was maintained, but the results underscore the need for a larger, more diverse dataset for stable results. For practical use, more recordings should be made with different drivers, vehicles, routes, and recording conditions. A diverse dataset would allow the model to better generalize to new data, contributing to the development of a robust, practical model suitable for real-world implementation.  
This study considered a limited set of distraction classes, justified by the practical need to capture sufficient samples within a short recording time. However, a real-world model would need to include a broader range of distractions. Despite the limited number of classes, the class imbalance was a challenge, with the _normal_driving_ class dominating. Loss function weighting was used to address this, but further exploration of methods such as artificial upsampling is recommended. The research was also limited in terms of feature engineering, focusing on standard procedures such as scaling and normalizing the data. Future work could include more advanced feature extraction and exploration of alternative model architectures such as bidirectional LSTM networks or different CNN architectures such as _VGG19_, _InceptionV3_, _ResNet_, or _DenseNet_.  
In summary, AI-based driver distraction detection systems hold potential for future implementation in vehicles. With further adaptation and improvement, these systems could accurately detect distractions and issue warnings, thereby reducing distraction-related accidents. Such systems are especially critical for partially automated vehicles, ensuring driver alertness and facilitating safe control takeovers, ultimately contributing to overall road safety.  
Conceptualization, M.C. and G.O.; Methodology, M.C.; Software, M.C.; Writing--original draft, M.C.; Writing--review & editing, M.C.; Supervision, G.O.; Project administration, G.O. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.  
The dataset presented in this study is available upon request from the corresponding author. The data are not publicly available due to privacy reasons.  
The authors declare no conflicts of interest.