# Driver Drowsiness Detection by Applying Deep Learning Techniques to Sequences of Images
## 6 Conclusions  
In this paper, two different implementations for a driver drowsiness detection system are proposed, where deep learning plays an important role. These systems use images of the driver to identify fatigue symptoms, but instead of predicting whether a driver is tried or not from a single image, in this work, a full sequence of 60 s is used to determine whether the driver is tired or not over the last minute.  
The first solution proposed uses a model based on deep learning for the estimation of the drowsiness level of the driver, using a combination of a convolutional neural network with a recurrent neural network. The second solution uses fuzzy logic for calculating the fatigue but needs to apply artificial intelligence and deep learning techniques to preprocess the data before using the fuzzy inference system.  
Testing was performed using a 5-fold cross-validation on 122 videos that have a duration of approximately 10 min per recording, which are provided by the UTA-RLDD database. The number of raised alarms was counted for each video, verifying in this way whether the system is reliable or not. Neither of the systems reported a satisfying performance, both of them obtaining an accuracy of around 65% over training data and over 60% on test data.  
However, the second alternative, which combines deep learning with fuzzy logic, reported promising results. This system is able to work continuously without bothering the driver when he or she is not drowsy, since among the 60 videos of attentive drivers, there was only one video in which the system raised an alarm incorrectly (raising an unnecessary alarm only in 7% of the cases where the driver was actually alert). This way, the minimization of the false positive rate obtained is considered a success.  
Its accuracy when correctly detecting the drowsiness of the driver, however, needs to be improved, because the system alerted the driver only in approximately 22 out of 61 videos where the subjects were drowsy (36% accuracy). This means that, although the conditions established for the fuzzy system are related to fatigue symptoms and can be used to detect fatigue, they do not represent all of the possible symptoms and thus cannot detect drowsiness on all videos.  
Both systems have great potential, and multiple ways of improving them were identified and will be addressed in the future. Detecting drowsiness from images of the driver is a complex problem that even commercial automotive brands struggle with. Further investigation will be needed before completion, for which this work stands as a solid baseline to improve upon.  
Conceptualization, M.P.S.; methodology, J.M.A.-W. and A.S.; software, E.M.; validation, E.M. and A.S.; formal analysis, M.P.S.; investigation, M.P.S. and J.M.A.-W.; resources, A.S.; data curation, J.M.A.-W.; writing--original draft preparation, E.M. and J.M.A.-W.; writing--review and editing, M.P.S. and A.S.; visualization, E.M.; supervision, M.P.S. and A.S.; project administration, A.S.; funding acquisition, A.S. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Not applicable.  
The data supporting reported results can be found publicly available at (UTA-RLDD) [[https://sites.google.com/view/utarldd/home](https://sites.google.com/view/utarldd/home)]([https://sites.google.com/view/utarldd/home](https://sites.google.com/view/utarldd/home)), accessed on 1 December 2021, (UTA-RLDD dataset [22], a dataset by [PERSON] et al.) and at (YawDD) [[https://ieee-dataport.org/open-access/yawdd-yawning-detection-dataset](https://ieee-dataport.org/open-access/yawdd-yawning-detection-dataset)]([https://ieee-dataport.org/open-access/yawdd-yawning-detection-dataset](https://ieee-dataport.org/open-access/yawdd-yawning-detection-dataset)), accessed on 1 December 2021, (YawDD dataset [24], a dataset by [PERSON] et al.).