# A CNN-Based Approach for Driver Drowsiness Detection by Real-Time Eye State Identification
## 5 Conclusions and Future Works  
This study presents an approach for drowsiness detection, where an enhancement method is proposed in the area surrounding the eyes to perform region of interest (ROI) extraction. Likewise, three CNNs are used as a basis: InceptionV3, VGG16 and ResNet50V2. A modification in the architecture of the fully connected network used in the classification process is proposed. For the experiments, a database was created from NITYMED videos. The results were obtained from 10 experiments performed and showed an exceptionally high accuracy in drowsiness detection using the architectures based on the three CNNs mentioned above, with values of 99.31%, 99.41% and 99.71%, respectively. The response times used for drowsiness detection by each CNN were shown to be relatively equivalent, with the VGG16-based CNN showing a small advantage. In addition, the Grad-CAM visual technique was used to analyze the behavior of each CNN, where the ResNet50V2-based CNN predominantly focuses on the eye region, achieving better performance in drowsiness detection. These results suggest that the proposed approach may be a good alternative for the implementation of the drowsiness detection system. Among the CNNs used, the ResNet50V2-based CNN presented the best performance, and considering the results of the examples in different scenarios (Figure 13), this architecture also presents higher robustness. When comparing the execution time for detection of this CNN with the other two CNNs (Table 5), it can be considered acceptable. When the system based on this proposal is implemented, it can be considered a valuable tool for the prevention of automobile accidents caused by driver drowsiness. As future work, we intend to make use of near-infrared (NIR) imaging to better focus on the eye region when there are illumination limitations. As a complement to this work, yawning detection can also be performed for preventive identification of drowsiness. Finally, the authors intend to implement this in an embedded system adapted to vehicular units. The following supporting information can be downloaded at: [[https://www.mdpi.com/article/10.3390/app13137849/s1](https://www.mdpi.com/article/10.3390/app13137849/s1)]([https://www.mdpi.com/article/10.3390/app13137849/s1](https://www.mdpi.com/article/10.3390/app13137849/s1)), Video S1: prueba-1.mp4. ; Video S2: prueba-2.avi. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Autor** & **Facial Method** & **ROI** & **Accuracy** \\ \hline [PERSON] et al. [17] & VGG-FaceNet & Face & 73.06\% \\ [PERSON] et al. [18] & Haar Cascade & Eyes & 96.42\% \\ [PERSON] et al. [19] & MTCNN & Face & 93.623\% \\ [PERSON] et al. [20] & Dlib & Face & 97\% \\ [PERSON] et al. [21] & Haar Cascade & Eyes & 96.82\% \\ [PERSON] et al. [22] & Haar Cascade/Dlib & Eyes & 98.15\% \\ [PERSON] and [PERSON] [23] & 3D-CNN+LSTM & Face & 96\% \\ [PERSON] et al. [24] & CNN+LSTM & Face & 97.31\% \\ [PERSON] et al. [25] & Dlib & Eyes & 80\% \\ [PERSON] et al. [26] & Dlib & Eyes & 94\% \\ \hline \hline \end{tabular}  
* Based on InceptionV3
* Based on VGG16
* Eyes & 99.31\%
* Based on ResNet50V2
* MediaPipe & Eyes & 99.71\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of drowsiness detection methods. **Author Contributions:** Conceptualization, methodology and software, R.F. and F.P.-Q. ; validation and formal analysis, R.F., F.P.-Q., T.P. and A.B.A. ; resources, F.P.-Q., R.J.C.-C. and J.C.H.-L.; data curation, R.F. ; writing--original draft preparation, R.F. and F.P.-Q. ; writing--review and editing, T.P. and A.B.A. ; supervision, A.B.A. and F.P.-Q. ; project administration, F.P.-Q. ; funding acquisition, F.P.-Q., R.J.C.-C. and J.C.H.-L. All authors have read and agreed to the published version of the manuscript. **Funding:** This research received no external funding. **Institutional Review Board Statement:** Ethical review and approval were waived for this study due to that the person included in the test of this research gave his consent. Being a first degree relative (father) of the first author.