# Driver Drowsiness Detection by Applying Deep Learning Techniques to Sequences of Images
## 3 Materials and Methods  
The aim of this work is to develop a system that is able to estimate the fatigue of a driver by using sequences of images that are recorded in such way that the face of the subject is visible.  
The drowsiness detection system developed in this work is part of a driver-based ADAS system [25; 26], with two important restrictions: early detection and minimization of the number of false positives. The idea is that the system will warn the driver only in real cases of fatigue, to avoid false positives, which would cause boredom in the driver, causing them to turn off the ADAS, without executing the rest of the functionalities.  
When it comes to the recording of the driver, it is important to determine the frame rate that the camera has to communicate to the system. A high frame rate will overload the system because of the high number of frames per second (FPS) that have to be evaluated, but a low amount of FPS can affect negatively the system performance. In this domain, it is necessary that there are enough FPS to appreciate details of the image sequence that have a very short duration, such as blinks.  
Since the average blink duration ranges from 100 to 400 ms [27], in this work, a frame rate of 10 FPS is used, which is enough to detect blinks and avoid overloading the system. This way, 600 frames are evaluated every time a new frame is captured by the camera. To do this, the system stores the previous 599 frames, so that a full sequence of 60 s is analyzed at each instant.  
As mentioned in Section 1, this work proposes two alternative solutions to estimate drivers' drowsiness. The first alternative uses a recurrent and convolutional neural network, while the second one uses AI and deep learning techniques to extract numeric features from images, and then introduces them into a fuzzy logic-based system. However, both solutions follow the same process structure, which consists of three phases: preprocessing, analysis, and alarm activation. Figure 1 shows the three different phases.  
As shown in Figure 1, the system receives 600 frontal images of the driver, corresponding to the last 60 s recorded by a camera placed on the vehicle (for example, on the driver's dash) at 10 frames per second (FPS).  
These images are received by the _preprocessing module_, whose objective is to transform the received image into data that can be used by the drowsiness detection model. The preprocessed data are then sent to the _analysis module_, which performs the fatigue detection tasks and assesses the level of drowsiness of the driver at that moment, based on the information from the last 60 s. Lastly, the calculated drowsiness level is transmitted to the _alarm activation module_, which uses the last levels of drowsiness to determine whether it is necessary to alert the driver or not.  
Figure 1: System overview.  
As mentioned above, one of the main objectives of the alarm activation module is to minimize the number of false positives of the system (drowsiness alerts when the driver is actually awake), since a high number of false positives disturbs the driver and increases the possibility of turning off the system. This is one of the reasons that motivate the experimentation over videos instead of frames and make the tests more exacting, since the activation of a single alarm in a 10 min video means that, regardless of what is detected before or after that moment, the classification of the video will be considered \"drowsy\".  
Once the system has made its decision on whether to alert the driver or not (a yes/no possible outcome), it will communicate its decision to the human-computer interaction system responsible for warning the driver by using visual and/or sound stimuli.  
The three modules (preprocessing, analysis and alarm activation) of each of the two alternative solutions are described below.