# Driver Drowsiness Detection by Applying Deep Learning Techniques to Sequences of Images
## 3 Materials and Methods
### Experimentation Methodology  
Experimentation is performed over the UTA-RLDD dataset, using videos of 60 different people on two different states: awake and drowsy. The training data used consist of 97 videos (48 awake, 49 drowsy), while the test data consist of 25 videos (12 awake, 13 drowsy). It is important to consider that UTA-RLDD is a realistic dataset, where the subjects are not simulating their drowsiness, so it is common that the recorded person does not show fatigue symptoms at every second of the video, but only at specific moments. Because of this, to evaluate the system's performance, each video is analyzed frame by frame, evaluating at each frame the drowsiness level of the driver. After this, the alarm activation module of the ADAS decides whether to alert the driver or not. The number of alarms raised during the video is counted, and this number is used to calculate the system's accuracy. This way, each video can count either as a hit (if the subject is awake and there are no raised alarms, or if the subject is drowsy and there is at least one alert) or as a miss (if there are no alarms when the driver is drowsy, or if the system raises an alarm when the driver is awake). If the system alerts the driver at least once, we consider that the video is classified as \"drowsy\", and if there are no alarms, it is classified as \"awake\". It is important to understand the following terms that are used while evaluating the system:  
* **FP (false positive)**: Video misclassified as \"drowsy\", where the subject was actually awake. * **FN (false negative)**: Video misclassified as \"awake\", where the subject was actually drowsy. * **TN (true negative)**: Video correctly classified as \"awake\", where the subject was awake. * **TP (true positive)**: Video correctly classified as \"drowsy\", where the subject was drowsy. As explained in Section 3.1.3, the alarm activation module works with two variables: _drowsiness_threshold_, which represents the minimum drowsiness level to consider that a driver is drowsy, and _min_time_, which represents how many seconds the driver has to be considered drowsy before raising an alarm. These two variables affect the system's performance: if the threshold is low and the minimum time is high, there are more possibilities of raising an alarm. This means that the driver is alerted at early phases of fatigue; however, it also increases the generation of false positives. Avoiding false positives is of the utmost \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Rule Number**} & \multicolumn{2}{c}{**Rule Description**} \\ \cline{2-3}  & **IF** & **THEN** \\ \hline
1 & (_avg\_blink_ is high) and (_yawns_ is low) & _drowsiness_ is medium \\
2 & (_avg\_blink_ is high) and (_yawns_ is high) & _drowsiness_ is high \\
3 & (_blinks_ is high) and (_avg\_blink_ is low) & _drowsiness_ is medium \\
4 & (_blinks_ is high) and (_avg\_blink_ is normal or _avg\_blink_ is high) & _drowsiness_ is high \\
5 & (_microsleeps_ is high) & _drowsiness_ is high \\
6 & (_yawns_ is low) and (_avg\_yawn_ is low) & _drowsiness_ is low \\
7 & (_yawns_ is low) and (_avg\_yawn_ is high) & _drowsiness_ is medium \\
8 & (_yawns_ is high) and (_avg\_yawn_ is low) & _drowsiness_ is medium \\
9 & (_yawns_ is high) and (_avg\_yawn_ is high) & _drowsiness_ is high \\
10 & (_blinks_ is normal) and (_avg\_blink_ is normal) and (_microsleeps_ is low) & _drowsiness_ is low \\
11 & (_blinks_ is low) and (_avg\_blink_ is low) and (_microsleeps_ is low) & _drowsiness_ is low \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Rules FIS. importance, since a system that raises alarms when it is not necessary will bother the driver, who will likely turn off the whole system. Because of this reason, we perform multiple tests with different combinations of thresholds and minimum times. We test the systems over the training data with a threshold ranging between 0.20 and 0.95 (using intervals of 0.05), and with a minimum time ranging between 10 and 60 (using intervals of 5).