## 5 Results
### Distracted Driver Modeling  
For this experiment we split the data as follows; data corresponding to any of the distraction segments is labeled as 'distracted', while data collected under the free-driving part is labeled as 'not-distracted'. To minimize the biases introduced by the relatively unfamiliar virtual-driving setup, we use five minute long data segments, extracted from the last seven minutes of the free-driving recording, when subjects were already used to the driving simulator.  
Similarly to the previous section, we perform a feature based analysis using the deep CNN-LSTM model. By observing the ROC curves of Figure 5 it would be safe to assume that identifying distracted behavior based on the selected feature-sets is relatively more challenging compared to detecting drowsiness. According to the AUC scores, all BVP related feature combinations provide by far the best results, indicating the strong relation of heart-rate related features to the task. More specifically, best results were achieved by the \"_top5 BVP_\" features of Table 2, with an AUC of 82% while second comes the \"_top3 BBI BVP_\" feature set with an AUC of 80% and third the set with \"_all BVP_\" related data with an AUC score of 79%. Of special interest is the very poor performance observed by the combination of raw BVP and respiration data, which provided the best results in the problem of drowsiness detection. Even though it is hard to clearly explain the very low performance of this feature set, we suspect that the overall poor performance of the respiration data on the task affects the results in a negative way. Judging from the AUC score achieved by the \"respiration\" feature set it seems that respiratory data are not as related to distracted behavior as they are to the drowsy one. However, these negative results need to be further evaluated in the future.  
Taking these findings into consideration, we evaluate the different classifiers on their ability to identify distracted driving based on the \"top5 BVP\" feature set. The results are presented in Table 4. Again, the CNN-LSTM model significantly outperforms all three baselines. The model provides the most balanced results with an average recall of 72%. The advantage of the CNN-LSTM model against its competitors, is less on its ability to identify distracted behavior and more on its balanced performance between sensitivity and specificity. In particular, the SVM model performs better on identifying non-distracted driving while its predictive ability with respect to distraction detection is almost random, thus making this approach the less appropriate of all for the purposes of the task. On the other hand, KNN and RF classifiers perform equally to the CNN-LSTM model on identifying drowsy driving. However, their high FP-rate makes them less appropriate  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Drowsiness Detection**} & & \\  & **SVM** & **KNN** & **RF** & **CNN-LSTM** \\ \hline Sensitivity (\%) & 0 & 17 & 17 & 93 \\ Specificity (\%) & 92 & 91 & 91 & 71 \\ Average Recall (\%) & 46 & 54 & 54 & **82** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Alert vs. drowsy classification using the combination of raw blood volume pulse (BVP) and respiration data as selected by the analysis in Figure 4.  
for modeling the problem as they have almost a 50% chance on marking a not-distracted driver as distracted.  
Overall, we can argue that the CNN-LSTM pipeline is by far the most effective on modeling both distracted and drowsy driver states, under the same experimentation conditions. That is both in terms of correctly identifying the condition of interest (more TPs) but also in discriminating against it (more TNs).