## 4 Results
### Verification  
Table 8 shows the average accuracies, drowsiness recalls, drowsiness precisions, AUCs, and training times of each model in the verification stage. Figure 8a shows the improvements in the hybrid models compared to the single models. Consistent with the results of the drowsiness detection stage, the hybrid model with principal component analysis and artificial neural networks showed the best performance while guaranteeing timeliness; the artificial neural networks performed better than the classic machine learning algorithms; and the hybrid models performed better than the single models in the verification stage. For more information, Appendix B shows the results of the verification.  
Table 9 shows the standard deviations of the accuracies, drowsiness recalls, drowsiness precisions, AUCs, and training times of the hybrid and single models; the best evaluation indexes in each model are bolded. The average standard deviations of the evaluation indexes of all the hybrid models were lower than 1%, which was lower than the single models. This showed that the hybrid models demonstrated a more stable performance than the single models. In addition, as shown in Figure 8b, most of the standard deviations in the accuracies, drowsiness recalls, drowsiness precisions, AUCs, and training times obtained by the hybrid models were lower than those of the single models. This showed that the performance of the hybrid models was more stable and less volatile than that of the single models.  
Figure 9 takes sample 9 to illustrate these findings in detail. The hybrid model with principal component analysis and artificial neural networks had the best performance; the accuracies, drowsiness precisions, and AUCs of the artificial neural networks were higher than the classic machine learning algorithms; and the hybrid models showed improvements compared to the single models in each evaluation index, which was consistent with the drowsiness detection stage. Moreover, the distribution ranges of the evaluation indicators of the single models were more comprehensive than those of the hybrid models. We speculated that the performance of the hybrid models was more stable than that of the single models.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{3}{*}{**Models**} & \multicolumn{6}{c}{**Hybrid Models**} & \multicolumn{6}{c}{**Single Models**} \\ \cline{2-9}  & \multicolumn{2}{c}{**Hybrid Model of Principal**} & \multicolumn{2}{c}{**Hybrid Model of Principal**} & \multicolumn{2}{c}{**Single Model of**} & \multicolumn{2}{c}{**Single Model of**} \\  & \multicolumn{2}{c}{**Component Analysis**} & \multicolumn{2}{c}{**Component Analysis**} & \multicolumn{2}{c}{**ANN**} & \multicolumn{2}{c}{**CMLA**} \\  & \multicolumn{2}{c}{**and ANN**} & \multicolumn{2}{c}{**and CMLA**} & \multicolumn{2}{c}{**} & \multicolumn{2}{c}{**} & \multicolumn{2}{c}{**} \\ \cline{2-9}  & **PCA-BPNN** & **PCA-CFNN** & **PCA-SVM** & **PCA-KNN** & **BPNN** & **CFNN** & **SVM** & **KNN** \\ \hline Accuracy (\%) & 97.3 & **97.4** & 94.8 & 94.7 & 96.3 & 96.5 & 93.7 & 93.4 \\ \hline Drowsiness recall (\%) & 98.4 & **98.5** & 98.2 & 97.0 & 97.6 & 97.8 & 97.8 & 96.1 \\ \hline Drowsiness precision (\%) & 97.3 & **97.4** & 94.1  
In general, the results of the verification stage were consistent with the results of the drowsiness detection stage. In addition, a hybrid utilizing principal component analysis can improve the stability of the model.  
Figure 8: Improvements in the hybrid models compared to the single models in the verification stage. ANN—artificial neural networks; CMLA—classic machine learning algorithms.  
\begin{table}  
\end{table}
Table 9: The standard deviation of the accuracies, drowsiness recalls, drowsiness precisions, AUCs, and training times of models in the verification stage.