# On-Road Driver Emotion Recognition Using Facial Expression
## 5 Results and Discussion
### Performances
Table 3 shows the comparison between the proposed method and other facial expression recognition SOTA work. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Models** & **Precision** & **Recall** & **F1-Score** & **ACC** \\ \hline \multicolumn{5}{c}{FERDERnet\_G VS. baseline} \\ \hline FERDERnet\_G & 0.881 & 0.879 & 0.88 & 0.888 \\ Googlenet & 0.793 & 0.78 & 0.781 & 0.803 \\ \hline \multicolumn{5}{c}{FERDERnet\_R VS. baseline} \\ \hline FERDERnet\_R & 0.939 & 0.956 & 0.947 & 0.956 \\ Resnet50 & 0.898 & 0.876 & 0.886 & 0.907 \\ \hline \multicolumn{5}{c}{FERDERnet\_I3 VS. baseline} \\ \hline FERDERnet\_I3 & 0.964 & 0.952 & 0.957 & 0.959 \\ InceptionV3 & 0.81 & 0.913 & 0.884 & 0.897 \\ \hline \multicolumn{5}{c}{FERDERnet\_I4 VS. baseline} \\ \hline FERDERnet\_I4 & 0.956 & 0.959 & 0.958 & 0.961 \\ InceptionV4 & 0.898 & 0.93 & 0.91 & 0.928 \\ \hline \multicolumn{5}{c}{FERDERnet\_X VS. baseline} \\ \hline FERDERnet\_X & 0.952 & 0.972 & 0.962 & 0.966 \\ Xception & 0.926 & 0.939 & 0.932 & 0.93 \\ \hline \hline \end{tabular} Note: G represents Googlenet backbone; R represents Resnet50 backbone; I3 represents InceptionV3 backbone; I4 represents InceptionV4 backbone; X represents Xception backbone.  
\end{table} 
Table 2: Model performance of on-road driver facial expression dataset. Figure 7: The confusion matrix of FERDERnet_X when trained on the on-road driver facial expression dataset. As shown in Table 3, the ensemble method reaches the best top 1 accuracy and F1-score, outperforming other networks. The training time reflects the network parameters and depth. It can be observed that DeepEmotion has the most straightforward network; hence it finished training in the shortest time, but obtained a much lower accuracy. The ensemble of five backbone networks required much longer training time; therefore, it is not acceptable for engineering application. On the other hand, FERDERnet_X used a relatively short training time to reach high accuracy. Considering demands on time and computation resources, FERDERnet_X is most suitable for practical application. Light conditions are various in real driving situations, hence, the model's performance under difficult conditions is important. An experiment to compare FERDERnet performance under different light conditions was conducted. Daytime driver facial images and nighttime driver facial images were randomly selected from the original dataset. Figure 8 shows some of the test samples. The test data included 140 day images and 140 night images (20 images for each emotion category) to compare the model's performance. Furthermore, the influence of a different training set was considered; the FERDERnet_X (FERDERnet with Xception backbone) model was trained on different datasets (day-images-only on-road driver dataset vs. the full on-road driver dataset (containing both day and night data)) to compare recognition accuracy. As illustrated in Table 4, the night images lower the model's recognition accuracy. However, by involving nighttime data in the training process, our model gains significantly improved nighttime recognition performance. Furthermore, comparing recognition accuracy of the model trained on different datasets, the nighttime recognition accuracy was much higher when the model's training set involved night data. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Method** & **Training Set** & **Test Set** & **ACC** & **F1-Score** \\ \hline FERDERnet\_X & ours & day-images & 0.966 & 0.962 \\ FERDERnet\_X & ours & night-images & 0.812 & 0.795 \\ FERDERnet\_X(D/O) & ours(day-images) & day-images & 0.958 & 0.951 \\ FERDERnet\_X(D/O) & ours(day-images) & night-images & 0.624 & 0.554 \\ \hline \hline \end{tabular} Note: FERDERnet_X stands for FERDERnet with Xception backbone, FERDERnet_X(D/O) stands for FERDERnet under the day-images-only on-road driver facial expression dataset.  
\end{table} 
Table 4: Comparison of model performance under day and night images.