# Reducing Videoconferencing Fatigue through Facial Emotion Recognition
## 4 Results  
We found that the VGG16 model performed best with a test accuracy of 84.0%, closely followed by the Xception model with 83.7%, the VGG19 model with 83.6%, and the VGG13 model with 82.6% (see the confusion matrices on the test set for each model in Figures 2-5). Figure 2 illustrates the confusion matrix on the test set for the VGG16 model including \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & **Anger** & **Fear** & **Surprise** & **Sadness** & **Neutral** & **Happiness** & **Total** \\ \hline AffectNet & 473 & 512 & 1379 & 569 & 1873 & 599 & 5405 \\ FERPlus & 2606 & 648 & 3950 & 3770 & 11,011 & 9066 & 31,051 \\ CK+ & 45 & 25 & 83 & 28 & 0 & 69 & 250 \\ JAFFE & 30 & 32 & 30 & 31 & 30 & 31 & 184 \\ BU-3 DFE & 92 & 92 & 89 & 88 & 84 & 77 & 522 \\ FFQH & 260 & 22 & 114 & 193 & 540 & 2326 & 3455 \\ \hline Total & 3506 & 1331 & 5645 & 4679 & 13,538 & 12,168 & 40,867 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Distribution of different emotions along the various FER data sets. of the speakers for each presentation using the ratio_audience(E) and ratio_speaker(E) features. The coefficients are captured in Table 2. We found that, firstly, the happier the speaker was, the happier and less neutral the audience was and, secondly, the more neutral the speaker was, the less surprised the audience was. Next, we computed the correlations between all features (18 audience and 8 speaker features) and the presentation scores. The significant features, their coefficients, and \(p\)-values are provided in Table 3.  
We found that the deviation in the audience's happiness, neutrality, and fear; the ratio in the audience's happiness and fear; the density in the audience's happiness as well as the ratio in speaker's happiness are positively related with the presentation score. Contrarily, the ratio in the audience's neutrality is negatively correlated with the presentation score. All of these correlations are significant with a \(p\)-value smaller than 0.05. The correlation between the deviation in audience's happiness and the presentation score is the most significant with a coefficient of 0.73 and a \(p\)-value of 7 \(\times\) 10\({}^{-8}\). Recall that the deviation in audience's happiness measures the variation in happiness throughout the presentation. The latter correlation is also illustrated in Figure 6, where we plotted the development in happiness for some selected presentations. We can see that the more variation in the audience's happiness we have, the better the score (see the blue dot in the upper right corner of Figure 6a) and vice versa (see the blue dot in lower left corner of Figure 6a). \begin{table}
\begin{tabular}{c c c c c c c} \hline  & **Ratio\_Speaker** & **Ratio\_Speaker** & **Ratio\_Speaker** & **Ratio\_Speaker** & **Ratio\_Speaker** \\  & **(Happy)** & **(Neutral)** & **(Fear)** & **(Sad)** & **(Surprise)** & **(Angry)** \\ \hline ratio\_audience(happy) & 0.5107 *** & \(-\)0.0754 & 0.0544 & \(-\)0.0532 & \(-\)0.2369 & \(-\)0.2082 \\ ratio\_audience(neutral) & \(-\)0.4347 *** & 0.0909 & \(-\)0.0432 & \(-\)0.0283 & 0.1850 & 0.1340 \\ ratio\_audience(fear) & 0.2240 & 0.0983 & \(-\)0.1953 & 0.0210 & \(-\)0.3097 & 0.1471 \\ ratio\_audience(sad) & \(-\)0.1185 & 0.0592 & 0.0165 & \(-\)0.0780 & \(-\)0.0153 & 0.2119 \\ ratio\_audience(surprise) & 0.2059 & \(-\)0.3147 ** & 0.1163 & 0.1995 & 0.1952 & 0.1602 \\ ratio\_audience(angry) & \(-\)0.0123 & 0.0013 & \(-\)0.0569 & 0.1650 & 0.0089 & \(-\)0.1450 \\ \hline \end{tabular}
\end{table} 
Table 2: Correlation coefficients between the emotions of the audience and the emotions of the speaker along all presentations (N = 41) (*** Correlation significant at 0.01 level; ** Correlation significant at 0.05 level).