# Comparing Approaches for Explaining DNN-Based Facial Expression Classifications
## 5 Results
### User Study Results  
For determining the significance of the results, we take \(\alpha=0.05\) for all questions. In Table 3, the results from the Wilcoxon signed-rank test on the question pairs from question set 1 can be found. Scores for questions 1, 3, and 4 have a significant difference between the first and second question batch. Question 4 was a control question and the significantly lower scores for this question shows that participants filled in the questions seriously and that posterior distributions provide significant information about prediction confidence. The significantly lower scores for question 1 shows that the participants' understanding of the model has increased after they saw the explanations. The participants also think the explanations are more sufficiently detailed, indicated by the significantly lower scores for question 3. Question 6 does not have a significantly higher score, which is a positive outcome. This indicates the participants did not find the explanations unnecessarily more complex than no explanation. Questions 5, 9, and 10 regard the trust in the model's performance. None of these questions show a significant increase in score after the participants saw the explanations. In line with the literature [55], this shows that building trust for a technology/model is not easy and may require longitudinal exposure to or testing of the technology. Figure 4: Example explanations for the CNN using Grad-CAM (second row) and DNN (third row) displaying all seven emotions. The images are the same and have the following codes: AF07 DIHR, AM11 AFHL, AF04 NES, AF03 ANHR, AF01 SAS, AM01 HAS, BM03 SUHL, respectively. The top and bottom row give the corresponding probability distributions for the images shown. The results of the Mann-Whitney U test on question set 2, where the explanations for the CNN and those for the DNN are compared, can be found in Table 4. The scores for questions 1, 3, 4, 7 and 8 are significantly below the median. Questions 5 and 6 do not show a significant difference in scores in either direction, which is to be expected. After all, the probability distributions for both models were given in all examples. Like before, these can be used as control questions. The scores for questions 1, 4 and 8 are significantly lower, which shows that participants' found the explanations for model 2 (the DNN with geometric features) more understandable, detailed, and precise than those for model 1 (the CNN). Moreover, the participants would also prefer the explanations for model 2 over those for model 1, as indicated by the significantly lower score for question 3. There is a significantly lower score for question 7 as well. That indicates that the participants found the explanations for model 2 to be more complex than those for model 1. Even so, it seems they would still prefer the more complex explanations. For questions 2 and 9, we see the same pattern as in the previous tests: the questions regarding trust in the models do not show a significantly lower score. Also, like in the previous tests, there is no significantly lower score regarding the predictability of the model outputs (question 8 for the first set and 10 for the second one). \begin{table}
\begin{tabular}{c c c c} \hline
**Question \#** & \(\mathbf{H_{a}}\) & **U-Value** & \(\mathbf{p}\)-Value** \\ \hline
1 & less & 42.0 & 0.031 * \\
2 & less & 66.0 & 0.364 \\
3 & less & 42.0 & 0.031 * \\
4 & less & 18.0 & 0.0003 * \\
5 & unequal & 48.0 & 0.105 \\
6 & unequal & 54.0 & 0.154 \\
7 & less & 36.0 & 0.011 * \\
8 & less & 24.0 & 0.001 * \\
9 & less & 66.0 & 0.364 \\
10 & less & 9.0 & 0.327 \\ \hline \end{tabular}
\end{table} 
Table 4: Results of the Mann–Whitney U test on questions from question set 2. Question numbers are the same as used in Section 4.4.2. \(H_{a}\) refers to whether the alternative hypothesis was that model 1 was evaluated as worse than model 2 (less) or a two-sided test (unequal). * indicates \(p\leq\alpha\).