# Comparing Approaches for Explaining DNN-Based Facial Expression Classifications
## 5 Results
### Discussion  
Our main goal in this paper was to compare alternative DNN-based approaches in terms of their predictive performance and explainability using both objective quantitative measures and a user study. However, these explanations may well be presented together, in case these two models are combined at the decision level. Since the geometric features-based DNN and appearance-based CNN models use alternative, complementary representations, their decision fusion is likely to improve the predictive performance [14; 56]. To further improve the predictive performance using the DNN and CNN in an explainable manner, we experimented with simple weighted fusion. Here, to fuse the posterior probabilities of the two models, a fusion weight \(\gamma\in[0,1]\) is optimized on the validation set with steps of 0.1. We found the best validation set accuracy of 0.8392 with \(\gamma=0.5\), which actually degenerates to unweighted score fusion. Using this setting, we reach a test set accuracy of 0.8418, which advances the state-of-the-art on this dataset using three poses (see Table 5). The final test set accuracy scores of the models and a few recent models on the KDEF set can be seen in Table 5. The state-of-the-art models were chosen by looking at recent research (published after 2018), where the KDEF dataset was used. It should be noted these do not necessarily use the same train-validation-test set split as we have used in this study. In [59], a combination of image pre-processing and different CNN models was used. The preprocessing consisted of converting the images to greyscale, data augmentation, cropping the images using Haar feature-based cascade classifiers [60], and downsampling the images to reduce memory usage. The CNN that obtained the best results on the KDEF dataset was the model fine-tuned over the model initially trained by [PERSON] and [PERSON] [61]. To obtain the result as reported in Table 5, they use only the frontal and half rotated images, as we have done. The result in [57] was obtained by detecting the faces using the method from [60] as well, then segmenting the image intro four parts (i.e., right eye, left eye, nose, mouth). Next, they extracted features from those segments using Gabor filters and these features are fed into a K-nearest neighbours model for classification. They used only the frontal oriented images from the KDEF set. In [58], a CNN model was trained only on the frontal images of the KDEF set. The faces were extracted from the images using the technique from [60], like the other examples. They trained CNN models with different architectures and used Grad-CAM and saliency maps to compare the different models on explainability. A simple decision fusion of our experimented models outperformed the state-of-the-art models, even though this was not the main target of the study. The geometric features-based DNN performs somewhat better than the CNN model (based on the combined score of the three DNN models). This could partly be attributed to the size of the dataset. The CNN is more likely to outperform the DNN if we had used a more substantial amount of data. A \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **Test Set Accuracy** \\ \hline [PERSON] et al. [57] (FR) & 0.8602 \\ [PERSON] et al. [58] (FR) & 0.8888 \\ GEO-DNN Frontal (FR) & 0.8117 \\ GEO-DNN Half Left (HL) & 0.7395 \\ GEO-DNN Half Right (HR) & 0.7913 \\ \hline GEO-DNN Combined (FR, HR, HL) & 0.7832 \\ CNN (FR, HR, HL) & 0.7595 \\ Fusing DNN \& CNN (FR, HR, HL) & 0.8418 \\ [PERSON] and [PERSON] [59] (FR, HR, HL) & 0.8086 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The test set accuracy scores for the final DNN models (and their concatenated score) and the CNN. Included are three state-of-the-art scores on the KDEF dataset. future work in this direction would be to use a multi-stage pretraining approach for CNN as done in [15] and include other FER corpora for training the DNN.