# Comparing Approaches for Explaining DNN-Based Facial Expression Classifications
## 1 Introduction  
The field of affective computing is concerned with providing computers the ability to examine and understand human affects and form their own human-like affects [1]. These notions are essential for creating empathetic computers that can interact appropriately with users, e.g., in situations such as (mental) health care, education, caring for the elderly, etc. One of the key elements of affective computing is emotion recognition. Emotions can be recognized using acoustic, visual and linguistic modalities [1]. In visual modality, emotions are mainly recognized from faces, under Facial Expression Recognition (FER), which is the area of focus for this research. In the context of using visual information (i.e., images and videos), FER can be categorized into two different approaches: conventional and deep-learning [2]. Traditionally, FER was done in three main steps: component detection, feature extraction, and finally the emotion classification. More recently, there has been a surge in the usage of deep-learning models for image- and video-based tasks, including but not limited to FER. Especially the use of convolutional neural networks (CNN) has increased immensely [3] and they have dramatically outperformed conventional models [4]. When using these models, the networks can extract features on their own and the images can thus be fed into the network directly instead of extracting the features beforehand. That means such models are not limited to human-extracted features. Thanks to CNNs being so powerful, they have become the dominant approach for state-of-the-art methods of affective computing and FER [2]. However, these models are also extremely opaque. CNNs belong to the class of black-box models, which means they cannot be interpreted by humans. Even if one looks at all the internal components of sucha model, one could still not comprehend what abstractions the model has learned and why it makes the decisions it makes. This is a problem as Artificial Intelligence (AI) models are getting more and more involved in our daily lives. Especially in high-stakes environments (e.g., the legal system, education, mental health care) it is important that we understand the actual decision making mechanism, as the model's decisions can have far-reaching consequences [5; 6]. Moreover, model transparency is an important factor for building trust and technology adoption [7; 8]. This is where the field of explainable artificial intelligence (XAI) comes into play. The broad goal of XAI is to make models more interpretable for humans [9]. In the context of affective computing and FER, researchers have started implementing XAI methods for the models they use [10; 11] and even challenges for explainable affective computing have been organized [12; 13]. In this study, we explore the interpretability of models based on geometric features. Furthermore, we compare the interpretability of such a model with that of a state-of-the-art CNN. We attempt this by constructing two models: a deep neural network (DNN) based on geometric features extended from [14] and a CNN using transfer learning on a pre-trained model developed in [15]. Both models are trained on images of facial expressions and perform an emotion classification task. We generate explanations for both the DNN and the CNN model. We evaluate the quality of the DNN explanations using several XAI measures and compare these explanations with the explanations for the CNN. Moreover, the explanations are assessed and compared via a user study. In short, the contributions of this work can be summarised as follows:  
1. Developing a new method of visually and textually explaining DNN predictions based on geometric features. 2. Making a direct comparison between the interpretability of a CNN and a DNN trained for an emotion classification task. 3. Performing a user study to evaluate and compare the quality of the explanations. This study is organized as follows: first we discuss background and related literature in Section 2.