## 3 Method and Experimental Setting
### Dataset  
There are several datasets which feature speaking subjects annotated with emotions that could be considered for our experiments. IEMOCAP [33] features 12 h of dyadic interactions, both acted and staged, annotated with the seven discrete emotional classes. The state of the art on IEMOCAP for visual AFER stands at 64% and is achieved by deep modelling of interactions via multiple modalities [39]. SEMAINE [40] and its modified counterpart AVEC [41] contain a total of 959 interactions between humans and artificially intelligent agents. While AVEC does not provide discrete emotions, SEMAINE annotations are provided in two flavors, real valued affective dimensionsand discrete emotions. However, the discrete emotions do not contain the full set of [PERSON]'s basic emotions but only a subset, lacking _surprise_ and adding _contempt_ and _amusement_, which reflects its purpose of being a dataset of dyadic interactions. The MELD [42] dataset contains about 13,000 utterances from multi-party dialogues from the TV-series Friends, with real valued annotations provided for the available modalities: audio, visual and textual. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) [5] consists of 24 professional actors vocalizing lexically matched statements. Speech includes all of [PERSON]'s basic emotions, namely, _calm_, _happy_, _sad_, _angry_, _fearful_, _surprise_ and _disgust_ expressions, with each expression produced at two levels of emotional intensity, with an additional _neutral_ expression at only a single intensity. RAVDESS is a very recent dataset, and has been adopted in research so far mainly in the field of emotion detection from speech. State of the art accuracy on the dataset for the problem reaches over 70% (see [43; 44; 45; 46; 47] for an overview). However, to our knowledge there is no work on visual only emotion recognition on the RAVDESS dataset.  
We turn to RAVDESS, it being particularly suitable for our experiments as it covers all the basic emotions, with the addition of _calm_ which we ignore, and it has multiple subjects which introduces diversity in the data, among which there is minimal variation in external conditions. In fact, the actors are directly facing the camera with consistent illumination and no occlusions, and all are uttering the same phrases, making the only variations in the data the visual aspects of the actors and their performances. The face and the mouth being fully visible in consistent conditions makes for optimal inputs to both visual AFER and lip reading tasks. This reduces any noise and variation fundamentally unrelated to the problem at hand, allowing one to discover to a greater extent than otherwise the potential of our approach in increasing AFER accuracy. Furthermore, the other mentioned datasets are mainly captured in a setting of dyadic interactions and geared toward modeling the same through multimodal approaches, or they feature a non standard subset of [PERSON]'s emotions as annotations. For these reasons we opted for RAVDESS, thereby allowing us to focus on [PERSON]'s emotions and evade the additional complexity introduced by an interaction setting.