## 2 Background and Related Works  
### What is an Expression?  
Facial expressions are a fundamental component of human interaction, along with prosody, body posture, hand gestures and other behavioral cues that carry non-verbal messages [11]. Facial expressions reveal the focal point of our attention; show signal comprehension or disagreement; indicate humorous or serious intent; and generally regulate and enrich interpersonal communication [12]. Consequently, automatic facial expression recognition (AFER) has consumed a lot of effort in the past two decades due to it being fundamental for human-computer interactions and having applications in a plethora of fields and problems, such as psychology, medicine, education, computer science, well-being assessment and ambient intelligence [13; 14; 15; 16].  
[PERSON] was the first to stipulate that certain facial configurations are expressions of certain emotion categories [17]. This idea of the universality of facial expressions across gender, race, ethnicity, culture, etc., later influenced [PERSON] [18] who proposed the discrete emotional model using six universal emotions--_happiness_, _surprise_, _anger_, _disgust_, _sadness_ and _fear_, together with a _neutral_ emotion. While critiques of [PERSON]'s basic emotions theory are aplenty and alternative theories of affect do exist [1], it stands that most research in AFER, as well as datasets, stand directly or indirectly on the shoulders of this theory aiming at classifying these (or a similar set of) discrete categories of emotion.  
There are two main modalities in AFER research: _static_ and _dynamic_. In static AFER, feature representation is obtained only from spatial information from a single image, while in dynamic AFER the temporal dynamic of an expression is also modeled [19]. Recent years have also seen a surge in multimodal systems where additional modalities are considered, such as speech (prosody) and physiological signals (see [20; 21] for in-depth reviews), so that more general simulation or prediction-based approaches become affordable [22].  
Early efforts in AFER research were driven by hand crafted features that were finally fed into a classifier [23]. However, with the advent of deep neural networks in recent years and the increase in data availability for AFER, deep learning became the state of the art for the problem by exceeding the performances of other methodologies by a large margin [21].  
Alongside the rise in the trend of using deep neural nets, the datasets in AFER evolved to be ever more challenging. AFER is a problem a lot of attention has been dedicated to; there are plenty of publicly available datasets such as CK+, FER+, MMI, AMHUSE [4; 24; 25; 26] and others (see [21; 27] for a comprehensive list). However, a large number of these datasets have been collected in controlled laboratory conditions with minimum variations in head pose, illumination and occlusions. Thus, they lack the uncertainty of a real-world setting, making any machine learning model trained on them have unrealistic biases and questionable robustness. Further, the size of these datasets is seldom sufficient to train the data-hungry deep models well. This has spurred on efforts toward establishing the so called datasets of facial expressions _\"in-the-wild\"_, which consist of spontaneous expressions collected in diverse and realistic settings, such as the AFEW dataset [28]. The performances of state-of-the-art models differ greatly when applied to acted data and in-the-wild data. For example, a recent method described in [29] obtained 98.06%, 82.74% and 61.52% accuracy on CK+, MMI and AffectNet datasets, respectively, of which a considerably lower result was obtained on AffectNet, the only in-the-wild dataset of the three.