# A Survey on Datasets for Emotion Recognition from Vision: Limitations and In-the-Wild Applicability
## 3 Datasets used in State-of-the-Art Techniques
The authors used the Amazon Mechanical Turk service to bring annotators and ask about their perceived emotions from people and context. A first round of annotations is proposed, in which a single annotator annotates one image. After this, the test and validation set receives up to two and four extra annotations, leading to three and five annotators for these sets, respectively. The 2017 version contains 18,316 different images containing 23,788 annotated subjects, of which 66% are males and 34% are females, according to the authors. They also annotated the perceived age of the subjects, of which 11% were children, 11% were teenagers, and 78% were adults. The 2019 version contains 23,571 images containing 34,320 annotated subjects, with the same gender division as the 2017 version, but containing 10% children, 7% teenagers, and 83% adults. The Context-Aware Emotion Recognition (CAER) dataset [17] was proposed to solve limitations perceived by the authors on other in and also in EMOTIC. According to the authors, although EMOTIC contains contextual information, they work on a different aspect and propose a large-scale dataset for context-aware emotion recognition with various context information. We propose an experiment to validate this affirmation later in this work. Another limitation, compared to EMOTIC, is that emotions are highly dynamic, and using images could be a limiting factor for techniques. Therefore, the authors propose the CAER dataset, which contains videos, and the CAER-S dataset, for static images, the most commonly used benchmark between these two. Another main differing factor between CAER and EMOTIC is the sample source. While EMOTIC focuses on reusing images from other popular datasets and complementing \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Dataset Name** & **Annotation Formats** & **Annotators** & **Samples** & **Demographics** \\ \hline \multirow{2}{*}{EMOTIC [53]} & Categorical—26 classes & \multirow{2}{*}{AMT 2} & \multirow{2}{*}{18,316 images} & 23,788 people—66\% M and 34\% F; \\  & Continuous—VAD model & & & 78\% A, 11\% T, and 11\% C \\ \hline \multirow{2}{*}{EMOTIC 1[16]} & Categorical—26 classes & \multirow{2}{*}{AMT 2} & \multirow{2}{*}{23,571 images} & 34,320 people—66\% M and 34\% F; \\  & Continuous—VAD model & & & 83\% A, 7\% T, and 10\% C \\ \hline CAER and & Categorical—6 classes + & \multirow{2}{*}{6 annotators} & 13,201 video clips & \multirow{2}{*}{Unknown} \\ CAER-S [17] & neutral & & 70,000 images & \\ \hline iMiGUE [54] & Categorical—2 classes & 5 (trained) annotators & 359 video clips & 72 adults—50\% M and 50\% F \\ \hline AfeW [51] & Categorical—6 classes + & \multirow{2}{*}{2 annotators} & \multirow{2}{*}{1426 video clips} & \multirow{2}{*}{330 subjects aged from 1 to 70 years} \\  & neutral & & & \\ \hline AfeW-VA [52] & Continuous—VA model & 2 (trained) annotators & 600 video clips &
\begin{tabular}{l} 240 subjects aged from 8 to 76 years \\ 52\% female \\ \end{tabular} \\ \hline \multirow{2}{*}{BoLD [55]} & Categorical—26 classes & \multirow{2}{*}{AMT 2} & \multirow{2}{*}{26,164 video clips} & 48,037 instances—71\% M and 29\% F; \\  & Continuous—VAD model & & & 90.7\% A, 6.9\% T, and 2.5\% C \\ \hline \multirow{2}{*}{GroupWalk [56]} & Categorical—3 classes + & \multirow{2}{*}{10 annotators} & \multirow{2}{*}{45 video clips} & \multirow{2}{*}{3544 people} \\  & neutral & & & \\ \hline \hline \end{tabular} \({}^{1}\) A second version of EMOTIC was published in 2019. 2 Amazon Mechanical Turk.  
\end{table} 
Table 2: Overview of the datasets commonly used in the state-of-the-art techniques. Each row contains a different dataset. VAD or VA are continuous annotation models. For the demographics, M and F stand for male and female, respectively, while A, T, and C stand for adult, teenager, and children. them with images from the web, CAER focuses on extracting images from television shows. This yields questions regarding the expressivity of the emotions in the scenes.