# A Survey on Datasets for Emotion Recognition from Vision: Limitations and In-the-Wild Applicability
## 3 Datasets used in State-of-the-Art Techniques
### Presence of Nonverbal Cues
#### 3.4.2 Body Keypoints Visibility  
Another essential cue that is present in these datasets is the body. Even though only specific datasets such as iMiGUE have annotated body language (see Table 3), scientists and engineers can extract insights from this cue by using different types of approaches. Ref. [19], for example, employed a body encoding stream that used a simple model proposed by [82] until its last convolutional layer, allowing the network to correlate the internal representations of this model and emotion. After that, more robust approaches were proposed, which focused on an activity recognition pipeline, such as [20; 56]. We propose an experiment to assess the visibility of the body keypoints on images from three datasets: EMOTIC and CAER, which are the main datasets used today in the literature, and also BoLD, given its body language focus.  
In this experiment, we loop through every image of the datasets, except for BoLD for which we choose a single random frame from every video. This different approach is motivated by some reasons, such as (a) there is low variability between frames in each video of BoLD and (b) the high number of recorded frames would make this experiment unfeasible due to the high processing time. Please note that, although BoLD has ground-truth body keypoint annotation, we chose not to use it in our evaluation to keep a comparable basis regarding body keypoint visibility.  
We use you only look once (YOLO) [83] for each image or video frame to detect the people present on the scene. EMOTIC has ground-truth annotations regarding this aspect. Again, we chose not to use them in our evaluation to keep a comparable basis among datasets. For each detected person, we feed the cropped region to MediaPipe [84] to extract the detected pose. We empirically chose MediaPipe for this experiment due to its capability of also predicting keypoint visibility. Next, we sum the visibility of each of the 33 keypoints for each dataset. Finally, we plot these points using a spatial distribution representing a neutral pose body. For the drawing, we normalized the keypoint visibility regarding the maximum value stored to allow better comparison between points in the same and different datasets. We show the results of this experiment in Figure 7.  
From the results of this experiment, we can notice that CAER-S is the dataset with the least visibility regarding the lower body parts. Given how the dataset is built upon video clips from television shows, it is common to have the lead performer centered on the screen and only show the lower body parts when relevant to the story. BoLD, although being focused on body language, also has less visibility for the lower body parts when compared to EMOTIC. Please note that BoLD provides body annotations and that this result may differ when theses annotations are used. Finally, EMOTIC has a well-distributed visibility  
Figure 6: A bar chart with the occurrence of the classifications of each image from CAER-S (red) and EMOTIC (blue) with the scene recognition approach proposed by [80]. The difference in the height of each bar indicates the difference in the number of samples of each dataset. A dataset can be considered balanced if the samples are distributed equally among the classes. For EMOTIC, we can see that this behavior is more visible than for CAER-S.  
of the whole body, which is justified by the multiple camera placements present in the dataset. All of the datasets have good visibility for the face, hands, and arms.