## 3 The Portrait Emotion3D (PE3D) Dataset
### Dataset Annotation  
Categorical emotion labels: The categorical model of PE3D comprises eight distinct categories: Neutral, Happy, Sad, Surprise, Fear, Anger, Disgust, and Contempt. The majority of these categorical labels align with those in EmoFAN [24] and other existing Art Facial Expressions datasets [44]. Upon labeling all images, art history experts will manually review and adjust these labels to represent the expressed emotions accurately. For example, when the model classifies the portrayed face as neutral, art experts are inclined to identify the image as happy, adjusting the final categorical label to 'happy'. Within these categorizations, neutral emotions constitute 56% of the dataset. Negative emotions, encompassing fear, sadness, surprise, and disgust, make up 27% of the emotional spectrum. Figure 5(b) demonstrates that most emotion labels are negative and neutral. This trend can be attributed to the penchant of Renaissance artists for depicting sacred  
Figure 4: Art movements of PE3D. First Row: HR: High Renaissance; IM: Impressionism; NR: Northern Renaissance; PI: Post-Impressionism (PI); RO: Rococo; UK: Ukiyo-e; ND: Not Defined. Third row: number of faces from each art movement. Figure 5: Workflow of the face detection algorithm. characters, who are predominantly portrayed with calm and solemn demeanors, with their facial expressions often appearing neutral or sad, thereby limiting the portrayal of intense, positive emotions like happiness. Conversely, in the Impressionist and Post-Impressionist periods, there was a marked shift toward conveying the artists' emotions, resulting in more varied and intense facial expressions, including pronounced anger and an emergence of negative emotions. Consequently, in the PE3D dataset, faces are depicted with a higher prevalence of neutral and negative emotions compared to real-world data. Emotional State Representation in a 3D Space: A 3D coordinate system demonstrated in Figure 2 represents emotional states. Each axis corresponds to a specific emotional dimension:  
* Z-axis: Dominance, ranging from \(-1\) (submissive) to \(1\) (dominant);
* X-axis: Valence, ranging from \(-1\) (unpleasant/negative) to \(1\) (pleasant/positive);
* Y-axis: Arousal, ranging from \(-1\) (calm) to \(1\) (excited). This representation facilitates a nuanced depiction of emotional states, wherein a high valence value signifies positive emotions such as happiness, and a low value denotes negative emotions like sadness. Figure 2 illustrates examples of emotions plotted within this 3D space. Generation of Emotional Labels: We employed the 3D labeling method described in [52] to obtain dominance, valence, and arousal values. Unlike traditional methods that convert categorical emotion labels into 3D coordinates to infer the same emotion, our approach augments this technique by utilizing the probabilistic vectors generated from eight emotion labels via the pre-trained facial emotion recognition model, EmoFAN [24]. These vectors convey a more nuanced array of information compared to the conventional  
Figure 6: Dataset Properties. (**a**) Resolution of Images. (**b**) Distribution of emotions. (**c**) Three-dimensional coordination and data distribution. (**d**) Three-dimensional emotion examples in PE3D. one-hot label vectors. Additionally, EmoFAN calculates valence and arousal rates for each image, thereby streamlining our methodology by necessitating the generation of only the dominance value. This integration with EmoFAN's valence and arousal estimates yields a holistic 3D representation of each emotional state, encompassing dominance, valence, and arousal (see Figure 7). Figure 8 presents several examples along with 3D labels from PE3D, showcasing four emotions: happiness, sadness, surprise, and fear derived from Impressionism (IM), as illustrated in subfigures (a-d). The analysis of average 3D label centers for IM in Table 5 reveals a predominance of negative emotions within IM, characterized by lower valence values.