# GCE: An Audio-Visual Dataset for Group Cohesion and Emotion Analysis
## 5 Experimental Results
### Discussion  
In this study, we created a novel group cohesion and emotion (GCE) dataset encompassing two modalities, audio and visual, meticulously segmented into 30 s intervals. The primary purpose of this dataset was to facilitate the prediction of group cohesion and the recognition of group emotion within conversational contexts. This resource holds a potential interest for scholars across diverse disciplines, including psychology, affective computing, and multimodal learning. Departing from the conventional approach of individual emotion recognition, this dataset allowed for an exploration of group dynamics, shedding light on the interplay between group emotional states and cohesion. To construct the dataset, video clips were queried using common human-human interaction terms, such as \"casual conversation\", \"interview\", \"meeting\", \"debate\", and \"discussion\", specifically focusing on groups comprising two to five individuals. A total of 1029 video clips were meticulously segmented to a predefined duration to streamline the development of deep learning models. Addressing the challenges posed by variable-length videos in deep neural networks, the segmentation facilitated efficient model construction, minimizing the complexity in the network design. The objectives of this dataset were as follows:  
1. **Data Creation**: The primary objective was to construct the Group Cohesion and Emotion (GCE) dataset, which comprised 1029 segmented videos sourced from YouTube with the creative common license to address copyright concerns. These videos showed a variety of group interactions, such as interviews, meetings, and casual conversations. Following the publication of this paper, we will make this dataset available for further research. This dataset served as a foundation for the in-depth analysis of group cohesion and emotion within social interactions. 2. **Annotation Process**: To create a reliable foundation for subsequent research, the dataset was meticulously annotated. Graduate psychology students assigned group cohesion levels within the range of 1 to 7 and categorized affective states as negative, neutral, or positive for each 30 s video segment. 3. **Inter-Rater Reliability Analysis**: To ensure the reliability and consistency of the dataset, the study employed the Fleiss Kappa statistic to assess the level of agreement among the seven annotators for categorizing group emotions. The analysis confirmed the dependability of the annotation process. 4. **Model Development**: The study aimed to establish a foundational model for predicting group cohesion and detecting group emotion within the GCE dataset. Advanced visual and audio embedding techniques, including RetinaFace, FaceNet, EfficientNet, SlowFast, OpenSMILE, and Wav2 Vec 2.0, were utilized for feature extraction. Multi-Head Attention (MHA) fusion was incorporated to enhance cross-representation learning. 5. **Unimodal and Multimodal Analysis**: The research conducted an in-depth analysis of group cohesion prediction and group emotion detection using both unimodal and multimodal techniques. This provided insights into the effectiveness of the different approaches and the interplay between the visual and audio features. 6. **Challenges of Class Imbalanced**: The dataset was constructed by collecting videos of group members having conversations, which were predominantly positive in real life. Consequently, the dataset reflected the real world and had a very high proportion of positive classes. Training a model on such an imbalanced dataset posed challenges, including difficulties in model convergence and a tendency to predict only the majority class. In this study, we aimed to improve the prediction accuracy for each class by enabling the model to learn the features specific to each class. While the baseline model performed well in predicting each class, even with an imbalanced dataset, it was limited to only increasing the prediction probability for the minority class.