# GCE: An Audio-Visual Dataset for Group Cohesion and Emotion Analysis
## 4 Baseline Model Development for Group Cohesion and Group Emotion Prediction
### Feature Extraction  
To provide a comprehensive understanding of the feature extraction process in our proposed baseline model, we will delve into its various components. Initially, in the visual domain, we isolate the visual frames of the video recordings and the audio waveforms. We will employ RetinaFace ([[https://github.com/serengil/retinaface](https://github.com/serengil/retinaface)]([https://github.com/serengil/retinaface](https://github.com/serengil/retinaface)), accessed on 1 May 2024) to crop the face images from each frame for the visual feature extraction. We will extract two types of representations: face attributes and facial expressions. We will utilize the FaceNet ([[https://github.com/davidsandberg/facenet](https://github.com/davidsandberg/facenet)]([https://github.com/davidsandberg/facenet](https://github.com/davidsandberg/facenet)), accessed on 1 May 2024) model (Face_1), which is pretrained on the VGGFace-2 ([[https://github.com/ox-vgg/vgg_face2](https://github.com/ox-vgg/vgg_face2)]([https://github.com/ox-vgg/vgg_face2](https://github.com/ox-vgg/vgg_face2)), accessed on 1 May 2024) dataset to generate the face feature vector. FaceNet [28] is a deep convolutional network that directly learns a mapping from face images to a compact Euclidean space where the distances correspond to a measure of face similarity. The model is trained using triplets of roughly aligned matching and non-matching face patches, optimizing the embedding itself rather using than an intermediate bottleneck layer. This approach achieves a greater representational efficiency, enabling a state-of-the-art face recognition performance with only 128-byte feature vectors per face. The VGGFace-2 dataset comprises approximately 3.31 million photographs, each depicting a unique human identity. In addition, we used the pre-trained EfficientNet [29] model (Face_2) trained on the AffectNet dataset to obtain the emotion-related features for each face. EfficientNet is a novel scaling method that enhances performance by balancing network depth, width, and resolution. This model utilizes a simple yet effective compound coefficient to uniformly scale all dimensions and is based on a newly designed baseline network through a neural architecture search. AffectNet is a substantial facial expression dataset containing approximately 0.4 million images that are manually labeled for the presence of eight different facial expressions, as well as the intensity of valence and arousal. We calculate the average of all facial attributes within a single frame and sample nine frames from the entire video using a sliding window approach. Consequently, we obtain the sequences of the encoding features, each with a size of 100 \(\times\) 512 pixels for facial attributes and 100 \(\times\) 1408 pixels for facial emotions. We employ the SlowFast ([[https://github.com/facebookresearch/SlowFast](https://github.com/facebookresearch/SlowFast)]([https://github.com/facebookresearch/SlowFast](https://github.com/facebookresearch/SlowFast)), accessed on 1 May 2024) network to extract a comprehensive representation of the video content. SlowFast [30] combines a slow, high-definition CNN (\"Fast\" pathway) and a fast, low-definition CNN (\"Slow\" pathway) to analyze both the static and dynamic content of the video. This parallel analysis enables a more thorough evaluation of a video's content. Overall, SlowFast generates a feature vector with 2304 dimensions derived from a video clip. The extraction of the audio features results in three distinct representations. Initially, we utilize the OpenSMILE ([[https://audeering.github.io/opensmile-python/](https://audeering.github.io/opensmile-python/)]([https://audeering.github.io/opensmile-python/](https://audeering.github.io/opensmile-python/)), accessed on 1 May 2024) toolbox to extract 88 characteristics from the Geneva Minimalistic Acoustic Parameter Set (GeMAPS), which is widely used in voice research and affective computing.