# GCE: An Audio-Visual Dataset for Group Cohesion and Emotion Analysis
## 5 Experimental Results
### Performance
Although the baseline model had the characteristic of a good prediction performance for each class, it showed a performance difference of about 0.2 in the AP and AUC metrics compared with the model that only uses a spectrogram due to the characteristics of the imbalanced dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Modality** & **Features** & **MAE \(\downarrow\)** & **MAPE \(\downarrow\)** & **MSE \(\downarrow\)** & \(\mathbf{R^{2}}\) \(\uparrow\) \\ \hline \multirow{3}{*}{Visual} & _Face\_1_ & 2.600 & 0.514 & 7.533 & \(-\)8.314 \\  & _Face\_2_ & 0.773 & 0.207 & 1.153 & \(-\)0.425 \\  & SlowFast & 5.436 & 1.136 & 30.645 & \(-\)36.885 \\ \hline \multirow{3}{*}{Audio} & OpenSMILE & 1.223 & 0.255 & 1.788 & \(-\)1.210 \\  & Wav2 Vec 2.0 & 1.929 & 0.423 & 2.481 & \(-\)3.883 \\ \cline{1-1}  & Spectrogram & 0.547 & 0.173 & 0.864 & \(-\)0.022 \\ \hline \multirow{3}{*}{Fusion} & Visual & 0.744 & 0.202 & 1.074 & \(-\)0.328 \\ \cline{1-1}  & Audio & 0.542 & 0.149 & 0.820 & \(-\)**0.013** \\ \cline{1-1}  & Audio & **0.077** & **0.116** & **0.012** & \(-\)0.160 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance on group cohesion prediction. Fusion (Audio) includes representations of OpenSMILE, Wav2 Vec 2.0, and a spectrogram. Fusion (Visual) includes representations of Face_1, Face_2, and the SlowFast model. \(\downarrow\) indicates that lower is better and \(\uparrow\) indicates that higher is better. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Modality** & **Features** & **Acc \(\uparrow\)** & **Pre \(\uparrow\)** & **Rec \(\uparrow\)** & \(\mathbf{F_{1}}\) \(\uparrow\) & **AP \(\uparrow\)** & **AUC \(\uparrow\)** \\ \hline \multirow{3}{*}{Visual} & _Face\_1_ & 0.35 & 0.25 & 0.23 & 0.23 & 0.30 & 0.39 \\  & _Face\_2_ & 0.65 & **0.33** & 0.34 & 0.28 & 0.43 & 0.65 \\  & SlowFast & 0.64 & **0.33** & **0.35** & 0.31 & 0.29 & 0.37 \\ \multirow{3}{*}{Audio} & OpenSMILE & 0.22 & 0.08 & 0.33 & 0.12 & 0.44 & 0.61 \\  & Wav2 Vec 2.0 & 0.60 & 0.21 & 0.30 & 0.25 & 0.22 & 0.25 \\ \multirow{3}{*}{Fusion} & Spectrogram & 0.60 & 0.25 & 0.31 & 0.27 & **0.53** & **0.73** \\  & Visual & 0.66 & 0.22 & 0.33 & 0.27 & 0.35 & 0.54 \\ \multirow{3}{*}{Fusion} & Audio & 0.66 & 0.22 & 0.33 & 0.27 & 0.33 & 0.49 \\  & Visual + & **0.91** & 0.30 & 0.33 & **0.32** & 0.36 & 0.56 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Performance on group emotion recognition. Fusion (Audio) includes representations of OpenSMILE, Wav2 Vec 2.0, and a spectrogram. Fusion (Visual) includes representations of Face_1, Face_2, and the SlowFast model. \(\uparrow\) indicates that higher is better. Figure 11 provides a graphical depiction of the confusion matrix, which presents a thorough evaluation of the effectiveness of both the unimodal and multimodal approaches in categorizing three separate emotional classes within the groups: negative, neutral, and positive. Upon a thorough examination of the outcomes, it became evident that a recurring obstacle faced by all models is the phenomenon of overfitting. This observation highlights a prevalent pattern in the models where there is a tendency to give more weight to predictions for the dominant class (positive). However, it is important to note that the OpenSMILE model stands out as an exception, because it shows a noticeable ability to accurately predict instances belonging to the negative class. Figure 11: Confusion matrix for group emotion recognition.