## 4 Experiments
### Results
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Annotation** & **Acc2** & **F1** & **MAE** & **Corr** \\ \hline EF-LSTM [35] & M & 69.37 & 81.91 & 59.34 & \(-\)4.39 \\ MFN [36] & M & 77.86 & 78.22 & 45.19 & 55.18 \\ MulT [14] & M & 77.94 & 79.10 & 48.45 & 55.94 \\ \hline LF-DNN [37] & M & 79.87 & 80.20 & 42.01 & 61.23 \\ MLF-DNN [11] & M, A, T, V & 82.28 & 82.52 & 40.64 & 67.47 \\ \hline LMF [29] & M & 79.34 & 79.96 & 43.99 & 60.00 \\ MLMF [11] & M, A, T, V & 82.32 & 82.66 & 42.03 & 63.13 \\ \hline TFN [37] & M & 80.66 & 81.62 & 42.52 & 61.18 \\ MTFN [11] & M, A, T, V & 82.45 & 82.56 & 40.66 & 66.98 \\ \hline Self-MM [34] & M, A, T, V & 80.74 & 80.78 & 41.90 & 61.60 \\ Human-MM [34] & M, A, T, V & 81.32 & 81.73 & 40.80 & 64.70 \\ \hline XMBT\({}_{\text{dra}}\) & M, A, T, V & **83.37** & **83.15** & **37.61** & **68.04** \\ \(\Delta\) & - & \(\uparrow\)0.92 & \(\uparrow\)0.49 & \(\downarrow\)3.03 & \(\uparrow\)0.57 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Test results on CH-SIMS dataset. fore, a grid search was conducted to identify suitable values for these hyperparameters to optimize model performance. The XMBT model was trained with DRA loss using a range of values for \(T\) from 6 to 14 and \(L\) from 1 to 2. As depicted in Figure 7, at \(L=2\), a \(T\) value of 10 maximized the average F1 score and weighted accuracy. At \(L=1\), the model trained with \(T=10\) outperformed others regarding the F1 score. However, when increasing \(T\) to 12, the weighted accuracy increased from 74.3 to 75.1, but the F1 score significantly decreased from 49.6 to 42.3. Overall, in both cases, \(L=1\) and \(L=2\), a temperature of 10 produced an effective and stable model, balancing modality representation and overall performance. A T-SNE visualization on the IEMOCAP dataset was applied to analyze the emotion representation learning of the proposed DRA training strategy. As depicted in Figure 8, the modal-specific representation displays a significant contrast when the model trains using the traditional method. In the traditional method, the audio modality can effectively discriminate the emotion state, leading to well-clustered emotional features, as depicted in the upper left plot in Figure 8.  
Conversely, the linguistic modality presents the most complex representation, making it challenging to distinguish emotional features. This complexity arises from the high-level semantic features in language, complicating extracting emotional cues from text. The model may become biased toward audio input, neglecting textual information. The proposed DRA strategy addresses this problem by balancing the learning effort from each modality. As illustrated in the lower plots in Figure 8, the visualizations of modality representations have a shared structure. The learned patterns are consistent across modalities, and modal-specific features are more discriminative than those trained without DRA. Moreover, the representations are visibly clustered into two groups: excited, happy, and neutral samples and angry, frustrated, sad, and neutral samples. This clustering suggests that the model successfully differentiates between positive and negative video clips. The model  
Figure 6: CMU-MOSEI test results of XMBT and open-ended MBT variants. Figure 7: CMU-MOSEI test results of XMBT by temperature and number of bottleneck layers. implicitly learns how to solve the sentiment analysis problem via training on the emotion recognition task.