## 4 Experiment
### Ablation Study
#### 4.3.1 Dual-Attention Mechanisms  
The MuIT model first uses multiple cross-modal attention mechanisms between the bimodal features and later uses a Transformer encoder. Throughout the training process, the model does not capture modality-specific intramodal information, but, rather, directly interacts cross-modally. While this enables unimodal features to perceive affective information from neighboring modalities upfront, this will lose the modality specific information. EMT splices the unimodal representation as a global representation and selects the Transformer encoder to interact with the global and unimodal representations, but in the process does not model each modality individually, which can result in the model failing to capture affective information that exists within a single modality. We designed a set of experiments to verify the effect of different mechanisms in dual-attention on model performance, as shown in Tables 7 and 8. The first rows of Tables 7 and 8 validate the model performance in the case of using only the unimodal self-attention mechanism, where we used the relevant coefficient to guide the unimodal representations. The final multimodal feature has only three unimodal representations and does not contain the global representation in standard CCDA. The second row verifies the model performance in the case where only the cross-modal attention mechanism is used, in which case the multimodal features are global representations, not containing unimodal representations, and the relevant coefficients cannot be used. Neither of these cases uses the cross-correlation loss. The third row indicates that we use the standard CCDA model for training. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Models** & **MAE (\(\downarrow\))** & **Corr (\(\uparrow\))** & **Acc-5 (\(\uparrow\))** & **Acc-3 (\(\uparrow\))** & **Acc-2 (\(\uparrow\))** & **F1 (\(\uparrow\))** \\ \hline TFN [24] & 0.437 & 0.582 & - & - & 77.1 & 76.9 \\ LMF [25] & 0.438 & 0.578 & - & - & 77.4 & 77.4 \\ MuIT [30] & 0.442 & 0.581 & 40.0 & 65.7 & 78.2 & 78.5 \\ MISA [48] & 0.447 & 0.563 & - & - & 76.5 & 76.6 \\ Self-MM [49] & 0.411 & 0.601 & 43.1 & 66.1 & 78.6 & 78.6 \\ MMIM [51] & 0.422 & 0.597 & 42.0 & 65.5 & 78.3 & 78.2 \\ AMML [50] & 0.437 & 0.583 & 41.2 & 64.2 & 78.0 & 78.1 \\ EMT [33] & 0.396 & 0.623 & **43.5** & 67.4 & 80.1 & 80.1 \\ \hline Ours & **0.393** & **0.628** & 43.3 & **68.3** & **81.1** & **81.0** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Experiments on CH-SIMS. Where \(\uparrow\) indicates that the higher the metric the stronger the performance of the model, and \(\downarrow\) is the opposite. Bold numbers indicate the model with the best results at that metric. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Models** & **MAE (\(\downarrow\))** & **Corr (\(\uparrow\))** & **Acc-7 (\(\uparrow\))** & **Acc-5 (\(\uparrow\))** & **Acc-2 (\(\uparrow\))** & **F1 (\(\uparrow\))** \\ \hline TFN [24] & 0.593 & 0.700 & 50.2 & - & 82.5 & 82.1 \\ LMF [25] & 0.623 & 0.677 & 48.0 & - & 82.0 & 82.1 \\ MuIT [30] & 0.564 & 0.731 & 52.6 & 54.1 & 83.5 & 83.6 \\ MISA [48] & 0.568 & 0.724 & - & - & 84.2 & 84.0 \\ Self-MM [49] & 0.533 & 0.766 & 53.6 & 55.4 & 85.0 & 85.0 \\ MMIM [51] & 0.536 & 0.764 & 53.2 & 55.0 & 85.0 & 85.1 \\ AMML [50] & 0.614 & 0.776 & 52.4 & - & 85.3 & 85.2 \\ EMT [33] & 0.527 & 0.774 & 54.5 & 56.3 & 86.0 & 86.0 \\ \hline Ours & **0.524** & **0.780** & **55.0** & **56.7** & **86.6** & **86.7** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Experiments on CMU-MOSEI. Where \(\uparrow\) indicates that the higher the metric the stronger the performance of the model, and \(\downarrow\) is the opposite. Bold numbers indicate the model with the best results at that metric. The data in the table show that when using self-attention, the model is unable to focus on cross-modal interaction information and only fuses the representations of each modality at a later stage.