# Cmdar: Cross-Modality Dual-Attention Fusion Network for Multimodal Sentiment Analysis
## 4 Experiments
### Ablation Study
#### 4.6.1 Effects of Different Modalities  
We investigate the effect of incorporating acoustic and visual modalities on the proposed model. As shown in Table 5, the removal of both acoustic and visual modalities leads to a noticeable decline in the overall performance of the model. Model performance degradation is more pronounced in the absence of inputs from acoustic and visual modalities. This phenomenon suggests that MSA can analyze and use multimodal information in a complementary way, removing semantic and affective ambiguity. The results above demonstrate that the textual modality provides richer and more accurate affective information than the other two modalities.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **ACC-2\(\uparrow\)** & **ACC-3\(\uparrow\)** & **ACC-5\(\uparrow\)** & **FI\(\uparrow\)** & **MAE\(\downarrow\)** & **Corr\(\uparrow\)** \\ \hline ALMT \({}^{a}\)[2] & 81.19 & 68.93 & 45.73 & 81.57 & 0.404 & 0.619 \\ ConFEDE \({}^{b}\)[28] & 82.23 & 70.15 & 46.30 & 82.08 & 0.392 & 0.637 \\ MMML \({}^{c}\)[24] & 82.93 & 69.37 & 49.38 & 82.90 & 0.332 & 0.732 \\ \hline CMDAF (ours) & 83.82 & 70.42 & 50.11 & 83.85 & 0.325 & 0.723 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison with SOTA: the CH-SIMS dataset demonstrates exceptional performance. Note: \({}^{a}\): results from [2]; \({}^{b}\): results from [28]; \({}^{c}\): results from [24].  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **MAE\(\downarrow\)** & **CORR\(\uparrow\)** & **ACC-7\(\uparrow\)** & **ACC-2\(\uparrow\)** & **FI\(\uparrow\)** \\ \hline LMF \({}^{a}\)[33] & 0.917 & 0.695 & 33.20 & -/82.50 & -/82.40 \\ TFN \({}^{a}\)[13] & 0.901 & 0.698 & 34.90 & -/80.80 & -/80.70 \\ MuIT \({}^{a}\)[11] & 0.861 & 0.711 & - & 81.50/84.10 & 80.60/83.90 \\ MISA \({}^{a}\)[18] & 0.804 & 0.764 & - & 80.79/82.10 & 80.77/82.03 \\ MAG-BERT \({}^{a}\)[15] & 0.712 & 0.796 & - & 84.20/86.10 & 84.10/86.00 \\ HyCon \({}^{a}\)[35] & 0.713 & 0.790 & 46.60 & -/85.20 & -/85.10 \\ Self-MM \({}^{b}\)[34] & 0.708 & 0.796 & 46.67 & 83.44/85.46 & 83.36/85.43 \\ ConFEDE \({}^{b}\)[28] & 0.742 & 0.784 & 42.27 & 84.17/85.52 & 84.13/85.52 \\ MMML \({}^{c}\)[24] & 0.643 & 0.838 & 48.25 & 85.91/88.16 & 85.85/88.15 \\ \hline CMDAF (ours) & 0.606 & 0.862 & 49.29 & 87.31/89.19 & 87.39/89.13 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison SOTA: the MOSI dataset demonstrates exceptional performance. Note: \({}^{a}\): results from [1]; \({}^{b}\): results from [28]; \({}^{c}\): results from [24].  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **MAE\(\downarrow\)** & **CORR\(\uparrow\)** & **ACC-7\(\uparrow\)** & **ACC-2\(\uparrow\)** & **FI\(\uparrow\)** \\ \hline LMF \({}^{a}\)[33] & 0.623 & 0.7 & 48.00 & -/82.0 & -/82.1 \\ TFN \({}^{a}\)[13] & 0.593 & 0.677 & 50.20 & -/82.50 & -/82.10 \\ MuIT \({}^{a}\)[11] & 0.580 & 0.713 & - & -/82.50 & -/82.30 \\ MISA \({}^{a}\)[20] & 0.568 & 0.717 & - & 82.59/84.23 & 82.67/83.97 \\ MAG-BERT \({}^{a}\)[15] & - & - & - & 84.70/- & 84.50/- \\ HyCon \({}^{b}\)[35] & 0.601 & 0.776 & 52.80 & -/85.40 & -/85.60 \\ Self-MM \({}^{b}\)[34] & 0.531 & 0.765 & 53.87 & 83.76/85.15 & 82.17/85.83 \\ ConFEDE \({}^{b}\)[28] & 0.522 & 0.78 & 54.86 & 81.65/85.82 & 82.17/85.83 \\ MMML \({}^{c}\)[24] & 0.517 & 0.791 & 54.95 & 86.32/86.73 & 86.23/86.49 \\ \hline CMDAF (ours) & 0.509 & 0.806 & 54.11 & 86.43/87.72 & 86.41/87.56 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison with SOTA: the MOSEI dataset demonstrates exceptional performance. Note: \({}^{a}\): results from [1]; \({}^{b}\): results from [28]; \({}^{c}\): results from [24].  
An interesting phenomenon is that removing acoustic modalities weakens the performance of the network more than removing visual modalities. This phenomenon also appears in recent work [1]. A plausible explanation for this observation is that the network can more easily extract key emotional information from the acoustic than from the visual modality. In other words, images typically contain more complex noise.