# Causal Inference for Modality Debiasing in Multimodal Emotion Recognition
## 5 Experiments
### Main Results

Table 2: Comparison on the IEMOCAP [31] dataset. † indicates our reimplementation. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Performance**} & \multicolumn{4}{c}{**Performance Drop Gap**} \\ \cline{2-10}  & **{{t,a,v}**} & **{{a,v}}** & **{{t,v}}** & **{{t,a}}** & **{{t}}** & **{a}** & **{v}** & **avg** & **std** \\ \hline MulT + [20] & \(81.8\) & \(\mathbf{-2.1}\) & \(-6.9\) & \(-1.2\) & \(-8.8\) & \(\mathbf{-4.1}\) & \(\mathbf{-9.5}\) & \(\mathbf{-5.4}\) & \(\mathbf{3.5}\) \\ + CausalMER & \(83.4\) & \(-5.5\) & \(\mathbf{-4.4}\) & \(-1.2\) & \(\mathbf{-6}\) & \(-7.8\) & \(-11.8\) & \(-6.1\) & \(\mathbf{3.5}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The performance drop under six possible missing-modality conditions and the full-modality condition (i.e., “[a]” indicates that only the audio modality is available, with both text and video modalities missing) on the IEMOCAP [31] dataset, utilizing the average accuracy metric. Performance drops that are minimal are highlighted in bold. † indicates our reimplementation. When one modality is absent in MulT [20], the drop in performance is more pronounced with the absence of the audio modality than with the absence of the text or video modalities. This is due to MulT [20] being trained dependent on the audio modality. Nonetheless, when integrated with CausalMER, it exhibited a reduced performance drop compared to MulT [20] in the absence of the audio modality, and when the video modality was absent, both demonstrated equivalent performance degradation; however, MulT [20] attained 73% performance, while CausalMER reached 77.4%, thereby achieving superior performance. This demonstrates that CausalMER has effectively eliminated the biases across various modalities. Moreover, although the average performance decline under the missing-modality condition is less pronounced for MulT [20], CausalMER exhibits a higher average performance of 77.3% under the same condition, surpassing MulT's [20] 75.9% by 1.4%. CausalMER outperforms the current MER backbone in scenarios involving modality absence. Table 4 presents the standard benchmarks for the CMU-MOSEI [32] dataset and the performance drop observed when the available modalities are limited, i.e., when certain modalities are missing. We use MulT [20], PMR [23], and DMD [24] as the backbone for CausalMER. Many multimodal models often rely on specific modalities [6], leading to varying performance drops when a modality is missing. For instance, in PMR [23], omitting the video modality resulted in a 2% performance drop, while the absence of the text modality caused an 11.3% decrease--a 9.3% difference. In contrast, when integrating PMR [23] with CausalMER, the video absence led to only a 0.3% drop, and the text absence resulted in a 6.7% reduction--a 6.4% gap, which is 2.9% smaller than the original PMR [23]. This phenomenon occurs because CausalMER calculates the pure modality effect and eliminates modality bias, leading to a more balanced contribution among modalities. As a result, the performance drops are more evenly distributed when certain modalities are absent. This is evident in the standard deviation of performance across modalities. A smaller standard deviation when each modality is unavailable indicates greater robustness to missing modalities. Integrating MulT [20], PMR [23], and DMD [24] with CausalMER reduces the standard deviation compared to their respective backbones, with standard deviations of 3.7, 3.7, and 2.8, respectively. Furthermore, the integration of the DMD [24] with CausalMER shows the minimal standard deviation compared to all other models in