# Causal Inference for Modality Debiasing in Multimodal Emotion Recognition
## 6 Conclusions and Future Work  
**Conclusions. ** CausalMER effectively mitigates modality bias in multimodal emotion recognition, enhancing robustness even when specific modalities are absent. By leveraging causal inference and counterfactual reasoning, it reduces the impact of modality-specific biases without requiring structural changes to existing models. Experimental results demonstrate that CausalMER improves the generalization of MER systems under missing-modality conditions. On the IEMOCAP dataset with the MulT backbone, CausalMER achieves an average accuracy of 83.4%. On the CMU-MOSEI dataset, the average accuracies with MulT, PMR, and DMD backbones are 50.1%, 48.8%, and 48.8%, respectively. These results highlight CausalMER's robustness in missing modality scenarios, as evidenced by its low standard deviation in performance drop gaps. Additionally, our evaluation shows that CausalMER achieves balanced contributions from each modality, effectively mitigating direct biases. CausalMER is particularly suited for real-world applications such as Human-Computer Interaction (HCI), robotics, and healthcare, where real-time emotion recognition is critical. In practical scenarios, sensor failures may lead to missing \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Row** & \(\mathcal{L}_{kl_{hal}}\) & \(NDE_{t}\) & \(NDE_{a}\) & \(NDE_{o}\) & **Acc** & **F1** \\ \hline
1 & ✓ & ✓ & ✓ & ✓ & **83.4** & **83.1** \\ \hline
2 & & ✓ & ✓ & ✓ & 82.0 & 81.9 \\ \hline
3 & ✓ & ✓ & ✓ & & 82.3 & 81.7 \\ \hline
4 & ✓ & & ✓ & ✓ & 81.6 & 81.4 \\ \hline
5 & ✓ & ✓ & & ✓ & 80.9 & 80.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Ablation studies on IEMOCAP dataset. The highest performance is highlighted in bold. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Row** & **Fusion Strategy** & **Acc** & **F1** \\ \hline
1 & MASK & 83.4 & 83.1 \\ \hline
2 & SUM & 72.4 & 74.0 \\ \hline
3 & HM & 71.8 & 73.1 \\ \hline
4 & SUM-tanh & 74.3 & 75.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Impact of different fusion strategies on IEMOCAP [31] dataset. modalities, potentially disrupting predictions if the absent modality has a dominant effect. By eliminating modality bias, CausalMER ensures reliable performance, even in such challenging conditions. **Future work. ** The proposed CausalMER exclusively examines the impact of each modality under full-modality conditions during training. Nevertheless, in different modality-absent scenarios, the modality effect may differ from the full-modality conditions observed. Previous experiments have demonstrated that debiasing derived from the modality effect assessed under full-modality conditions results in strong performance in missing-modality situations. Training the model to account for modality-missing scenarios could enhance its robustness by enabling adaptation to variations in modality effects. Subsequent research will concentrate on broadening this methodology to incorporate these conditions. Conceptualization, Y.C. and J.K.; methodology, J.K.; software, J.K.; validation, J.K. and J.H. ; formal analysis, J.K. and J.H. ; investigation, J.K.; resources, Y.C. ; writing--original draft preparation, J.K. and J.H. ; writing--review and editing, J.K. and J.H. ; visualization, J.K. and J.H. ; supervision, Y.C. ; project administration, Y.C. ; funding acquisition, Y.C. All authors have read and agreed to the published version of the manuscript. This work was partly supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2022-0-00331, Development of emotion recognition/generation-based interacting edge device technology for mental health care, 50%), by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2024-RS-2024-00437494, 25%) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation), and by the faculty research fund of Sejong University in 2024 (25%).