Multimodal Causal Reasoning Benchmark: Challenging Multimodal Large Language Models to Discern Causal Links Across Modalities (2025)

Abstract: Multimodal Large Language Models (MLLMs) have showcased exceptional Chain-of-Thought (CoT) reasoning ability in complex textual inference tasks including causal reasoning. However, will these causalities remain straightforward when crucial hints hide in visual details? If not, what factors might influence cross-modal generalization? Whether we can effectively enhance their capacity for robust causal inference across both text and vision? Motivated by these, we introduce **MuCR** - a novel **Mu**ltimodal **C**ausal **R**easoning benchmark that leverages synthetic siamese images and text pairs to challenge MLLMs. Additionally, we develop tailored metrics from multiple perspectives, including image-level match, phrase-level understanding, and sentence-level explanation, to comprehensively assess MLLMs’ comprehension abilities. Our experiments reveal that current MLLMs fall short in multimodal causal reasoning compared to their performance in purely textual settings. Additionally, we find that identifying visual cues across images is key to effective cross-modal generalization. Finally, we propose the **VcCoT** strategy that better highlights visual cues, and our results confirm its efficacy in enhancing multimodal causal reasoning.

## introduction causal reasoning is the process of identifying the relationship between a cause and its effect, which is regarded as a fundamental capability of artificial intelligence @xcite . recent advancements in cot reasoning capabilities of mllms @xcite have driven significant progress in complex analytical tasks, including causal reasoning within the textual modality @xcite @xcite . these developments in-figure 1 : an example from mucr challenges mllms with weather-related causality across two modalities. following this, we propose three key questions: can mllms achieve the same level of causal reasoning comprehension as they do in textual modality? if not, what factors might influence crossmodal generalization? how can we enhance their capacity for robust causal inference? we find that most existing benchmarks fail to address such comparisons or support further exploration in this area. especially, as shown in figure 2 (a), we identify two major drawbacks in previous benchmarks: absence of visual modality: linguistic causal reasoning benchmarks @xcite @xcite @xcite fail to assess visual comprehension ability of mllms. incomplete of cross-modal analysis: most causal reasoning vqa tasks @xcite @xcite neglect cross-modal comparison. recently, some benchmarks @xcite have begun exploring this domain. for instance, blink @xcite examines crossmodal comparisons and conducts basic generalization analyses involving factors like shape and size. as illustrated in figure 2 (b), our proposed mucr comprehensively evaluates causal reasoning at the image, phrase, and sentence levels and offers a multi-faceted analysis of cross-modal generalization that encompasses both visual form factors and semantic elements. moreover, we propose a novel vccot strategy to further enhance cross-modal generalization by improving visual cue perception. we evaluate current state-of-the-art (sota) mllms on our mucr benchmark. experiment results indicate that all models fall short of human performance, particularly in multimodal settings. moreover, they exhibit a pronounced cross-modal gap when discerning causal links across modalities. in addition, we conduct in-depth generalization analysis and demonstrate that visual semantic factors, especially the ability to identify visual cues across siamese images, play a pivotal role. our contributions are summarized as follows: ## the mucr dataset in this section, we detail the construction of the mucr dataset. ## cross-modal generalization analysis and enhancement in this section, we examine the factors that may affect cross-modal generalization. building on previous findings that attribute these gaps primarily to the visual component, we focus on two main categories: visual format factors and visual semantic factors. in addition to investigating these cross-modal generalization factors, we also explore potential enhancement strategies based on our findings. ## conclusion in this paper, we introduce mucr, a novel multimodal causal reasoning benchmark that challenges mllms to discern causal links across different modalities by leveraging synthetic siamese images and text pairs. we also propose comprehensive metrics to assess mllms' understanding from multiple perspectives, including image-level alignment, phrase comprehension, and sentence-level explanation. our experimental results reveal that current mllms exhibit a cross-modal gap in causal reasoning compared to their strong performance in purely textual settings. in-depth analysis highlights that effective visual cue identification is key to enhancing generalization, as mllms often struggle with implicit causal dependencies hidden in visual details. in response, we propose vccot, a method designed to improve visual cue identification for causal inference, with experimental results demonstrating its effectiveness.