Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning
Authors: Yucheng Wang
Yifan Hou
Aydin Javadov
Mubashara Akhtar
Mrinmaya Sachan
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal large language models
Modality interactions
Multimodal reasoning
Model interpretation
Attention probing
Causal analysis
Task composition
Modality bias
Logic-driven evaluation
Primary area: interpretability and explainable AI

Abstract: Multimodal large language models (MLLMs) promise enhanced reasoning by integrating diverse inputs such as text, vision, and audio. 
Yet, despite their perceptual strengths, their reasoning ability across modalities remains underexplored, with conflicting reports on whether additional modalities help or harm performance. 
These inconsistencies stem from a lack of controlled evaluation frameworks and analysis of models' internals to isolate when and why modality interactions support or undermine reasoning.
We address this gap through a logic-grounded evaluation framework that categorizes multimodal reasoning into six interaction patterns, varying how facts are distributed across modalities and logically combined. 
Empirically, additional modalities enhance reasoning only when they provide independent and sufficient reasoning paths, while redundant or chained entailment support often hurts performance.  
Besides, models recognize cross-modal facts reliably and always reason on text effectively.
Moreover, reasoning degrades in three systematic ways: weaker modalities drag down overall performance, conflicts bias preference toward certain modalities, and joint signals from different modalities fail to be integrated effectively. Therefore, we identify two core failures: task-composition bottleneck, where recognition and reasoning cannot be jointly executed in one pass, and fusion bottleneck, where early integration introduces bias. 
For further investigation, we find that attention patterns fail to encode fact usefulness, but a simple two-step prompting (recognize then reason) restores performance, confirming the task-composition bottleneck.
Moreover, modality identity remains recoverable in early layers, and softening attention in early fusion improves reasoning, highlighting biased fusion as another failure mode.
Overall, our findings show that integration, not perception, is the main barrier to multimodal reasoning, suggesting composition-aware training and early fusion control as promising directions.

Published as a conference paper at ICLR 2026
COMPOSE AND FUSE: REVISITING THE FOUNDATIONAL BOTTLENECKS IN MULTIMODAL REASONING
Yucheng Wang"
Yifan Hou"
Aydin Javadov
Mubashara Akhtar
Mrinmaya Sachan
yuchwang, yifan. hou,
mr crrinmaya.sachin artttr
@inf (einf.eth..c.. ethz.ch
a- htw.adooentee.eehh...ooo
zürich
ABSTRACT
Multimodal large language models (MLLMs) promise enhanced reasoning by inte- grating diverse inputs such as text, vision, and audio. Yet, despite their perceptual strengths, their ability reason across modalities remains underexplored, with conflicting reports on whether additional modalities help or harm performance. These inconsistencies stem from a lack of controlled evaluation frameworks and analysis of models' internals to isolate when and why modality interactions support or undermine reasoning. We address this gap through logic-grounded evaluation framework that categorizes multimodal reasoning into six interaction patterns, varying how factual information is distributed across modalities and logically combined. Empirically, additional modalities enhance reasoning only when they provide independent and sufficient reasoning paths, while redundant or chained entailment support in extra modalities often hurts performance. Besides, models recognize cross- modal facts reliably and always reason on text effectively. More- over, reasoning degrades in three systematic ways: weaker modalities drag down overall performance, conflicts bias preference toward certain modalities, and joint signals from different modalities fail be integrated effectively. Therefore, we identify two core failures: task-composition bottleneck, where recognition and reasoning cannot be jointly executed in one pass, and fusion bottleneck, where early integration introduces bias. For further investigation, we find that attention patterns fail to encode fact usefulness, but a simple two-step prompting (recognize then reason) restores performance, confirming the task-composition bottleneck. Moreover, modality identity remains recoverable in early layers, and softening attention in early fusion improves reasoning, highlighting biased fusion as another failure mode. Overall, our findings show that integration, not perception, is the main barrier to multimodal reasoning, suggesting composition-awe aware training and early fusion control as promising directions
INTRODUCTION
Multimodal large language models (MLLMs) extend traditional language models beyond text incorporate additional modalities such as vision and audio (Li et al. [222c Xu et al. 20255] Yu et al.) 2025 Abouelenin al. 2025). By integrating complementary signals, MLLMs can form richer and more grounded representations of the world. Text offers structured and abstract information, audio encodes temporal and prosodic cues, and images convey spatial and visual context, together capturing facets of meaning that no single modality can express in isolation (Clark & Brennan 1991 Mayer 2002). Through such cross-modal integration, MLLMs aim to overcome the limitations of unimodal systems, enabling more robust understanding, stronger perceptual grounding, and support for more
complex reasoning (Li et al. 2023 Bie et al. 2025 Raza et al. 2025 Coburn et al. 2025).
Equal contribution
'Our code and data are publicly available.

---

Published as a conference paper at ICLR 2026
While MLLMs hold promise for enhanced reasoning by integrating diverse signals, the precise influence of additional modalities remains unclear, especially under complex reasoning scenarios. Existing studies offer conflicting observations: some report that incorporating vision audio can
improve model performance (Li et al. 2023 Guan et al. 2024, Fu et al. 2025). while others suggest that additional modalities introduce interference confusion (Bie al.
2025 He et al. 2025 Hou
et al., 2025). However, these findings are often anecdotal or
lacking unified
framework to systematically assess when and how multimodal input contributes to or undermines
reasoning Guupta et al. 2024 Coburn et al. [025 Hao et al. [2025] Li et al. 2025b Bi et al. 2025).
In particular, it still remains unclear under what conditions additional signals strengthen reasoning.
add little, or actively impede it (Wu et al.] 2025 Zhang et al., 2025). As a result, the role of modality
interaction in reasoning, whether beneficial or detrimental, remains underexplored.
Moreover, most evaluations treat MLLMs as black-box systems, emphasizing external performance while leaving their internal mechanisms poorly understood (Liu et al.) 2024bb [iang et al.) 2023 Li
et al. 2024). Even when empirical patterns emerge, such as degraded reasoning with added modalities, they are rarely accompanied by interpretability analyses that examine how models internally encode modality identity, assess evidence relevance, or perform cross-modal integration (Peng al. 2025
Sinha et al. 2024) Yu et al. [0224] Wadekar et al.) 2024). One contributing factor could lie in how these models are trained: current MLLMS are typically optimized using alignment-style objectives that pair vision or audio with text through paired supervision, contrastive learning. or instruction
tuning (Zhao et al. 2024 Lin et al. 2024 Xie & Wu 2024] Jiang et al., 2025). These objectives prioritize perceptual matching over cognitive composition, reinforcing shallow correlations rather than fostering deeper reasoning. As result, while MLLMs often perform well on perception-heav
tasks (Li et al. 2023 Liu et al., 2024a), they struggle to generalize when reasoning demands flexible integration of multimodal information. Without interpreting the internal representations and fusion behaviors, it remains difficult to pinpoint where these limitations arise or how they might be overcome.
Prior evaluations report mixed effects of adding modalities to reasoning because they rarely control where decision-relevant facts appear or how those facts must be logically combined. We focus on logical reasoning using information from multiple modalities: how models use cross-modal facts to infer answers, rather than simple perception of unimodal content. To make effects measurable, we introduce six canonical interaction types ($ D. grounded in propositional logic, that jointly vary (i) where the crucial facts are placed across modalities and (ii) how those facts must be combined to solve the task: Equivalence (=, redundant encoding), Alternative (V, distinct but individually sufficient paths), Entailment (->, chained support across modalities), Independence (0. single modality carries the relevant fact), Contradictory (#, mutually exclusive conclusions), and Complementary (A, jointly necessary pieces). Instantiated through controlled, synthetic multiple choice reasoning tasks, this framework allows us to assess not only when added modalities that help or hurt, but also why.
To understand when added modalities help or hurt reasoning, we analyze performance across our six interactions. Alternative yields slight gains: added modalities help when they provide independent, individually sufficient reasoning paths. Equivalence offers no benefit in the presence of strong modality (e.g.. text), suggesting that redundant perceptual support rarely improves high unimodal performance. Entailment consistently degrades accuracy, showing that splitting multi-hop reasoning chains across modalities makes inference brittle ($ 3.2). Independence reveals performance bias, where reasoning accuracy heavily depends on which modality carries the decisive fact. Contradictory exposes preference bias, as models could favor certain modalities when inputs conflict. Complemen- tary highlights a weakness in fusion, where models struggle to integrate necessary signals jointly (S 3.2). Moreover, across all settings, text-only baselines approach ceiling and models reliably recognize facts across modalities. Therefore, we identify two core bottlenecks: (i) task composition: models struggle to jointly perform recognition and reasoning when information is split across modal- ities; and (ii) multi-source source fusion: models lack robust mechanisms to select, weight, and combine heterogeneous information, leading to performance, preference, and fusion biases ($ 3.44.
To explain why these bottlenecks arise, we probe the internal behaviors of MLLMs in controlled setting. First, although models recognize facts well and reason effectively (in text), their internal attention patterns fail to encode usefulness (i.e., distinguishing relevant facts from distractors). As a result, accuracy drops when recognition and reasoning must be composed within a single step, revealing a task-composition bottleneck. Explicitly decoupling the two stages through two- step prompt substantially alleviates this issue ($ (T.. Second, models preserve modality identity throughout processing, with the strongest signal concentrated in early decoder layers, but this
2

---

Published as a conference paper at ICLR 2026
Logical Reasoning Example
Fact
Rule
Conclusion
is
Bob
Curious
Curious people purple purple
c) Evaluation Pattern
Audio
Faict
+
Rulles
Input
Model
Output
Three Input Modalities
Figure 1: Multimodal logical reasoning setup and evaluation pipeline. (a) Logical reasoning example: single-step deduction where fact "Bob curious" conclusion "Bob is purple." (b) Modality renderings: the same fact is rendered as text ("Bob is curious"), as audio via ncural TTS, and as schematic visual using graph visualization. (c) Evaluation prompt pattern: the model receives modality-specific fact blocks (text, audio, vision), followed by the rule set and the question with multiple-choice options; the model outputs the predicted answer.
rule "Carious people are purple" entail
preservation leads to biased weighting across modalities. Targeted interventions in these layers (e.g.. softening early attention via increased temperature) significantly improve cross- modal reasoning, whereas modifications at later layers have little effect. This pattern confirms fusion bottleneck, where biased early integration prevents balanced use multimodal evidence ($ 4.2). Together, these insights indicate that additional modalities yield limited and often inconsistent benefits, with failures rooted not in perception but in integration. This calls for models that incorporate composition- training, supervision for evidence selection, and architectural mechanisms for early fusion control, so that extra modalities become assets for reasoning rather than sources of interference.
2 CATEGORIZATION: MODALITY INTERACTION IN REASONING
This section introduces evaluation framework for multimodal logical reasoning. We first describe the general task setup: how facts, rules, and questions are constructed and how facts are rendered across modalities ($ 2.11. We then introduce details of the six canonical interaction types, which specify how useful information distributed across modalities ($ 2.2... These settings allow us to systematically assess whether and how models integrate, ignore, prioritize cross-modal evidence.
2.1 MULTIMODAL LOGICAL REASONING SETUP
Fig. illustrates reasoning task setup. Each instance consists of a of facts, a set of rules (always in text), and multiple-choice question. Facts convey information through different modalities: text, audio, vision, and we control which facts are assigned which modality (Fig.) bb).
(a) Logical reasoning example. To isolate modality interaction, we adopt a simplified single-step reasoning setting inspired by Clark et al. (200)). Full details of the data construction are App. B.1 For example, given the fact "Bob is curious" and the rule "Curious people are purple," the model should infer "Bob is purple." This setup avoids multi-hop complexity and directly tests the model's ability to identify and utilize relevant information across modalities.
(b) Three input modalities. To minimize confounders from low-level perception, we encode each fact in three controlled modalities (see Fig. ():: (i) a short text sentence (e.g., "Bob is curious")) (ii) audio synthesized via neural TTS F and (iii) a schematic visual rendered using graph visualization These controlled renderings ensure interpretability and reduce variability due to acoustic visual complexity, allowing us to focus on reasoning and modality integration.
(c) Evaluation prompt pattern. Each prompt presents randomized set of fact blocks from different modalities, followed by the text-based rule set and a multiple-choice question. To assess robustness and bias, also inject noivy facts (irrelevant distractors) into the input. In the next subsection,
We use Vosce TTS to convert text to speech. We use GraphViz generate simple entity-attrihun diagrams.
3

---

Published as a conference paper at ICLR 2026
define six modality interaction types that vary how decision-relevant facts are distributed, enabling us to test different forms cross tross-modal fusion, redundancy, and conflict.
2.2 INTERACTION TYPES
Equivalence (=), All modalities redundantly encode the same fact (see Fig. in App. B.3 for more details). For example. "Erin is friendly" appears in vision, audio, and text. This tests whether redundancy helps or harms reasoning. revealing models' ability to aggregate repeated evidence.
Alternative (V), Each modality presents different fact, but all facts independently satisfy disjunctive rule (Fig. 5). For example, the rule "Friendly person is clean. Purple person is clean. Red person is clean." is matched by "Erin is friendly", "Erin is person", and "Erin is red", in separate modalities. This setting assesses whether models can leverage distinct but semantically aligned reasoning paths.
(-).. Facts are distributed across modalities to form a multi-hop reasoning chain (Fig. 6) For example, "Erin is bouncy" "Erin is bright" - "Erin is friendly" - "Erin is purple", with each step in a different modality. Only the final fact directly supports the answer. This setting probes models' capacity for chained inference across modalities.
Entailment
Independence (0). Only one modality contains the decision-relevant fact, while the others include unrelated distracting facts (Fig. 7). For example, only the vision modality presents "Erin is friendly." " while text and audio contain irrelevant attributes. This setting tests per-modality reasoning and robustness to irrelevant signals.
Contradictory (+). Each modality leads to different conclusion (Fig. 8). For example, text implies "Erin is clean", vision implies "Bob purple", and audio implies "Erin tasty". This interaction reveals the model's default preference when faced with conflicting evidence across modalities.
Complementary (A), Each modality contributes a fact required for a conjunctive rule (Fig. D.. For instance, "Erin is friendly", "Erin is purple", and "Erin is red', are distributed across modalities, and the rule "If a person is friendly and purple and red, then the person clean." " must be applied. This interaction evaluates the ability to integrate information across modalities for multi-source reasoning.
EVALUATION OF MODALITY INTERACTIONS: PROS AND CONS
We now present systematic evaluation of MLLMs across the six canonical interaction types. This section first outlines our experimental setup, then investigates when multiple modalities help or hurt reasoning, and finally synthesizes key bottlenecks revealed by the results.
3.1 PREPARATION
We begin by describing the models, prompting strategy, decoding procedure, and evaluation metric that together form a controlled testbed for analyzing modality interactions.
Models. We evaluate four recent open-source MLLMs that support at least three input modalities and generate text outputs. Baichuan- Cachhhunn-Omii--e Omni-1.5dl (7B) (Li al. 2025c Baichuan) is designed for
efficient and balanced multimodal reasoning. Qwen2.5- 5-mmi (7B) (Xu et al. 2025 Qwen) is a state-of-the-arr model that handles text, vision, audio, and video, with streaming outputs, serving as strong open baseline for multi-source reasoning. MiniCPPM--22. (8B) (Yao et al. 2024 MiniCPM) processes text, vision, and audio, and optimized for real-time multimodal streaming and on-device deployment. Phi-4 Multimodal (5.6B) (Abouelenin et al., 2025 Phi4) extends the Phi family to vision and audio, emphasizing compactness and efficiency over scale This suite covers both high-capacity and lightweight systems, enabling comparison of modality interaction across diverse architectures.
Prompt design and decoding. We use a unified prompt format across all models to ensure fair comparison. Each prompt includes system instruction, a of fact blocks in random modality order (text, vision, audio), a series of textual reasoning rules, and four-way multiple-choice question
*We refer these models by shortened names in all experiments.
4

---

Published as a conference paper at ICLR 2026
(Fig. [)). To encourage step-by-step reasoning, we insert concise CoT hints while minimizing behavioral interference. Decoding follows HuggingFace defaults with greedy sampling to produce stable outputs, from which the final answer is automatically extracted. All models are evaluated on the same synthetic dataset using identical prompt templates. More details are provided in App. B.3
Evaluation metric. We report accuracy as the primary evaluation metric. We evaluate the reasoning performance by multiple-choice question answering format with four options, and the model's selected answer is automatically extracted from its output response. Since there four options, the random guessing would yield a baseline accuracy of 25%. To ensure robustness, each experiment is conducted on 1,000 synthetic instances per condition.
3.2 Do MULTIPLE MODALITIES HELP REASONING?
One motivation for using MLLMs is to enhance reasoning by incorporating additional information from multiple modalities. But does adding useful input from another modality always help? To explore this, evaluate three controlled interaction types designed to probe distinet modes of cross-modal benefit: redundancy, optionality, and composition. Results are summarized in Tab. with full breakdowns in Apps. C.I C.3
Setup. In Equivalence, the same decisive fact is redundantly placed in all modalities, testing whether repetition reinforces reasoning. In Alternative, each modality contains distinct but individually sufficient fact for solving the problem, allowing multiple independent reasoning paths. In Entailment, reasoning chain (A B + C Answer) is split across modalities, requiring integration of cross-modal premises for successful inference. To assess the added value of multimodal input, we compare each multimodal setting to its unimodal baselines, where only one decisive fact is one modality (text, vision, audio) and the others are omitted. From this comparison we can directly analyze the additional value brought by additional information in extra modalities.
Table 1: Does Multimodality Help Reasoning? Accuracy (%) and performance deltas (A) relative to unimodal baselines across three interaction types: Equivalence (redundant facts across modalities), Alternative (indepen- dent reasoning paths), and Entailment (multi-hop hop chains split across modalities, with final-step facts in V/A/T respectively). Alternative settings slightly boost performance, Equivalence yields marginal decrease (compared to text), while Entailment causes notable accuracy drops reasoning.
Multimodal (=, V, +)
Accuracy (%)
Equivalenceav.AN.r
Alternativeav.AA.ar
Entailment: VAv, Asa. TAT
Baichuan
84.85.4 1,9.6,
97.619.6 17.81,00
79.52.o
75.66.4L
80.713.60
Qwen
98.92.91,466,,099
100.0s.71,611,2277
78.416.4
86.6s.2
83.912.8L
MiniCPM
94.8r.41,,227,2222
99.17.1,80072291
81.811.44
80.012.0
88.46.84
Phi4
84.125.31,23.9t,
97.920.31, 26,3,.3.
73.02.21
69.3p.7u
79.718.0)
Average
10.9,.7 1.77
18.712.77,,488,177
78. 78.27.a1
77.97.1
83.212.80
Findings. In Equivalence, models show marginal gains when the decisive fact is in vision (+9.7%) or audio (+10.9%), but performance drops when the fact is already in text (-5.7%). This suggests that redundancy only helpful when the original modality is weak. In Alternaive,, consistent im- provements are observed across all modalities (+ 1277%% vision, +14.8% audio, + +1.7% text), indicating that semantically independent reasoning paths are successfully leveraged. However, Entailment leads to substantial drops in accuracy across all modalities (-7.8% vision, -7.1 % audio, -12.8% text), highlighting the difficulty of cross-modal multi-hop composition.
Observation 1. Multimodal input improves reasoning only when it contributes additional, se- mantically independent reasoning paths. In contrast, redundant information provides little benefit, particularly when strong modality (text) is already sufficient, and distributing multi-step reasoning chains across modalities often reduces accuracy. These results suggest that the core bottleneck in mudtimodal reasoning lies not in recognizing facts, since individual modalities suffice in many cases.
5

---

Published as a conference paper at ICLR 2026
Real-World World Validation. To assess whether our findings extend beyond controlled synthetic settings, we additionally evaluate the Equivalence interaction on IsoBench (Fu et al. (024)), a real-world vision-text benchmark containing paired representations of the same symbolic content. The results exhibit the same pattern as our synthetic experiments: multimodal (T+V) inputs provide little or no gain strong text-only baselines. Detailed experimental setup and results provided in App.
3.3 Do MULTIPLE MODALITIES HURT REASONING?
While certain forms of multimodal input can aid reasoning ($ adding modalities could also introduce errors. In this section, we evaluate three controlled settings: Independence, Contradictory, and Complementary: to identify specific failure modes where multiple modalities degrade reasoning.
3.3.1 MODALITY PERFORMANCE BIAS: INDEPENDENCE
We first explore whether models exhibit consistent reasoning performance across modalities.
Setup. the Independence setting, a decisive fact appears in one modality (text, vision, or audio), while the remaining modalities contain only distractors. We compare multimodal reasoning, where facts are distributed across modalities, to unimodal baselines, where all facts (both decisive and distracting) are presented within single modality.
shown in Tab. 2 (full results can be found in App. (.4). models perform best in the text-only condition (94.45% on average), but accuracy drops sharply to 70.29% when facts are distributed across modalities. This is well below text-only performance but above vision- only or audio-only baselines, confirming that weaker modal- ities introduce noise when combined with stronger ones.
Findings.
Table 2: Performance the Independence interaction. Fach instance in- cludes decisive fact placed in single modality, while the others contain distractors. Multimodal reasoning accuracy falls between the best (text) and worst (vision) unimodal conditions, suggesting that modality inconsistency introduces error when aggregating information across modalities.
Unimodal
Accuracy (%)
Multimodal (0) AA,
Baichuan
60.2
72.0
94.8
47.67,41;444,,77,,
Qwen MiniCPM
73.3
94.3
95.5
75.21.91.19.1,,20
77.6
83.7
91.2
78.71.1...0..12..1
Phi4
49.9
48.9
96.3
59.79.st.1.84.3000
Average
65.3
74.7
94.5
70.3s.u 24.4
Observation 2. Unequal reasoning capabilities across modalities, what we refer to as performance bias, contribute significantly to degraded multimodal reasoning. When weaker modalities are added, they can dilute confuse the signal from stronger ones, like text.
3.3.2 MODALITY PREFERENCE BIAS: CONTRADICTORY
We next ask whether models exhibit internal preferences for certain modalities.
Setup. the Contradictory setting, each modality provides distinct and individually sufficient reason- ing path, but the answer options are mutually exclu- sive[] This setup exposes which modality model relies on when conflicting information is presented. Crucially, preference here refers selection behavior under conflict, not standalone performance.
Table 3: Performance on the Contradictory inter- action, where each modality leads to different. answer. Models show clear modality preferences, highlighting inconsistent reliance on input sources.
Multimodal (+)
Answer Ratio (%)
A
T
Results. As shown in Tab. 3 models display clear preference patterns: Baichuan favors vision-based answers (49.0%) Qwen tends toward audio (44.6%),
Baichuan
49.0
14.9
33.7
Qwen
17.2
44.6
37.6
MiniCPM
22.6
27.2 .2
49,0
Phi4
31.9
19.1
46.1
and both MiniCPM and Phi4 prefer text (49.0% and
Note that one of the four answer options is always incorrect by design, the sum of selection ratios across modalities does not equal 100%.

---

Published as a conference paper at ICLR 2026
46.1%). These choices are often misaligned with the models' unimodal strengths, suggesting implicit biases in modality selection under conflicting input.
Observation In addition to performance bias, MLLMs also suffer from preference bias: when modalities conflict, models favor certain modalities instead of strong modalities, often inconsistently with the actual performance. This misalignment introduces further risk in multimodal reasoning.
3.3.3 MODALITY FUSION BIAS: COMPLEMENTARY
Finally, we examine whether models can fuse complementary evidence across modalities when each input is necessary for inference.
Setup. In Complementary setting, each modality contains one of three facts that are jointly required to solve the reasoning task. Unlike prior settings with single decisive fact, here all facts must be composed across modalities. We compare this condition to unimodal baselines where all three facts provided within a one modality.
all mod- els perform worse in the mul- timodal setup than in any uni- modal condition, even when confined to weaker modalities like vision. If biased perfor- mance is the only issue, mul- timodal accuracy should between the best and worst unimodal conditions. Instead, distributing complementary facts across modalities intro- duces
Results. In Tab. 4
Table 4: Performance on the Complementary interaction. Each modality provides one necessary fact, requiring integrate all three to get the answer. The multimodal reasoning accuracy is lower than any unimodal condition. indicating that performance degradation stems not only from modality in- consistency but also from true cross-modal composition bottleneck.
Unimodal
Accuracy (%)
Multimodal (A)aV.AA.TT
V
A
T
Baichuan
50.5
59.4
87.7
30. 220.3 29.24,67.5
Qwen
87.5
98.8
98.8
49.937. 6 48.9 45.9
MiniCPM
74.8
89.3
92.4
48.826.04. 40.5 43.64
failure mode: models are unable to compose multiple weak signals into coherent reasoning chain.
Phi4
80.0
82.2
99.6
79.10.91..11,20511
Average
73.2
82.4
94.6
52.021.2 00.44 42.6
Observation MILMS struggle to integrate complementary information across modalities, even when all inputs are individually comprehensible. This reveals that beyond biased performance and preference, there is a third failure mode in multimodal reasoning: fusion bias.
3.4 FROM OBSERVATIONS TO BOTTLENECKS
The preceding evaluations reveal consistent patterns in the way models handle multimodal reasoning. We now synthesize these findings to identify the underlying bottlenecks.
Observation 5. Across all settings, the best performance consistently comes from the text-only baseline, often approaching near-perfect accuracy. This shows that models can already perform logical reasoning reliably when inputs are centralized in single strong modality. Combined with Observation 1 (models can recognize facts across modalities), this indicates that the weakness lies not in perception or reasoning in isolation, but in how these components are combined.
Bottleneck 1: Task Composition. The conjunction of Observation (strong recognition) and Observation 5 (strong unimodal reasoning) points to first core bottleneck: task composition. Models falter when recognition and reasoning must be performed jointly across modalities. That is. while they can detect facts and apply reasoning rules when each task is isolated, performance drops sharply when these steps must be integrated within single inference pass.
Bottleneck 2: Multi-Source Fusion. Observations 2-4 collectively point to second, orthogonal failure point: the inability to fuse information from multiple modalities in a reliable and unbiased manner. First, performance bias (Observation 2) shows that weak modalities dilute reasoning when
7

---

Published as a conference paper at ICLR 2026
mixed with stronger Second, preference bias (Observation 3) reveals that models often favor certain modalities under conflict, even when those modalities underperform in isolation. Third, fusion bias (Observation 4) shows that models fail to integrate complementary information spread across modalities, even when all inputs are individually comprehensible. These findings suggest that MIL.Ms lack robust internal mechanisms for selecting, weighting, and composing evidence from heterogeneous sources, leading to systematic failures in multimodal reasoning.
4 INTERPRETATION: BOTTLENECKS OF MULTIMODAL REASONING
We now investigate the underlying causes of multimodal reasoning failures by probing the internal mechanisms of MLLMs. Guided by the two key bottlenecks identified in $3 task composition and multi-source fusion, we analyze how models represent modality and information usefulness, and whether targeted interventions can mitigate these bottlenecks
4.1 TASK COMPOSITION: ANALYSIS AND IMPROVEMENT
We begin with Bottleneck 1, which concerns the model's inability compose recognition and reasoning across modalities within a single inference step. While MLLMs can perceive facts from different modalities and reason abilities must be integrated. We investigate this bottleneck by analyzing internal attention patterns and exploring whether prompting strategies mitigate the failure.
them in isolation, their performance degrades when these
100
100
Boichuan
80
80
MiniCPM
60
Recognition (MM)
Reasoning (T)
Reasoning (MM)
Two-Step Prompt
Baichuan
Qwen MiniCPM
(a) Probe Acc. (%)
(b) Reasoning Accuracy (%)
Figure 2: Attention probing and reasoning performance. (a) Modality probing for information usefulness shows moderate accuracy, suggesting models cannot clearly distinguish useful from distractor facts. (b) Although models excel in fact recognition and text-only reasoning, their performance drops significantly multimodal reasoning, indicating that the key limitation lies composing recognition and reasoning across modalities.
Probing Information Usefulness. We first assess whether models can internally distinguish useful facts from distractors. linear probe trained on decoder attention distributions: for each modality, compute mean attention over all generated tokens and use these vectors to classify whether each fact is relevant for reasoning (more probing details are in App. B.2). Results (Fig. 2a) show that the attention patterns do not reliably signal semantic usefulness, indicating that the models struggle to prioritize decision-relevant content based on attention alone.
Interpretation Results. To isolate where failures occurs, we conduct two controlled diagnostics: (1) a formal recognition test, where models identify facts across modalities without requiring additional reasoning (see Fig. IO), and (2) two-step prompting setup, where recognition and reasoning are separated across prompts (see Fig. IT). As shown in Fig. 2bb] models perform near- pear-perfectly on recognition and maintain high accuracy on unimodal reasoning. However, when both recognition and reasoning are combined in single multimodal prompt, accuracy drops substantially, confirming that core failure lies in the integration of these two capabilities.
Improving Composition via Two-Step Prompting. In the two-step prompting strategy, models first extracting all facts, then reason over them, substantially improving performance across all models.
'In this section, we conduct our analysis under the Independence setting to avoid multi-hog confounds and ensure interpretability. We select three representative models, Baichuan, Qwen, and MiniCPM, based on their differing modality preferences identified in 3.3.2 which favor vision, audio, and text, respectively.
8

---

Published as a conference paper at ICLR 2026
This shows that the failure arises not from deficiencies in perception reasoning alone, but from their joint composition within a single inference step. By decoupling these processes and providing a more explicit recognition goal, the cosp-oompoiiiion bottleneck is effectively alleviated.
Takeaway. These findings highlight a core weakness in current MLLMs: despite strong perception and reasoning abilities isolation, they lack mechanisms to integrate these steps across modalities. This reflects a broader limitation in training objectives, which emphasize shallow alignment rather than compositional inference. Prompt-level task decomposition offers simple yet effective remedy.
4.2 MODALITY FUSION: ANALYSIS AND IMPROVEMENT
We now address Bottleneck 2, which concerns the model's difficulty in fusing information across modalities. Even when individual facts are recognized correctly, reasoning often fails due to biased or ineffective modality integration. To better understand this bottleneck, we analyze how modality identity is internally represented and whether early fusion contributes to systematic emrors.
Vanilla
Early
0.1
Middle
Lake
100
Head Index
-0022
6 10
80
80
4- 1-10
75
1-10-2
60
70
22
Raichuan
Qwen MiniCPM
Layer Index
0.4
(a) Probe Acc. (%)
(b) Linear probe weights for Qwen
(c) Reasoning Acc. (%) for Qwen
Figure 3: Modality probing based on attention patterns. (a) All models achieve perfect probe accuracy in predicting the modality using attention patterns. (b) For Qwen, linear probe weights show that modality information is primarily captured in first four layers. (c) Attention manipulation different 4 layers (by adjusting head temperature from 0.4 1 1.8), where performance significantly improves in the early layers.
Probing Modality Identity. We assess whether modality type (text, vision, audio) is preserved in the model's 's internal representations. A logistic regression classifier is trained on attention-deriver features: for each input fact, similarly, compute average attention from all generated tokens and flatten this into a feature vector. The classifier predicts the modality of each fact. As shown in Fig. [3a modality identity is perfectly recoverable, indicating that even after fusion, the model maintains strong internal signal of the input modality.
Interpretation Results. To locate where modality fusion occurs, we visualize layer-wise probe weights. F shows that the first four decoder layers carry the strongest modality signal, suggesting that fusion predominantly occurs early the language module. Beyond this point, modalities appear to be processed more uniformly.
Improving Fusion via Attention Manipulation. Motivated by this early-fusion pattern, we modify attention behavior by adjusting the softmax temperature from 0.4 to 1.8 in the first four decoder layers (Farly), with the default set to 1.0. As shown in Fig.3c this simple intervention of increasing the temperature for early layers yields significant improvements in reasoning accuracy by encouraging more balanced attention across modalities. In contrast, adjusting the temperature in middle or late layers has little effect, supporting the causal role of early fusion in downstream reasoning outcomes.
Takeaways. These findings confirm that while modality identity is well preserved, early-stage fusion introduces systematic biases that impair reasoning. lightweight causal intervention, reshaping early attention distributions, can significantly enhance multimodal integration, highlighting the importance of fusion dynamics over perceptual bottlenecks.
Summary of Interpretation Findings. Across above interpretation analyses, find that failures in multimodal reasoning stem not from deficiencies perception or unimodal reasoning, but from weak compositional integration and crosy-modal fusion. Although MI.LMs retain modality-specine

---

Published as a conference paper at ICLR 2026
signals and can recognize facts reliably, they often default to shallow alignment behavior rather than selective integration. Without architectural biases or training objectives that explicitly encourage multimodal composition, these models remain brittle in complex reasoning tasks.
RELATED WORK
Multimodal Large Language Models (MLLMs). Recent MLLMs extend text-only LLMs to process images, audio, and video. End-to-end "omni" systems such as Qwen2.5-0mmi (Xu et al. 2025), MiniCPM-o-2.6 (Yu al. 2025), and Baichuan-Omn' (Li et al. 2025e) integrate multiple modalities within unified encoder encoder-decoder pipelines, while adapter-basse based approaches (e.g., Phi-4 Multimodal (Abouelenin et al., 2025)) extend compact LLMs with podality-speciffe modules. Other efforts emphasize scaling and long-context multimodal reasoning (e.g., Gemini (Reid et al. (022)) LLaVA-OneVision (Li et al., 2025a),, VILA-1.5 (Lin et al., (024)). Despite rapid architectural progress, understanding how modalities interact during reasoning remains limited.
MLLM Evaluation and Reasoning. A growing benchmark ecosystem probes MLLM capabilities. General-purpose suites such as MMBench (Liu al., 2024b), MME (Fu et al. 2023), and SEED- Bench (Li et al., (2023) measure broad multimodal competence, while reasoning focusee datasets like MMMU (uue al. 2024) stress expert-level multimodal problem solving. Robustness-oriented benchmarks, including POPE (He et al., (025) and HallusionBench (Guan et al., 2024), reveal hallucination and modality conflicts. However, these evaluations typically conflate perception and reasoning. Recent work highlights that MLLMs are often stronger at recognition than structured reasoning (e.g., VERIFY (Bi (025), STARE eL et al. (0255), POLYMATH (Gupta et al.) (0224) EMMA (Hao al. 2025)). Our work explicitly separates fact recognition, modality recognition, and reasoning to isolate interaction effects.
Information Interaction and Modality Dominance. Several studies implicitly examine cross- modal logical relations. Visual entailment tasks (Thomas et al., 2022) and qualitative taxonomies of redundancy and synergy (Sinha et al. 2024) relate to interaction patterns such as equivalence and complementarity. MMOE Yu et al. 2024) models redundancy and synergy via expert routing, while architectural analyses explore implicit fusion strategies (Wadekar et al. 2024). Concurrently, research on modality dominance shows that text often overwhelms other modalities due to architectural and
scaling biases (Wu et al. 2025| Zhang et al. 2025) Haouhat et al. 2023). motivating fusion control mechanisms (Shihata 2025 Lin & Hu 2023
Our Position. In contrast to prior work that measures overall capability or studies fusion implicitly, we define six fine-grained logical interaction types (independent, equivalence, alternative, entailment, complementary, contradictory) and embed them into controlled reasoning tasks. This enables systematic analysis of when multimodal inputs help, hurt, or remain redundant, and allows us to directly diagnose task-composition and fusion bottlenecks in multimodal reasoning.
CONCLUSION
This work presents systematic study of how MLLMs integrate information across modalities for logical reasoning. We introduce a logic-driven evaluation framework with six canonical interaction types, enabling controlled analysis of when additional modalities help hinder reasoning. Our results reveal a consistent pattern: modalities help only when they contribute independent, sufficient reasoning paths, while redundancy or cross modal chaining often degrades performance. Text-only baselines already approach ceiling accuracy, underscoring that the key bottleneck is not perception but integration. Through probing and causal interventions, we identified two core bottlenecks: a task-composition bottleneck, where recognition and reasoning cannot be reliably combined in single inference step, and a fusion bottleneck, where early-layer integration introduces modality bias. Simple remedies such as two-step prompting and attention temperature adjustments alleviate these issues, highlighting the importance of designing models and objectives that explicitly support evidence selection and unbiased fusion. We hope our framework and findings inspire future work toward composition-awee aware training and architecture choices that transform added modalities from sources of interference into assets for reasoning.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This research does not involve human participants, private or sensitive data, or applications with foresecable negative societal impact. All datasets employed are publicly available and widely used within the vision-language and reasoning research communities. We adhere to standard best practices in data handling, model evaluation, and reporting, and our study complies fully with the ICLR Code of Ethics.
REPRODUCIBILITY STATEMENT
We have taken care to make our work reproducible. The paper and appendix provide detailed descriptions of the models, datasets, and experimental setups. Code, data generation scripts, and evaluation protocols are included in the supplementary material and will be released publicly upon publication to facilitate replication and further research.

---

Published as a conference paper at ICLR 2026
REFERENCES
AbdeIrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congeong Chen, Dong Chen, Dongdong Chen, Jun-Kun Chen, Weizhu Chen, Yen-Chun Chen, Yi-ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao, Amit Garg, Abhishek Goswami, Junheng Hao, Amr Hendy, Yuxuan Hu, Xin Jin, Mahmoud Khademi, Dongwoo Kim, Young Jin Kim, Gina Lee, Jinyu Li, Yunsheng Li, Chen Liang, Xihui Lin, Zeqi Lin, Mengchen Liu, Yang Liu, Gilsinia Lopez, Chong Luo, Piyush Madan, Vadim Mazalov, Arindam Mitra, Ali Mousavi, Anh Nguyen, Jing Pan, Daniel Perez-Becker, Jacob Platin, Thomas Portet, Kai Qiu, Bo Ren, Liliang Ren, Sambuddha Roy, Ning Shang, Yelong Shen, Saksham Singhal, Subhojit Som, Xia Song. Teryana Sych, Praneetha Vaddamanu, Shuohang Wang, Yiming Wang. Zhenghao Wang. Haibin Wu, Haoran Xu, Weijian Xu, Yifan Yang, Ziyi Yang. Donghan Yu, Ishmam Zabir, Jianwen Zhang. Li Lyna Zhang, Yunan Zhang, and Xiren Zhou. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture- loras... CoRR, abs/2503.001433, 2025. doi: 0044850/ARXXIV2553.0011 URL htt.thtt///ooooo org 41w.11..4555/B80 art.v/riiiivv. 2503. 01743
Jing Bi, Junjia Guo, Susan Liang, Guangyu Sun, Luchuan Song, Yunlong Tang. Jinxi He, Jiarui Wu, Ali Vosoughi, Chen Chen, and Chenliang Xu. VERIFY: A benchmark of visual explanation and reasoning for investigating multimodal reasoning fidelity. CoRR, abs/2503. 2025. doi: 10. 48550/ARXIV.2500. 1557. URL htLLhttp::////
org 10. 48550/ /ar.vi//aaxvv..// 2503 11557
Fuqing Bie, Shiyu Huang. Xijia Tao, Zhiqin Fang, Leyi Pan, Junzhe Chen, Min Ren, Liuyu Xiang, and Zhaofeng He. Omniplay: Benchmarking omni-modal models on amni-modal game playing. 2025. URL https:: // arxiv org/ abs /2088. 04361
Herbert H. Clark and Susan E. Brennan. Grounding in communication. In Lauren B. Resnick, John M. Levine, and Stephanie D. Teasley (eds.), Perspectives socially shared cognition, pp.
127-149. American Psychological Association, 1991. doi: 10.1037/10096-000 006. URL https
//doi.org// 10000.000//0001077/000----
Peter Clark, Oyvind Tafjord, and Kyle Richardson. Transformers soft reasoners over language. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 3882-3890. ijcai.org, 2020. doi: 10.24963/JCCA1222005577
URL http::ttp//// //tp::://dooio wrgpi.o.g/lii 24963/ htt.aaaaa.....a....... 2020/537
Bruce Coburn, Jiangpeng He, Megan F. Rollo, Satvinder S. Dhaliwal, Deborah A. Kerr, and Fengqing Zhu. Evaluating large multimodal models for nutrition analysis: A benchmark enriched with
contextual metadata. CoRR, https/550.00040.. 2025. doi: 10.45550AAXX..255000770... URL
https://doi. org/10 11888550/araia 2507 07048
Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al. Cosyvoice Scalable streaming speech synthesis with large language models. arXiv preprint arXiv:2412.1011
2024.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. MME: A comprehensive evaluation benchmark for multimodal large language models. CORR, abs/2306. (ttp/2230,1133399 13394, 2023. doi: 10.48550/ ARXIV.2306. 13394. URL https:ttt:://// doi. www. 48550488550xii//
13394
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Caifeng Shan, Ran He, and Xing Sun. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal 1lms in video analysis. In IEEF/CVF Conference on Computer Vision and Patfern Recognition, CVPR 2025, Nashville, TN, USA, June I 11-55, 2025, pp. 24108-241 -218141 Computer Vision Foundation IEEE, 2025. doi: 10. www.10///VVR522344.20 202500225... content/ www.eeett///RR22222 RCCCV//222222//2225222 Evaluation http_/on__eehharrrcccc Sattscoofo_Muuiiie modal LLMs in CVPR_ 20122220522222 2011-__pperr.. html
02244.2255422 URL https: htts::///pppaccsss.o thecvf. com/
Video MME The First Eeeesstt-verr..e Comprehensive
Deqing Fu, Ruohao Guo, Ghazal Khalighinejad, Ollie Liu, Bhuwan Dhingra, Dani Yogatama, Robin Jia, and Willie Neiswanger. Isobench: Benchmarking multimodal foundation models on isomorphic representations. arXiv preprint arXiv:2404.01266, 2024.
12

---

Published as a conference paper at ICLR 2026
Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusion- bench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In /EEF/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024 pp. 14375-14388 IEEE, 2024. URL /ttp:tts::// ottp://///oo..////./.. 11t.:1...11199..111
doi: 10. 1w/.///VV227...0200000000 2024. 01363
Himanshu Gupta, Shreyas Verma, Ujjwala Anantheswaran, Kevin Scaria, Mihir Parmar, Swa- roop Mishra, and Chitta Baral. Polymath: A challenging multi-modal mathematical reason- ing benchmark. CoRR, abs/2410. 14702, 2024, doi: 1048550/ARXX:2441144700 URL
/00rg/10.4855000 48540/5503330 arxiv 2410 14702
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang. Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? EMMA: an enhanced multimodal reasoning
benchmark. CoRR, abs/2501.05444 2025. doi: (0.48550/ARXIV22500 2504000002200..0444....4 URL https
//doi.org/10 48550/ http://aaxivv.......... 2ww. 05444
Abdelhamid Haouhat, Slimane Bellaouar, Attia Nehar, and Hadda Cherroun. Modality influence in multimodal machine learning. CoRR, abs/2306. 06476, 2023. doi: 10.44550/AARXIV22300..444
URL / www.oorg....... 48550/ a 2306 06476
Yixiao He, Haifeng Sun, Pengfei Ren, Jingyu Wang, Huazheng Wang, Qi Qi, Zirui Zhuang, and Jing Wang. Evaluating and mitigating object hallucination in large lisounlanguaaee models: Can they still see removed objects? In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations the
the Americas Chapter of the Association for Computational Human Language Technologies, NAACL 2025 Volume I: Long Papers, Albuquerque, New Mexico, USA, April 29 May 4, 2025, pp. 6841-68855 1-6858. Association for Computational Linguisties, 2025. doi: 10.
Linguistics:
URL http:::tt:///e /doii org/10
18653/v1
349
Yifan Hou, Buse Giledereli, Yilei Tu, and Mrinmaya Sachan. Do vision-language models really understand visual language?, 2025. URL https::::/// /ectps://arxiv wrw. at/////s222110 00199
Songtao Jiang. Yan Zhang, Ruizhe Chen, Tianxiang Hu, Yeying Jin, Qinglin He, Yang Feng, Jian Wu, and Zuozhu Liu. Modality- fair preference optimization for trustworthy milm alignment, 2025.
URL https: httthhtts://aallit //tp:v/aaavvvv//aa org/ wwts///10000 15334
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li. Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. Trans. Mach. Learn. Res., 2025, 2025a. URL stttss://openreveww.nett zKv8qULV6n
Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Bench- marking multimodal lims with generative comprehension. CoRR, abs/2307.16125, 2023. doi: 10. 48550/ARXII 11.50AAARXV.230..61122
2307. 16125
Linjie Li, Mahtab Bigverdi, Jiawei Gu, Zixian Ma, Yinuo Yang. Ziang Li, Yejin Choi, and Ranjay Krishna. Unfolding spatial cognition: Evaluating multimodal models visual simulations. CoRR,
abs/2506.004333, 2025b. doi: 10.48550AARXI.2550..4666
URL
48550/arXiv 200....50666555.
Yadong Li, Jun Liu, Tao Zhang, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, Chong Li, Yuanbo Fang, Dongdong Kuang, Mingrui Wang, Chenglin Zhu, Youwei Zhang, Hongyu Guo, Fengyu Zhang, Yuran Wang, Bowen Ding, Wei Song, Xu Li, Yuqi Huo, Zheng Liang. Shusen Zhang, Xin Wu, Shuai Zhao, Linchu Xiong, Yozhen Wu, Jiahui Ye, Wenhao Lu, Bowen Li, Yan Zhang, Yaqi Zhou, Xin Chen, Lei Su, Hongda Zhang, Fuzhong Chen, Xuezhen Dong, Na Nie, Zhiying Wu, Bin Xiao, Ting Li, Shunya Dang, Ping Zhang, Yijia Sun, Jincheng Wu, Jinjie Yang, Xionghai Lin, Zhi Ma, Kegeng Wu, Jia Li, Aiyuan Yang, Hui Liu, Jianqiang Zhang, Xiaoxi Chen, Guangwei Ai, Wentao Zhang, Yicong Chen, Xiaoqin Huang. Kun Li, Wenjing Luo, Yifei Duan, Lingling Zhu, Ran Xiao, Zhe Su, Jiani Pu, Dian Wang, Xu Jia, Tianyu Zhang, Mengyu Ai, Mang Wang, Yujing Qiao, Lei Zhang, Yanjun Shen, Fan Yang.
13

---

Published as a conference paper at ICLR 2026
Miao Zhen, Yijie Zhou, Mingyang Chen, Fei Li, Chenzheng Zhu, Keer Lu, Yaqi Zhao, Hao Liang, Youquan Li, Yanzhao Qin, Linzhuang Sun, Jianhua Xu, Haoze Sun, Mingan Lin, Zenan Zhou, and Weipeng Chen. Baichuan-omni- technical report. CoRR, abs/2501.15368, 2025c. doi: 10. 48550/ARXIV.25011 15368. URL htt.t:tt////to //ooi org/10.g100. wtlgg/...45550 08550/45505350 2501, 15368
Yizhi Li, Ge Zhang. Yinghao Ma, Ruibin Yuan, Kang Zhu, Hangyu Guo, Yiming Liang, Jiaheng Liu, Jian Yang. Siwei Wu, Xingwei Qu, Jinjie Shi, Xinyue Zhang. Zhenzhu Yang. Xiangzhou Wang. Zhaoxiang Zhang, Zachary Liu, Emmanouil Benetos, Wenhao Huang, and Chenghua Lin. Omnibench: Towards the future of universal omni-language models. CoRR, abs/2409. 15272, 2024. doi: 10.48550/ARXIV224099 15272. URL htttps://doi.ore http:://ool.ooo1...45553355 18550.4.8550000 http855//aw.ii... 15272
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong. Hongyu Ren, Huaxiu Yao, Jue Wang. Keshav Santhanam, Laurel J. Orr, Lucia Zheng, Mert Yüksckgönüil, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang. Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. Holistic evaluation of language models. Trans. Mach. Learn. Res., 2023, 2023. URL net
/ktps://openrestt
Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. VILA: on pre-training for visual language models. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. 186679266889 IEFF, 2024. doi: 10.1 D0.1199/CVRS5233320024444 2022 URL httpt:://///mm. /ww. org 10. 1109/CVPR52733 2024. 02520
Ronghao Lin and Haifeng Hu, Missmodal: Increasing robustness to missing modality in multimodal sentiment analysis. Trans. Assoc. Comput. Linguistics, 11:1686-1702. 2023. doi: 10.1162/TACL
_AL00628. htRhhtts::00
11w2/ a 00628
Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang. and Yu Qiao. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models. In Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), Computer Vi- sion ECCV 2024 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LVI, volume 15114 of Lecture Notes in Computer Science, pp. Springer, 2024a. doi: 10. 978-3- 000-0001199----1 22
22.
org 10. 10077 1007/
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your mullti-moda model an all-around player? In Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), Computer Vision ECCV 2024- 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part VI, volume 15064 of Lecture Notes in Computer Science, pp. 216 233. Springer, 2024b. doi: 10. 10010099778-300 1188--3031--76555- 13. URL https:://dii or orgg C. 1007/0788 1-77778--01-1 02133117765833.1
Richard Mayer.
Multimedia learning. In Psychology of learning and motivation, volume 41, pp- 85-139. Elsevier, 2002.
Jing Peng, Yucheng Wang, Bohan Li, Yiwei Guo, Hankun Wang, Yangui Fang, Yu Xi, Haoyu Li, Xu Li, Ke Zhang, al. A survey on speech large language models for understanding. IEEE Journal of Selected Topics in Signal Processing, 2025.
Shaina Raza, Ashmal Vayani, Aditya Jain, Aravind Narayanan, Vahid Reza Khazaie, Syed Raza Bashir, Elham Dolatabadi, Gias Uddin, Christos Emmanouilidis, Rizwan Qureshi, and Mubarak Shah. Vidbench evaluating multimodal disinformation with regulatory alignment, 2025. URL
hittps://awe wwt.:/oggggabsaeeee 01366012..111.2
14

---

Published as a conference paper at ICLR 2026
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap. Jean- Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou. Orhan Firat, Julian Schrittwieser, loannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, James Molloy, Jilin Chen, Michael Isard, Paul Barham, Tom Hennigan, Ross Mcllroy, Melvin Johnson, Johan Schalkwyk, Eli Collins, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Clemens Meyer, Gregory Thornton, Zhen Yang, Henryk Michalewski. Zaheer Abbas, Nathan Schucher, Ankesh Anand, Richard Ives, James Keeling, Karel Lenc, Salem Haykal, Siamak Shakeri, Pranav Shyam, Aakanksha Chowdhery, Roman Ring, Stephen Spencer, Eren Sezener, and et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens doi: 10445500AXXX.2240005555 URL https 485544555//aa. 055500
of context. CoRR, abs/2403.05534 0350000000000. 2024,
Yusuf Shihata. Gated recursive fusion: A stateful approach to scalable multimodal transformers. org/
CoRR, hbw/2550.0029s. 2025. doi: 10.48550/ARXIV.550..2 022995. URL
10. 00.48855aaxii10
2507.02985
Anoop K. Sinha, Chinmay Kulkarni, and Alex Olwal. Levels of multimodal interaction. In Hayley Hung, Catharine Oertel, Mohammad Soleymani, Theodora Chaspari, Hamdi Dibeklioglu, Jainendra Shukla, and Khiet Truong (eds.). Companion Proceedings of the 26th International Conference on Multimodal Interaction, ICMI Companion 2024, San Jose, Costa Rica, November 4-8, 2024, pp. 51-55. ACM, 2024. doi: 10.1 141114/66621156660000 3686215. 36901533
URL. htttp::::/// /doi: .rr 1145/
Christopher Thomas, Yipeng Zhang, and Shih-Fu Chang. Fine-graine grained visual entailment. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022 17th European Conference, Tel Aviy, Isruel, October 23-27, 2022, Proceedings, Part XXXVI, volume 13696 of Lecture Notes in Computer Science, pp. 398-416. Springer, 2022. doi: 10.100779788--031-2000 978 3- 031 20059-5
23. URL
1007/
Shakti N. Wadekar, Abhishek Chaurasia, Aman Chadha, and Eugenio Culurciello. The evolution of multimodal model architectures. CoRR, abs/2405. 17927, 2024. doi: 10.48550/ARXIV.24000 17927. arxiv 2405. 17927
URL p htt.htttps///ol..r////////
Huyu Wu, Meng Tang, Xinhan Zheng, and Haiyun Jiang. When language overrules; Revealing text
dominance in multimodal large language models, 2025. URL
2508. 10552
Zhifei Xie and Changqiao Wu. Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities. CoRR, abs/24100.11999 2024. doi: 10.4850/ARXI..211.1111 URL
https::/doi.orgg
48550/ arxiv. 2410.
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang. Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report. CORR, abs/2503 20215, 2025. doi: 10.44550/ARXX1.25555222 20215.. URL https
20215
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408.n 01 800, 2024.
Haofei Yu, Zhengyang Qi, Lawrence Jang, Russ Salakhutdinov, Louis-Philippe Morency, and Paul Pu Liang. Mmoe: Enhancing multimodal models with mixtures of multimodal interaction experts. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.). Proceedings of the 2024 Conference on Empirical Methods in Natural Lunguage Processing, EMNLP 2024, Miami, FI, USA, November 12-16, 2024, pp. 10006- 10030. Association for Computational Linguisties, httptt/ps//.....oo/oooooooo.
2024. doi:
v1 v1/2024. emnip caammmpppmain.... 558
Tiayyu Yu, Haoye Zhang, Qiming Li, Qixin Xu, Yuan Yao, Da Chen, Xiaoman Lu, Ganqu Cui, Yunkai Dang, Taiwen He, Xiaocheng Feng, Jun Song, Bo Zheng, Zhiyuan Liu, Tat-Seng Chua,
15

---

Published as a conference paper at ICLR 2026
and Maosong Sun. RLAIF-VE V: open-source Al feedback leads to super GPT-4V trustworthiness. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN. USA, June 11-15, 2025, pp. 19985-19995. Computer Vision Foundation IEEE, 2025. doi: 10.1 1099/C/PR527344223...0 861. URL htt.tttts://ttooto pttp:tttss/////eacccessss tww.wf..h..ccccccmm com/ cttp///onttt//////m
CVPR2025/html - Open Open- Feercee Btaroaaac___eed_tt___________
GPT 4V Trustworthiness CVPR 2025 1r12_ppaerr.hta html
Xiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. 9556 9567. IEEE, 2024. doi: 00.1099/VVPR52233.22224 00913. URL http:: doi org 10. 11t.:119//C00 CVPR52 733 ....3..22222222 2024 00913
Yu Zhang, Jinlong Ma, Yongshuai Hou, Xuefeng Bai, Kehai Chen, Yang Xiang, Jun Yu, and Min Zhang. Evaluating and steering modality preferences in multimodal large language model. CoRR, 00022255..09977 2025. doi: 00448550/AXXIV22552220777 URL htttps://ao..rrg// 48550/arxiv www.exxiii......... 20977
Fei Zhao, Taotian Pang, Chunhui Li, Zhen Wu, Junjie Guo, Shangyu Xing, and Xinyu Dai. Aligngpt: Multi-modal large language models with adaptive alignment capability, 2024.
16

---

Published as a conference paper at ICLR 2026
A RELATED WORK
Multimodal Large Language Models (MLLMs). Recent MLLMs extend text-only LLMs to handle images, audio, and often video, with two broad design trends. On the end-to-end omni side, with streaming I0 via block-wise encoder
Qwen2.5-Omnn processes wwwvvviiionauiiovviiii stack and a Thinker-Talkee architecture for joint text/speech generation, achieving state-of-the-art results on omni benchmarks (Xu al., 2025). MiniCPM-o-2.6 follows similarly integrated pipeline (SigLIP + Whisper + CharlTSS + Qwen2.5 backbone) to enable real-time speech and multimodal live streaming on resource-constraaied devices (Yu et al. scheme, multimodal alignment, then multitask fine-tuning
2025). Baichuan-Omn proposes a two-stage to support concurrent inputs of image, 2025c). In contrast, adapter- approaches
audio, video, and text in an open 7B model (Li et al. extend a compact LLM with modality-specinf speciffc LoRA routers (e.g.. Phi-4 Multimodal), delivering strong vision-audio performance while retaining efficiency (Abouelenin et al. 2025). Beyond open models, frontier systems like Gemini emphasize extremely long multimodal context across documents, video, and audio (Reid et al. 2024). Complementary open MLLM lines (LLa LLLaVA-One' VAA-A-----iionnnnn (Li et al. (2025a), VILA-1.5 (Lin et al. 2024h) focus on unifying single-image, multi-image. and video scenarios and scaling families across 3B 404 parameters, respectively. Finally, efforts like Mini- 0mni2 target GsT-lo-style, real-time visual- audio assistants, underscoring the field's push toward unified, low-latency multimodal interaction (Xie & Wu (202).. Recent surveys also review progress in Speech LLMs, formally defining speech understanding and analyzing Speech-LLM architectures, training, and evaluation through a structured taxonomy (?).
MLLM Evaluation. A rapidly growing ecosystem of benchmarks now probes different facets of the MLLM ability. General-purpose suites such as MMBench (Liu et al. 2024b), MME (Fu et al.)
2023), and SEED-Bench (Li al., 2023) emphasize breadth and stable, objective MCQ evaluation across perception and language understanding, including text-rich images and video. Reasoning- centric datasets like MMMU push models toward college-level, multi-disciplinaryy problem solving with heterogeneous visual artifacts (charts, diagrams, tables), exposing large gaps in expert-level multimodal reasoning Yuue et al.] 20244. Complementing capability scores, robustness-focused evaluations, including POPE (He et al. (025)) for object hallucination and HallusionBench (Guan ct al. 2024) for language-vs-visood conflicts, and visual illusions-diagnose systematic failure modes and modality biases that broad benchmarks can obscure. Beyond static images, Video-MME (Fu et al., 2025) targets temporal understanding, while MM- SafeyyeeyBenchh (Liu et al. (2024) stress-tests safety in multimodal settings. These lines reveal pattern: While breadth benchmarks track steady gains, targeted diagnostics consistently uncover modality dominance, hallucination, and fusion brittleness, motivating frameworks (like ours) that isolate and measure information interaction in reasoning.
Recognition vs. Reasoning. While many benchmarks and papers demonstrate that MLLMS adept at recognizing objects, attributes, diagrams, or patterns, few dissect how well these models reason with those recognized elements, especially when evidence must be drawn from multiple modalities. Benchmarks like VERIFY (Bi et al.) 2025) highlight that perception (recognition of visual input) is often less challenging for MLLMS than reasoning, models often fail when inference, explanation, or abstract relationships are required. STARE (Li et al.) (025b) similarly reveals that while simple spatial or 2D transformations can be handled, tasks that need multistep visual simulation or 3D understanding are far more error-prone. POLYMATH (Gupta et al. 2024) and EMMA (Hao ctt al.] 2025) further push into domains (math, diagrams, cross-mode reasoning) where recogmition alone insufficient; these tasks expose gaps when models must combine interpret recognized information rather than just identify it. Our work builds on this line by explicitly separating and quantifying fact recognition, modality recognition, and reasoning performance, and by introducing interaction types (e.g., independence, entailment, complementary) to isolate how modalities help or hurt when reasoning required.
Information Interaction and Logical Relations. A number of studies have implicitly or explicitly considered how information from multiple modalities interacts, especially in logical or semantic rea- soning contexts. For example, Thomas et al. (2022) asks whether textual hypotheses entail, contradict, or are neutral with respect to images, and breaks down hypothesis claims, which correspond roughly to "entailment" and "contradiction" " in our framework. Meanwhile, Sinha et al. (2024) proposes qualitative taxonomy that includes redundancy and synergy between modalities, similar to our
17

---

Published as a conference paper at ICLR 2026
"equivalence" and "complementary" types, but without formal logical rules or controlled tasks to isolate their effects. The recent MMOE work Yu et al., (024) directly trains separate experts for redundancy, uniqueness, and synergy, which is closely aligned with some of our interaction types (independent, alternative, complementary), yet still coarser. Architectural analyses (Wadekar et al. 2024) show different fusion strategies (early vs late cross-attention) that implement interactions implicitly rather than measuring them via logical operators. Our work builds on and extends this prior art by defining six fine-grained logical interaction types (independent, equivalence, alternative, entailment, complementary. contradictory) and embedding them into synthetic reasoning tasks with rules, facts, and contrastive answer options. This allows us to evaluate not just whether models fuse modalities, but also how and when different interaction patterns boost or degrade reasoning performance.
Fusion Mechanisms and Modality Dominance. Research on fusion architectures and modality dominance has rapidly advanced, revealing how design choices often tip the scale in favor of one modality, typically text. For instance, WW et al. (2025) introduces quantitative metrics to measure how strongly models depend text, showing that fusion architectures and token redundancy play key roles in producing text dominance. Similarly, [Zhang et al. (2025) analyze modality dominance in MLLMs and attribute imbalances in scaling, alignment, and representation, showing that text often overwhelms other modalities not only due to token abundance, but also architectural biases. Their controlled ablations further highlight that dominance emerges from systemic training choices rather than dataset artifacts. Meanwhile, Haouhat et al. (2023) uses masked-modalin ablation across tasks (sentiment, emotion, disease detection) to confirm that, in many settings, non-text modalities add little when text is present. Architectural solutions have been proposed: [Shihata (2025) implements gating and sequential fusion to control how each modality contributes. [Lin Hul (2023) further show that handling missing modalities and aligning fcature/geometrric spaces can reduce dominance bias. Our work links to this by not only observing modality dominance in reasoning tasks, but also by dissecting when and why dominance arises via logical interactions and attention probing. We contribute by explicitly measuring both performance (modality competence) and preference (which modality is used when conflicting or combined), under controlled logical reasoning settings.
B SUPPLEMENTARY SETUP
B.1 FACT CONSTRUCTION
We construct facts and rules following the format of Clark et al. (2020). using the data generation code from Liang al. 2023 Below we describe the details fact generation and rule generation.
Fact generation. Each fact consists of subject, a predicate, and an object. The predicate is always the copula "is"; variability comes from the choice of subject and object. To keep the setting interpretable and avoid conflicts with commonsense priors, we prevent antonyms or synonyms from appearing together within the same instance. Details are as follows:
Subjects. A subject is randomly sampled from three categories:
- Persons (13 names): Alice, Bob, Carol, Dan, Erin, Frank, George, Harry, Iris, Jack, Kevin, Lance, Miller.
Animals (14 types): dog, cat, rabbit, mouse, tiger, lion, bear, squirrel, cow, panda, hedgehog, elephant, giraffe, hippo.
Fruits (15 types):
apple, banana, orange, grape, strawberry, blueberry, watermelon, pineapple, mango, peach, cherry, pear, kiwi, lemon, plum.
Predicate. Always "is" (e-g.. "Bob is curious").
Objects. Objects are adjective attributes describing the subject. We use pool of 34 attributes: young, soft, scary, hot, smart, clean, beautiful, red, blue, green, purple, boring, strong, happy, round, big, noisy, fast, sticky, bouncy, spiky, furry, bright, shiny, magical, striped, spotted, tasty, juicy, toxic, friendly, curious, loud, sleepy. "The code is from this GitHub repository
18

---

Published as a conference paper at ICLR 2026
Rule generation. Rules are constructed following Liang et al. 2023) with minor modifications for brevity and formatting. Each rule specifies that one to three attribute conditions on a subject imply a new attribute. Concretely, randomly sample attributes and instantiate rules of the form:
If [subject] is attribute; and/or or attribute, and/or attributes, then [subject] is b.
For each instance, generate more than four rules ensure sufficiently large pool of rule- -eloggrodded attributes. This enables the construction of challenging negative answer options whose attributes appear in at least one rule, preventing trivial elimination.
Following Liang et al. (2023), rules may apply either to a specific subject (c.g., "Alice") or to a subject category (e.g., "a person"), requiring the model to perform basic abstraction across individual and category-leve reasoning.
Most rules contain either and or or, depending on the interaction type being instantiated (e.g., and for Complementary interactions, or for Alternative interactions).
three conditions. When multiple conditions are present, the conjunction is
For surface realization, we adopt a compact form for single-condition rules to reduce token usage (e.g.. "Weak cow is small" instead of "If cow is weak, then the cow is small"). For multi-condition rules, we retain the explicit ih-then structure to preserve clarity, e.g., "If a person is smart and young, then the person is curious".
Image & audio generation. For the multimodal reasoning experiments, we require factual information in both visual and auditory formats. To generate images, we employ the Graphviz toolkit, which effectively converts structured data into clear, labeled diagrams suitable for model interpretation. For audio synthesis, we utilize Cosy Voice 2 (Du al. (2024), ) one of the top Text-to-Specch (TTS) model renowned for its high consistency in timbre, achieving human-parity synthesis quality. Since the audio is solely intended provide semantic information for subsequent logical reasoning within the MLLM, and we are not investigating the model's 's advanced audio understanding capabilities, we prioritize accuracy over variability the synthesized speech. Therefore, we use Cosy Voice 2's default settings to ensure the speech is clear and precise, without introducing unnecessary characteristics that could potentially interfere with the reasoning task.
B.2 EXPERIMENT SETUP
LLM inference setup. During evaluation, all models are run float16 precision. For models that support audio output, we disable this feature and only generate text. The maximum number of generated tokens
1024, which is typically sufficient for the model produce a complete response, as long as the model does not exhibit output repetition. To ensure coherent and consistent output, we set 'do sample=False" for all models, which is also the default setting for these models and considered best practice.
Linear probing setup. To construct the attention feature as the input of the probe, we extract and pool attentions. Specifically, for one data sample, the attention matrix shape should be N x L x H x xO. Here, N is the number of tokens representing one fact (in one modality); is the number of layers in the model; H is the number of attention head in the model; and O is the number of generated tokens of the response for the model. Since O and N could vary in samples (or facts), we take the mean pooling in term of these two dimensions. The final attention feature for each sample is shaped as I, H. We flatten the feature as one attention feature for one fact in one sample as the input for the probe classifier.
For the linear probe classifier, we use a 5-fold cross-validation setup with GroupKFold to evaluate the models. The models are trained with Logistic Regression using regularization strength of C=1.0, an 12 penalty, and class class_weige weeght="balannce" normalized using the standard score method, which standardizes the features to have zero mean and unit variance. We apply these configurations to the task at hand. These configurations ensure a robust and consistent evaluation across all models tested.
to handle class imbalances. The feature vectors are
19

---

Published as a conference paper at ICLR 2026
B.3 PROMPT TEMPLATES
We list the prompt templates for all six types of interactions, the recognition task, and the two-step reasoning task. All models have the same input (template and data).
20

---

Published as a conference paper at ICLR 2026
Prompt:
input and output (Equivalence)
System:
You assistant tasked with solving multiple-choice questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given-do not rely outside commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format Annwer: X' where X is the chosen letter A/B/C B/C/D.
Input:
Erin
friendly
Erin is friendly
(Audio information: Erin is friendly)
Rules follows: Blue person is tasty. Red person is clean. If a person is smart sleepy. then the person is curious. Friendly person is purple. Which of the following options can be inferred based on the given facts and rules? A) Erin is curious. B) Erin is purple. C) Erin is tasty. D) Erin is clean.
Output: To determine which option can be inferred based on the given facts and rules, let's analyze the information step by step:
1. Given Information: Erin is friendly.
2. Rules: Blue person is tasty. Red person is clean. If person is smart or sleepy,
then the person is curious. Friendly person is purple.
3. Analysis: From the rule "FFiendlyy person is purple,"" we can infer that since
Erin is friendly, Erin must be purple.
Checking the Options:
Erin is curious. The rule does not provide direct link between being friendly and being curious. Therefore, we cannot infer this.
B) Erin purple. This directly follows from the rule "Friendly person is purple."
C) Erin is tasty. The rule does not provide direct link between being friendly and being tasty. Therefore, we cannot infer this.
D) Erin is clean. The rule does not provide direct link between being friendly and being clean. Therefore, we cannot infer this.
Based on the analysis, the only option that can be inferred is:
Answer:
Figure Prompt template model (Qwen) output (Equivalence).
21

---

Published as a conference paper at ICLR 2026
Prompt:
input and output (Alternative)
System:
You assistant tasked with solving multiple-choie questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given-do not rely outside or commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format "Answrr X' where X is the chosen letter A / B / C//.
Input:
friendly
Erin is purple
(Audio information: Erin is red)
Rules are as follows: Friendly person clean. If person is smart or sleepy, then the person is curious. Purple person is clean. Blue Erin is tasty. Spotted Erin is beautiful.
Red person is clean.
Which of the following options can be inferred based on the given facts and rules? A) Erin is clean. B) Erin is tasty. C) Erin is beautiful. D) Erin is curious.
Output:
To determine which option can be inferred based on the given facts and rules, let's analyze the information step by step:
Given Information:
Erin is purple.
Erin is red.
Erin is friendly.
2. Rules: - If person smart or sleepy, then the person is curious. Blue Erin is
tasty. Spotted Erin is beautiful. Friendly person is clean. Purple person clean.
Red person is clean.
3. Analysis: Since Erin is purple and red, and the rules state that purple and red people are clean, we can infer that Erin is clean. The rules do not provide information about Erin being smart, sleepy, curious, beautiful, or tasty based on the given facts.
Conclusion: The only inference we can make from the given information that
Erin is clean.
Answer:
Figure 5: Prompt template and model (Qwen) output (Alternative).
22

---

Published as a conference paper at ICLR 2026
Prompt: input and output (Entailment)
You are an assistant tasked with solving multiple-choice questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given-do not rely outside or commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format Answer: X' where X is the chosen letter A / D.
System:
Input:
Erin
is
friendly
Erin is bright
(Audio information: Erin is bouncy)
Rules are as follows: If a person is smart sleepy, then the person is curious. Red person is clean. Blue person is tasty. Bright person is friendly. Friendly person is purple. Bouncy person is bright.
Which of the following options can be inferred based on the given facts and rules? A) Erin is curious. B) Erin tasty. C) Erin purple. D) Erin is clean.
Output: Let's analyze the given information step by step:
Erin is bouncy.
Bouncy Erin is bright.
Erin is friendly.
4. Friendly person is purple.
Bright Erin is friendly.
From the information, we can deduce the following:
Since Erin is bouncy, Erin is bright.
Since Erin is bright, Erin is friendly.
Since Erin is friendly, Erin is purple.
Now, let's evaluate each option:
A) Erin is curious. This cannot be inferred because the rules do not connect being smart sleepy to being curious.
B) Erin is tasty. This cannot be inferred because there is no information linking being bouncy, bright, or friendly being tasty. C) Erin is purple. This can be inferred because Erin is friendly, and friendly people purple.
D) Erin is clean. This cannot be inferred because there is no information linking being bouncy, bright, or friendly being clean. Answer: C
Figure 6: Prompt template and model (Qwen) output (Entailment).
23

---

Published as a conference paper at ICLR 2026
Prompt:
input and oulput (Independence)
System:
You assistant tasked with solving multiple-choice questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given-do not rely outside or commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format Annwer: X' where X is the chosen letter A / B C/D.
Input:
Erin
friendly
Dan is sleepy.
(Audio information: Erin is spiky.)
Rules are follows: Blue person is tasty. Red person is clean. If person is smart sleepy, then the person is curious. Friendly person is purple. Which of the following options can be inferred based on the given facts and rules? A) Erin is curious. B) Erin is purple. C) Erin is tasty. D) Erin is clean.
Output: To determine the correct answer, let's analyze the given information step by step:
1. Erin is friendly. 2. Friendly person is purple. 3. Blue person is tasty.
From the rules provided: If person smart or sleepy, then the person is curious. Red person is clean.
Since Erin is friendly and friendly people are purple, we can infer that Erin is purple
Now, let's evaluate the options:
A) Erin is curious. This cannot be inferred from the given information.
B) Erin is purple. This can be inferred from the given information.
C) Erin is tasty. This cannot be inferred from the given information.
D) Erin is clean. This cannot be inferred from the given information.
Answer: B
Figure 7: Prompt template and model (Qwen) output (Independence).
24

---

Published as a conference paper at ICLR 2026
Prompt:
input and output (Contradictory)
You are an assistant tasked with solving multiple-choice questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given-de do not rely on outside or commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format Answer: X" where X is the chosen letter A / D.
System:
Input:
Erin
friendly
Erin is red
(Audio information: Erin is blue)
Rules are as follows: Blue person is tasty. Red person is clean. If a person is smart or sleepy, then the person is curious. Friendly person is purple. Which of the following options can be inferred based on the given facts and rules? A) Erin is curious. B) Erin tasty. C) Erin purple. D) Erin is clean.
Output:
Let's analyze the information step by step:
Erin is blue.
Blue people are tasty.
3. Erin is friendly.
Friendly people are purple.
From the rules, we can infer the following:
Since Erin is blue, she tasty.
Since Erin is friendly, she is purple.
Now, let's evaluate each option:
A) Erin is curious. This cannot be inferred because the rules do not connect being blue or friendly to being curious.
B) Erin is tasty. This can be inferred because blue people tasty.
C) Erin is purple. This can be inferred because friendly people are purple.
D) Erin clean. This cannot be inferred because the rules do not connect being blue or friendly to being clean.
Based on the given facts and rules, the options that can be inferred are B and C. However, since the question asks for single answer, we need to choose the one that directly supported by the rules without additional assumptions. Answer: B
Figure 8: Prompt template and model (Qwen) output (Contradictory).
25

---

Published as a conference paper at ICLR 2026
Prompt:
input and output (Complementary)
You are an assistant tasked with solving multiple-choice questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given- do not rely on outside or commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format Answer: X' where X is the chosen letter A / B C/D.
System:
Input:
Erin
friendly
Erin is purple
(Audio information: Erin is red)
If a person purple and red and not friendly, then the person soft. a person is friendly and purple and not red, then the person is big. If person is friendly and red and not purple, then the person is scary. If person is friendly and purple and red, then the person is clean.
Which of the following options can be inferred based on the given facts and rules? A) Erin is soft. B) Erin is scary. C) Erin is clean. D) Erin is big.
Output:
To determine which option can be inferred based on the given facts and rules, let's analyze the information step by step:
1. Erin is purple.
2. Erin is friendly.
3. Erin is red.
Now. let's apply the rules Erin:
Rule 1: If a person is purple and red and not friendly, then the person is soft. Erin purple and red and friendly, so this rule does not apply Erin. Rule 2: If a person is friendly and purple and not red, then the person is big. Erin friendly, purple, and red, so this rule does not apply to Erin.
Rule 3: If person is friendly and red and not purple, then the person is scary. Erin friendly, red, and purple, so this rule does not apply to Erin.
Rule 4: If person is friendly and purple and red, then the person is clean. Erin is friendly, purple, and red, so this rule applies to Erin.
Based the analysis, the only rule that applies to Erin is Rule 4, which states that if a person is friendly and purple and red, then the person is clean. Answer:
Figure 9: Prompt template and model (Qwen) output (Complementary).
26

---

Published as a conference paper at ICLR 2026
Prompt:
input and oulput (Recognition)
You assistant tasked with solving multiple-choice questions about knowledge diagrams. Use only the information explicitly given-lo not rely on outside commonsense knowledge. The facts are given in image, audio and text. Read the question and given information, and directly answer the question in the following format: AAnswe: X' where X is the chosen letter A / CCC/D.
System:
Input:
Erin
friendly
Dan is sleepy.
(Audio information: Erin is spiky.)
Question: Which fact is mentioned the given information in image, audio or text?
Options: A) Erin is sticky.. Erin is friendly.. C) Erin is scary., D) Erin green.
Output:
Figure 10: Prompt template and model (Qwen) output for the fact recognition.
27

---

Published as a conference paper at ICLR 2026
Prompt: input and oulput (Two-Step Reasoning: Step 1)
System Prompt 1: You are assistant tasked with solving multiple-choice questions about knowledge diagrams. Use only the information explicitly given-do not rely on outside or commonsense knowledge. The facts are given in image, audio and text. Read the question and given information, and directly answer the question.
System Prompt 2: You are an assistant tasked with solving multiple-choice questions that require logical reasoning over the supplied knowledge diagrams. Use only the information explicitly given-do not rely on outside or commonsense knowledge. Read the question and given information, think step-by-step and answer the question. At the end of your answer, answer precisely in the format Answer: X' where X is the chosen letter A / B C/D.
Input
Erin
friendly
Dan is sleepy.
(Audio information: Erin is spiky.)
Question: Check the given information and list all the facts the given image, audio and text, respectively.
Output
Facts from the image: Erin is spiky.
Facts from the audio: Dan is sleepy.
Facts from the text: Erin is friendly.
28

---

Published as a conference paper at ICLR 2026
Prompt: input and oulput (Two-Step Reasoning; Step 2)
Input 2 (Including Input 1 Output 1):
Rules follows: Blue person is tasty. Red person is clean. If person is smart sleepy, then the person is curious. Friendly person is purple. Which of the following options can be inferred based on the given facts and rules? A) Erin is curious. B) Erin purple. C) Erin is tasty. D) Erin is clean.
Output 2:
Let's analyze the information step by step:
1. Erin is friendly.
Erin is spiky.
3. Dan is sleepy.
From the rules provided:
If person is smart sleepy, then the person is curious. FFiennly person is purple
Blue person is tasty. Red person is clean.
Since Erin is friendly, according to the rules, Erin must be purple.
Now, let's evaluate the options:
A) Erin is curious. We don't 't have enough information infer this from the given facts and rules.
B) Erin is purple. This directly inferred from the rule that friendly person is purple.
C) Erin is tasty. We don't have enough information to infer this from the given facts and rules. D) Erin clean. We don't have enough information to infer this from the given facts and rules. Answer:
Figure Prompt template and model (Qwen) output for the step reasoning.
29

---

Published as a conference paper at ICLR 2026
C SUPPLEMENTARY RESULTS
C.I MODALITY INTERACTION: EQUIVALENCE
Setup. We now evaluate the Equivalence interaction, where the same decisive fact is redundantly placed in all three modalities (vision, audio, text). The unimodal conditions serve baselines, where the only fact is presented in one modality. This setting examines whether redundant presentation across modalities can reinforce fact recognition and improve reasoning robustness.
Table Performance on the Equivalence interaction. Each modality redundantly contains the same decisive fact. Results show that adding equivalent facts from other modalities provides no consistent improvement, suggesting that unimodal fact recognition is not the reasoning bottleneck.
Unimodal
Accuracy (%)
Multimodal (= (=)AV.AAAA
V
A
T
Baichuan
79.4
75.0
95.9
Qwen
96.3
94.4
98.0
98.9,.1,455,009
MiniCPM
89.4
89.6
95.0
Phi4
58.8
60.2
96.6
24.1
Average
91.0
79.8
96.4
10.91.77
Results. As shown in Tab. adding equivalent facts in vision and audio brings little to gain the text-only setting, which already achieves near-perfect accuracy. In some cases (e.g., Baichuan and Phi4), the multimodal setting even leads significant performance drop. This suggests that additional modalities do not enhance reasoning and may introduce unnecessary interference. Since models already recognize text-based facts effectively, improving recognition robustness does not translate into better multimodal reasoning. The bottleneck appears to lie elsewhere, specifically, in how the recognized facts are composed during reasoning.
Takeaway. Adding redundant evidence across modalities fails to improve reasoning and can even hurt performance. This suggests that text-based fact recognition is not the limiting factor. Instead, cross-modal fusion and composition seem to be the main challenges multimodal reasoning. Since simply reinforcing facts does not help, we next ask: can additional modalities help by introducing new and independent reasoning paths? We investigate this in the following setting.
C.2 MODALITY INTERACTION: ALTERNATIVE
Setup. In this setting, each modality contains a unique antecedent that individually sufficient to trigger the same rule and yield the correct conclusion. Thus, the model can arrive the correct answer by reasoning over any one of the modalities. This evaluates whether models can flexibly leverage semantically diverse cues when multiple independent reasoning paths available.
Table Performance on the Alternative interaction. Each modality provides an independent reasoning path, the correct answer can inferred from any one of them. Multimodal accuracy improves slightly over the text-only baseline, showing that models can benefit from semantically diverse cues across modalities. This suggests that introducing complementary reasoning paths can help mitigate modality-specife limitations.
Unimodal
Accuracy (%)
Multimodal (VAV.
V
A
T
Baichuan
78.0
79.8
97.3
97. 97.6
17.11.033311
Qwen
96.3
93.9
97.4
100.0a.1...11:2200
MiniCPM
92.0
91.1
96.2
99.12.1...01,291
Phi4
77.6
71.6
96.9
97.920.31. 26.31,1.1.
Average
86.0
83.9
97.0
98.7 98.71..7
30

---

Published as a conference paper at ICLR 2026
Results. As shown in Tab. 6 models achieve strong performance across unimodal settings, par- ticularly in the text-only case. Notably, when all three sufficient premises are presented across modalities, performance consistently improves across models. For example, Qwen improves from 97.4% (text-only) to 100.0%, and MiniCPM improves from 96.2% to 99.1 %. Although the gain is modest compared to the text baseline, the upward trend suggests that models can effectively integrate multiple alternative cues to reinforce the reasoning process, even when distributed across modalities.
Takeaway. These findings suggest that unlike redundant (Equivalence) setups, providing alternative reasoning paths across modalities can support more robust reasoning. Multimodal information helps when it offers diverse routes to the same conclusion, rather than simply reiterating or fragmenting the required information. Since introducing alternative single-step reasoning cues across modalities leads to measurable gains, it raises a natural follow-up question: can models also benefit from more indirect, multi-hop cues, such as entailment chains spread across modalities? We explore this next.
C.3 MODALITY INTERACTION: ENTAILMENT
Setup. In the Entailment interaction, the model must perform multi-hop reasoning through a chain of rules. Specifically, three facts form reasoning chain: A + B - C, where the final answer is entailed by C. While the decisive fact is always placed in one fixed modality (vision, audio, or text), the earlier support facts A and B are distributed across the other two modalities. This setting evaluates whether the model can integrate indirect, cross-modee evidence to support the final-step reasoning.
Table 7: Performance on the Entailment interaction. Each modality carries a fact needed for multi- hop reasoning, with the final-step premise fixed in one modality (V/A/T). Models consistently perform worse in multimodal setups compared to their unimodal baselines. This suggests that spreading multi multi-hop reasoning steps across modalities introduces substantial integration errors, regardless of where the final step placed.
Unimodal
Multimodal (+)); Final-Step Fact
Accuracy (%)
A
T
Vav
Aaa
TAr
Baichuan
81.5
82.0
94.3
79.52.04
75.6g.41
80.71a.6.
Qwen
94.1
94.8
96.7
78. 78.41
86.65.25
83.912.8.
MiniCPM
93.2
92.9
95.2
81.811.44
80.012.91
88 48.46..1
Phi4
75.2
70.0
97.7
73.0
69.3o.71
79.718.0u ou
Average
86.0
84.9
96.0
78.27
77.97.1
83 83.212.8u
Results. As shown in Tab. 7 all models experience significant performance drops in the multimodal condition compared their unimodal counterparts, regardless of which modality carries the final-step fact. For instance, when the final-step fact is in text, we observe up to 12.8% drop compared to the text-only baseline. This pattern holds consistently across models and configurations. These results suggest that the modsl composition of reasoning chains introduces substantial integration errors, even when the decisive premise remains in a strong reasoning modality.
Takeaway. Unlike alternative information, which offers parallel reasoning paths, entailment infor- mation provides indirect, chained support, which proves to be not only unhelpful but actively harmful to reasoning performance. This highlights that current MLLMs struggle with cross-modal multi-hop reasoning, even when all required facts are present. We have now evaluated three types of auxiliary information: equivalence, alternative, and entailment, and found that only alternative reasoning paths offer modest improvements. In contrast, redundant or indirect information often introduces confusion and leads to performance degradation.
C.4 MODALITY INTERACTION: INDEPENDENCE
Tab. presents detailed results under the Independence interaction, where each instance contains single decisive fact placed in one modality (vision, audio, or text), while the other modalities contain only irrelevant distractors. We evaluate both unimodal reasoning (all facts are given in one modality) and multimodal reasoning (the decisive fact is mixed with distractors across modalities).
31

---

Published as a conference paper at ICLR 2026
Table Performance the Independence interaction with cross-modal distractors. Each instance contains one decisive fact placed in specific modality (V: vision, A: audio. T. text), while the other modalities contain only noisy facts. We report accuracy (%) for each unimodal condition as well as multimodal reasoning when decisive facts are distributed.
Unimodal
Multimodal (Decisive Facts)
Accuracy (%)
V
A
T
V
A
T
Random
Baichuan
60.2
72.0
94.8
74.3
53.5
74.9
67.6
Qwen
73.3
94.3
95.5
50.8
90.8
84.1
75.2
MiniCPM
77.6
83.7
91.2
66.8
78.2
91.0
78.7
Phi4
49.9
48.9
96.3
58.0
50.4
70.7
59.7
Average
65.3
74.7
94.5
62.5
68.2
80.2
70.3
Across all models, text consistently yields the highest unimodal accuracy (average 94.45%), while vision and audio vary significantly in performance. Interestingly, when the decisive fact remains in fixed modality but is surrounded by irrelevant facts from other modalities, we observe substantial performance drops for vision and audio (e.g.. Qwen drops from 73.3% to 50.8% when adding distractors to vision). The "Random" column averages over all cross- settings with randomly selected decisive modalities, showing that even single irrelevant modality can degrade reasoning.
D VALIDATION ON REAL- WORLDWOLDD ISOMORPHIC REPRESENTATIONS (ISOBENCH)
To verify that our findings are not artifacts of synthetic data, we conduct partial validation on real-world isomorphic representations using IsoBench (Fu et al. 2024). IsoBench provides paired text and image inputs that encode the same underlying symbolic information (e-g.. mathematical expressions, graphs, and board states), enabling evaluation of the Equivalence interaction in natural visual formats.
Although existing real-world benchmarks do not support all six interaction types in our framework, IsoBench allows us to replicate the Equivalence setting for the vision-text modality pair. Using the same models in our main experiments, we evaluate unimodal text (T), unimodal vision (V), and multimodal text+vision (T+V) inputs under identical prompting and evaluation protocols.
Table 9: Performance on IsoBench under the Equivalence interaction. We report accuracy (%) for unimodal
vision (V), unimodal text (T). and multimodal text+vision (T+V).
Accuracy (%)
V
T
T+VAv.AT
Baichuan
58.6
67.1
61.52.91.5.4
Qwen
57.6
68.1
66.2g.6t.194
MiniCPM
58.1
63.3
63.7a.61.044
Phi4
60.3
66.2
63.
63.1241,3.1
Average
58.7
66.2
63.6..1.2.41
As shown in Tab. observe the same pattern in our synthetic experiments: redundant multimodal inputs provide little or no improvement over strong text-only baselines, and in some cases slightly degrade performance. This consistency indicates that the task-composition and modality- fusion bottlenecks identified in our framework extend beyond controlled synthetic settings to real- world isomorphic representations.
LLM USAGE
We used ChatGPT as general purpose assistant in preparing this paper. In particular, LLMs were employed for grammar refinement, clarity improvements, LaTeX formatting, and debugging minor
general-purpose
32

---

Published as a conference paper at ICLR 2026
code snippets. They were not involved in research ideation, experimental design, or the development of theoretical contributions.
33


Reviews:
[{"review_id": "ZopDKMlh9n", "reviewer": "Reviewer_2P4W", "summary": "This paper investigates the mechanisms and limitations of Multimodal Large Language Models (MLLMs) in performing logical reasoning across diverse modalities (text, vision, audio). Addressing conflicting reports in literature regarding whether additional modalities aid or hinder MLLM performance, the authors introduce a controlled, logic-grounded evaluation framework. This framework categorizes multimodal reasoning into six interaction patterns: Equivalence, Alternative, Entailment, Independence, Contradictory, and Complementary.\n\nUsing a synthetic dataset where facts are rendered as text, synthesized audio, and schematic visuals, the study evaluates several open-source MLLMs (Baichuan-Omni, Qwen2.5-Omni, MiniCPM, Phi-4) and identifies several observations and bottlenecks. The authors validate these bottlenecks through interpretability probes and interventions, showing that two-step prompting (decoupling recognition and reasoning) and softening attention in early layers can mitigate these failures.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The introduction of the six canonical interaction types (Equivalence, Alternative, Entailment, etc.) makes sense, which moves multimodal evaluation beyond simple accuracy metrics on varied tasks to a structurally grounded analysis of how information is combined.\n\n- The paper moves beyond just reporting performance drops. It offers diagnostic evidence for why these drops occur.\n\n- By identifying that integration, not perception, is the main barrier, the work offers actionable direction for future modeling (e.g., composition-aware training objectives, explicit fusion control mechanisms).", "weaknesses": "- The synthetic data generation process and dataset details are not clearly described in the paper.\n\n- The observations and conclusions made in the paper are based on synthetic data. Ideally, it would be nice to see how (some of) these observations could be made on more realistic settings with real data (e.g., OmniBench).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nNsWwiBLVk", "reviewer": "Reviewer_uT7W", "summary": "This paper uses a pipeline to test cross-modal reasoning. Specifically, it tests the audio, vision, and text modalities and their combinations. The authors obtained their findings from controlled experiments. They have conducted several interesting experiments on different categories of reasoning.", "questions": "NA", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "The authors define different types of the logics to test the performance.\n\nWith the defined categories, the authors clearly identify the core bottlenecks.", "weaknesses": "1. The study's conclusions are built on overly simplified, synthetic data. Facts are basic statements like \"Alice is awesome\", which are then turned into simple schematic graphs or clean TTS audio. This setup doesn't reflect the complexity of the real world, where models must reason over natural images, noisy audio, and complex text. This \"toy\" data makes it questionable if the findings will hold in more realistic scenarios.\n\n2. Many of the paper's \"findings\" are intuitive consequences of its data design. For example, It's not surprising that performance drops in the Complementary setting, where models must fuse necessary facts from different modalities. It's also expected that performance drops in the Independence setting when \"weaker\" audio or visual modalities are added, as the text is designed to be the clearest source of information (as vision and audio are converted from the text). The conclusion that text is the strongest modality seems pre-determined by this design rather than being a deep insight. This makes the paper's takeaways feel unsolid.\n\n3. The paper's writing and structure make its contribution unclear. The \"Related Work\" section is relegated to the appendix, so the reader cannot easily compare this benchmark to existing ones. The authors claim previous studies are \"anecdotal or domain-specific\" but never justify in the main paper why their own synthetic data is a better alternative. This omission makes it difficult to judge the paper's novelty or true contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JOmZiQHnv3", "reviewer": "Reviewer_YhaC", "summary": "This paper investigates when and why multimodal large language models (MLLMs) succeed or fail at logical reasoning across modalities. The authors introduce a logic-grounded evaluation framework with six canonical interaction patterns (Equivalence, Alternative, Entailment, Independence, Contradictory, Complementary) that systematically vary how facts are distributed across text, vision, and audio modalities. Through controlled experiments on four open-source MLLMs (Baichuan-Omni, Qwen2.5-Omni, MiniCPM-o, Phi-4), they find that additional modalities only help when providing independent reasoning paths, while redundant or chained evidence often hurts performance. The authors identify two core bottlenecks: (1) task-composition bottleneck where models struggle to jointly perform recognition and reasoning, and (2) fusion bottleneck where early-layer integration introduces modality bias. Probing and intervention experiments validate these bottlenecks and show that simple remedies like two-step prompting and attention temperature manipulation can alleviate these issues.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Clear writing and presentation**: The paper is well-written with clear motivation and systematic exposition.\n3. **Systematic evaluation**: Six logic-grounded interaction types enable controlled analysis of information distribution across modalities.\n4. **Controlled design with actionable insights**: Synthetic data with standardized renderings isolates reasoning mechanisms, and probing analysis provides causal understanding beyond black-box evaluation.", "weaknesses": "1. Unrealistic modality representations: Your modalities use artificial encodings (GraphViz graphs, monotone TTS) rather than naturalistic isomorphic representations commonly seen in practice. IsoBench (Fu et al., 2024, https://arxiv.org/abs/2404.01266) uses real-world formats (e.g., math expressions as text vs. diagrams, chess as notation vs. images) and finds 28.7-point drops for Claude-3. Could you validate with similar real-world isomorphic representations?\n2. Oversimplified tasks: Single-template facts (\"subject + is + adjective\") and single-step reasoning lack complexity of real multimodal reasoning (spatial relations, temporal dynamics, multi-hop inference). Do findings generalize to realistic scenarios?\n3. Limited model scope: Only 4 small models (5-8B) are tested, leaving unclear whether findings generalize to larger systems. Could you add at least one large-scale model to validate the bottlenecks persist at scale?\n4. No statistical testing: Despite 1,000 instances per condition, no error bars or significance tests. Are modest differences (+1.7%, -5.7%) statistically meaningful?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "76FofZX2zA", "reviewer": "Reviewer_Wn7A", "summary": "Disclosure: Claude is used to refine this review.\n\nThis paper investigates when and why additional modalities help or hurt reasoning in multimodal large language models. The authors introduce a logic-grounded evaluation framework with six canonical interaction types (Equivalence, Alternative, Entailment, Independence, Contradictory, Complementary) that systematically vary how facts are distributed across modalities. Through controlled experiments on synthetic reasoning tasks, they identify two core bottlenecks: task-composition (inability to jointly perform recognition and reasoning) and fusion (biased early-layer integration). Simple interventions like two-step prompting and attention temperature adjustment substantially improve performance.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The six interaction types provide a principled, logic-grounded framework for isolating different aspects of multimodal reasoning. The controlled setup with synthetic data allows precise measurement of effects that would be confounded in naturalistic settings.\n- The paper goes beyond black-box evaluation to probe internal mechanisms through attention analysis and targeted interventions. The connection between external performance patterns and internal representations is clearly demonstrated.\n- The identification of task-composition and fusion bottlenecks is well-supported, and the proposed interventions (two-step prompting, attention temperature control) are simple yet effective. The paper convincingly shows that integration, not perception, is the main barrier.\n- The evaluation includes proper baselines, multiple models, and systematic ablations. The paper is well-written and easy to follow.", "weaknesses": "- The main limitation is that the analysis is done with synthetic data. The entire evaluation uses controlled synthetic reasoning tasks. While this enables precise control, it raises serious questions about generalizability to real-world multimodal reasoning where visual complexity, ambiguity, and grounding matter significantly. The claim that these findings inform real MLLM behavior needs empirical validation on naturalistic tasks. \n- Only single-step reasoning tasks are considered. It's unclear whether findings hold for larger models, more complex reasoning chains, or diverse domains.\n- While attention probing and interventions demonstrate effects, the paper doesn't deeply explain why these patterns emerge or how they relate to training objectives. \n- Several related work has covered a subset of the findings in the paper, e.g., [1] [2]. The difference should be discussed.\n\n[1] Wu et. al. Mitigating Modal Imbalance in Multimodal Reasoning. COLM 2025 https://www.arxiv.org/abs/2510.02608\n[2] Fu et. al. Hidden in plain sight: VLMs overlook their visual representations. COLM 2025 https://arxiv.org/abs/2506.08008", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)