MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning
Authors: Jiachun Li
Shaoping Huang
Zhuoran Jin
Chenlong Zhang
Pengfei Cao
Yubo Chen
Kang Liu
Jun Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: multimodal reasoning
multimodal benchmark
multi-image benchmark
thinking models
Primary area: datasets and benchmarks
TL;DR: We introduce MMR-Life, a large and diverse benchmark for evaluating multimodal multi-image reasoning in MLLMs across real-life scenarios.

Abstract: Recent progress in the reasoning capabilities of multimodal large language models (MLLMs) has empowered them to address more complex tasks such as scientific analysis and mathematical reasoning. Despite their promise, MLLMs' reasoning abilities across different scenarios in real life remain largely unexplored and lack standardized benchmarks for evaluation. To address this gap, we introduce MMR-Life, a comprehensive benchmark designed to evaluate the diverse multimodal multi-image reasoning capabilities of MLLMs across real-life scenarios. MMR-Life consists of 2,646 multiple-choice questions based on 19,108 images primarily sourced from real-world contexts, comprehensively covering seven reasoning types: abductive, analogical, causal, deductive, inductive, spatial, and temporal. Unlike existing reasoning benchmarks, MMR-Life does not rely on domain-specific expertise but instead requires models to integrate information across multiple images and apply diverse reasoning abilities. The evaluation of 37 advanced models highlights the substantial challenge posed by MMR-Life. Even top models like GPT-5 achieve only 58% accuracy and display considerable variance in performance across reasoning types. Moreover, we analyze the reasoning paradigms of existing MLLMs, exploring how factors such as thinking length, reasoning method, and reasoning type affect their performance. In summary, MMR-Life establishes a comprehensive foundation for evaluating, analyzing, and improving the next generation of multimodal reasoning systems.

Published as a conference paper at ICLR 2026
MMR-LIFE: PIECING TOGETHER REAL-LIFE SCENES FOR MULTIMODAL MULTI-IMAGE REASONING
Jiachun Li Shaoping Hu Zhuoran Jin Chenlong Zhang!? Pengfei Cao
Yubo Chen Kang Liu Jun Z.2
'School of Artificial Intelligence, University of Chinese Academy of Sciences
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences {jiachun. 1i, pengfei cao, zhueran. jin ac.cn https: /mmr- life silfieeeeenne bench. github.
ABSTRACT
Recent progress in the reasoning capabilities of multimodal large language mod- els (MLLMs) has empowered them to address more complex tasks such as scien- tific analysis and mathematical reasoning. Despite their promise, MLLMs' rea- soning abilities across different scenarios in real life remain largely unexplored and lack standardized benchmarks for evaluation. To address this gap, we intro- duce MMR-Life, comprehensive benchmark designed to evaluate the diverse multimodal multi-image reasoning capabilities of MILMs across real-life scenar- ios. MMR-Life consists of 2,646 multiple-choice questions based on 19,108 im- ages primarily sourced from real-world contexts, comprehensively covering seven reasoning types: abductive, analogical, causal, deductive, inductive, spatial, and temporal. Unlike existing reasoning benchmarks, MMR-Life does not rely on domain-spectin expertise but instead requires models to integrate information across multiple images and apply diverse reasoning abilities. The evaluation of 37 advanced models highlights the substantial challenge posed by MMR-Life. Even top models like GPT-5 achieve only 58% accuracy and display consider- able variance in performance across reasoning types. Moreover, we analyze the reasoning paradigms of existing MLLMs, exploring how factors such think- ing length, reasoning method, and reasoning type affect their performance. In summary, MMR-Life establishes comprehensive foundation for evaluating, an- alyzing, and improving the next generation of multimodal reasoning systems.
Broad Reasoning Types
Reasoning Real-life Senarios
SOTA MLLM's Evaluation
eir Abductive
BiBl Deductive
GPT-S Look et the couple the
Acc
Multihop
Compsstion tedurtion Skp Deductio..
Oman boiding vhits
tee momimnn Are budoe
Analogical
Inddcttive
186 mage they and ewn rasrenno the po In suage
In image they ore for back
65
Bakiter Trbreince
Tarasa Tus acter
5044
Cousal
28 Spotial
comara The mon in piik T-diim das moves kemors unsoe angge.
6:32
Counterfacual
Phodicter.
Routs Planiig.
What tempored ong innoe thosr imagae?
than
conmnter veft Tei progroiion 4-2-333 Meraerr
Temporal
lis Rotucton
(00 3-2--4 00 60-03
Chsdee
ol-itii Genen-22 GPT-5
Figure 1: Overview of the MMR-Life. Left: 7 reasoning types and 21 tasks. Middle: A typical example of multi-image reasoning in real-life scenarios. Right: Extensive evaluation reveals a gap between humans and SOTA MLLMs on some real-life reasoning tasks.
INTRODUCTION
Reasoning is the process of generalizing from known premises to new conclusions. and it is con- sidered key capability for Al systems on the path to artificial general intelligence (AGI) (Sun
"Equal contribution. 1 "ooresponniig authors.

---

Published as a conference paper at ICLR 2026
et al., 2024; Wang et al., 2025a; Li et al., 2024; 2025c; Jin et al., 2025). Recently, with the great success of reasoning large language models (RLLMs) in tasks such as mathematical reasoning (DeepSeck-Al et al., 2025; Muennighoff et al., 2025; Li et al., 2025d), there has been widespread exploration of transferring this reasoning-enhanced paradigm to multimodal large language models (MILMs). Representative models such as Gemini-2.5-PP (Anthropic, 2025b), and GPT-5 (OpenAI, 2025b) leverage long Chain-of-Thougee (CoT) (Wei et al., 2022) style reasoning capture key visual information, decompose complex problems, thereby achieving or even surpassing human-level performance in diverse reasoning scenarios.
(Comanici et al.,, 2025), Claude-Sonnet-4
With the advancement of MLLM reasoning capabilities, there has been an increasing demand for more challenging and realistic multimodal reasoning benchmarks. Recent work mainly evaluates the reasoning ability of MILMs through two approaches: One line of research collects expert- level domain-specific problems to assess the model's reasoning based on knowledge in areas such as scientific knowledge answering (Tie et al., 2025; Xi et al., 2025; Yue et al., 2024) and math problem solving (Wang al., 2025c; He et al., 2024). The other line of research attempts to separate knowledge from reasoning by using synthetic problems like symbolic puzzles to assess reasoning capabilities across different difficulty levels (Song et al., 2025; Yuan et al., 2025; Chia et al., 2024).
Despite significant progress, current benchmarks still exhibit a considerable deviation from real-life reasoning scenarios. (1) From the task design perspective, the tasks in existing benchmarks are not commonly encountered in everyday reasoning. Both knowledge-intensive tasks and synthe- sized guzzle-based tasks remain misaligned with the authentic reasoning demands that arise in eryday situations. For the former, daily reasoning seldom relies on expert-level knowledge, whereas for the latter, the symbolic input images differ substantially from those encountered in real-world scenarios. (2) From the perspective of input images, current benchmarks fail to include multi- image inputs that span a diverse range of reasoning types. large portion of multimodal gen- eral reasoning benchmarks focus exclusively on single-image inputs (Yue et al., 2024; 2025a; Song et al., 2025), which contrasts with real-world conditions where we perceive visual information as a sequence of images rather than single one. For multi-image benchmarks, existing work either in- corporates non-reasoning tasks or focuses on limited reasoning type (Cheng et al., 2025; Kil et al., 2024; Liu et al., 2024; Meng et al., 2025b), making it difficult to support further comprehensive evaluation of MLLM reasoning performance.
To address these issues, we introduce MMR-Life, a comprehensive benchmark designed to evaluate the multimodal multi-image reasoning capability of MLLMs across rial-life scenarios. MMR-Life contains 2,646 carefully curated questions, covering 7 distinet reasoning types (see Figure/ 2), which broadly encompass the reasoning abilities necessary for everyday situations. In MMR-Life, each question is associated with a set of images, primarily taken in real-world scenarios. The an- swers do not require domain-specific expertise but instead ask models to extract key information from multiple real-life images and derive new conclusions. This design aligns MMR-Life more closely with the reasoning types found in everyday life. Figure shows an example from MMR- Life. To address the temporal ordering problem, the model needs to detect individuals recurring across different surveillance images and track their movements, selecting the correct order.
Extensive evaluations on 37 advanced MILLM demonstrate that the real-world reasoning scenarios in MMR-Life remain highly challenging. As illustrated in Figure 1, even the most advanced mod- els, including GPT-5 and Semmini-2.5-Pras reach only 58.69% and 56.86% accuracy on MMR-Life, falling short of human performance by 14%. Besides, the evaluation results demonstrate substantial performance disparities across reasoning types. Existing MLLMS perform relatively well on ana- logical, deductive, and inductive reasoning, but encounter notable bottlenecks in causal, spatial, and temporal reasoning. Based on MMR-Life, we conduct an analysis of MLLM reasoning paradigms and obtain several key findings, including that long thinking benefits only limited reasoning types, RL's weaker generalization in small models, and the clustering of reasoning types into patterns.
In summary, our contributions include: (1) We propose MMR-Life, the first comprehensive bench- mark for evaluating multimodal multi-image reasoning in real-life scenarios across seven reasoning types. (2) Through an extensive evaluation of 37 state-of-the-art MLLMs on MMR-Life, we find that existing models struggle considerably in real-life reasoning, especially in causal, spatial, and temporal tasks. (3) Based on LifeLife, we conduct an in-depth analysis of current MLLM reason- ing paradigms, revealing key findings such as the limited effectiveness of long thinking to certain
2

---

Published as a conference paper at ICLR 2026
Abductive
Causal
Deductive
Spatial
glr
siiver case
mmske Laman
Evarr
longles
hod explovriion
please choose the cornect arder
(msole Colee
Euapin 45" (100 Clockene 801 thin
ingrestents
(0) Twe oue come
for ccol
b-0- 2-4
(10) Ceccvite AD1, ther
To prizery
kicomon
(2) The oon cuse moved
brown moved
green conndrr
6-5-12034
2-5-4
118 Cocower oo: then
ccocon se 220
Jespin
Domestic Life
Image Type: Physical Phenomenion
Daiy
Image Type: Everyday Utjects Sub Retation Estinsation
Humon Actiity
Sub Task Nr. Nr.t--op Cooural Prediction
Recipe Step Deduction
Analogical
Inductive
Temporal
D: Chocos
60
Tourth orimal
image each That
ordtagy
080
sports dlpp
Plenser choose
beteween
appaser or the
most Holly
[C)
comnesoonde
andlogy
should the ned
(30
noxt iomeent
from the options.
between
05)
Type: Neturcl Creatures
Activitiss
Traffic Driving Sequence Prediction
Tosk
Figure 2: MMR-Life examples from each reasoning type.
reasoning types, the weaker generalization of RI small models, and the presence of pattern clus- tering across reasoning types.
2
THE MMR-LIFE BENCHMARK
2.1
OVERVIEW
We introduce the Multimodal Multi-image Reasoning benchmark under real-Life scenarios (MMR- Life), novel benchmark meticulously curated to evaluate the ability of MILMs to perform diverse types of reasoning in everyday situations. MMR-Lifc consists 2,646 multiple-choice questions based 19,108 images, comprehensively covering 7 reasoning types (i.c., abductive, analogical, causal, deductive, inductive, spatial, and temporal) and 21 tasks. Each task is based on a set of multi-images, predominantly sourced from real-life contexts, such domestic life, daily dining. and sports activities. See Figure for examples in MMR-Life and Table for dataset statistics. We further discuss the key concepts (e.g., real-life scenarios) of our benchmark in Appendix B.
2.2
DATA CURATION PIPELINE
Table 1: Key statistics of MMR-Life
Data Collection. We initiate our pipeline by collecting real-life images from variety of sources, including: (1) Public image datasets: We select high-resolution real- world image datasets from Kaggle (Kaggle, 2025), suring that the images within each dataset are related (e.g., temporal relationships), to facilitate the construc- tion of multi-image inputs for our questions. (2) Open web resources: We take screenshots from publicly avail- able web resources to collect real-world multi-image data. For example, we obtain bird distribution density images from the eBird website (eBird, 2025). (3) Public video sources: Given the inherent correlation between frames in video, they are ideal for multi-image data. We ex- tract frames from publicly available video datasets to
Statistics
Number
Total Questions
2,646
Total Reasoning Types/Tasks
Imagee Types
15
Reasoning Type
Abductive Reasoning
307 (11 (.0%))
Analogical Reasoning
(21.47%)
Causal Reasoning
263 (9.94%)
Deductive Reasoning
282 (10.66%)
Inductive Reasoning
Spatial Reasoning
255 19.64%
Temporal Reasoning
542 (20.48%)
Text Options
1454 95%)
Image Options
1192 (45.05%)
Average Image Counts
7.22
create images, while ensuring the clarity of each frame. (4) Other existing benchmarks: Finally, we collect data from existing multi-image video reasoning bench-
Average Question Length
283

---

Published as a conference paper at ICLR 2026
marks, extract frames from the videos, and remove images with low quality. The detailed collection protocol and data sources for each task are reported in Appendix C.I.
Task Design. To make our benchmark more aligned with real-life scenarios, aim cover broader range of reasoning types, reflecting diverse everyday situations. Specifically, based on the collected images, we design 7 distinct reasoning types (see Figure 2 for examples): (1) Abductive Reasoning (Abd): Given the observed event, inferring the most plausible explanation for why the event occurred. (2) Analogical Reasoning (Ana): Inferring conclusions about a new situation by identifying similarities with a known case. (3) Causal Reasoning (Cau): In contrast to abductive reasoning, based on the cause, inferring the effect. (4) Deductive Reasoning (Ded): Based on general rules or premises, drawing logically certain conclusions about specific cases. (5) Inductive Reasoning (Ind): Generalizing rules or patterns from specific observations. (6) Spatial Reasoning (Spa): Understanding and reasoning about the locations, movement, and spatial relations of objects. (7) Temporal Reasoning (Tem): Reasoning about the order, duration, and timing of events.
Question-Answer Generation. We generate question-answe annwerr pairs using either automatic synthe- sis or manual annotation, depending on the task type. In some cases, the explicit information con- tained within the multi-image set we collect is already sufficient to fulfill the task's requirements. For example, in the temporal reasoning example of Figure 2, the images themselves contain sequen- tial information, which is sufficient for the sequence prediction task. In these cases, we can define heuristic rules and use code to automate the synthesis of question- answer pairs using the informa- tion. However, some tasks require reasoning over implicit information in images. For instance, in the abductive reasoning example of Figure 2, we need to identify causal event pairs within the scene to construct the questions. In these cases, we manually design question-answer pairs according to the reasoning type ensure the quality of the data. This process leads to the creation of diverse set of 3.2K questions from multiple sources. See Appendix C.2 for detailed annotation guidelines and Appendix for task details.
Negative Option Generation. Given that many reasoning tasks do not have a single correct an- swer (e.g.. providing a plausible explanation in abductive reasoning), we design all questions in a multiple-choice format, where the model must choose the most appropriate answer from five op- tions. Each option is presented as either an image or text (with the distribution provided in Table I). For image options, we use heuristic rules to sample incorrect candidates. As an example, in the tem- poral reasoning example in Figure 2, we construct negative options by choosing frames that either precede the input images or occur at much later time steps. For text options, we invoke GPT-5-mini (OpenAI, 2025b), GPT-4o (OpenAI, 2024), and Qwen2.5-VL-32BB (Bai et al., 2025) to generate responses (see prompts Appendix C.3). From all generated incorrect responses, we manually choose the four highest-quality erroncous options to serve as the final incorrect choices.
Data Quality Control. To further control the quality of our data, we perform three steps of data filtering. (1) Difficulty filtering: We employ three smaller models, Qwen2.5-VL-7B (Bai et al., 2025), Gemma3-4B (Kamath et al., 2025), and InternVL3.5-8B (Wang et al., 2025e), to generate answers for each question. If all models answer correctly, this suggests that the questions are too easy for existing MLLMs, and they therefore filtered out. (2) Format filtering: The model- generated incorrect options may have significant format differences (c.g.. length) compared to the human-constructedd correct answers, which may result in the model relying on shorteuts. To mitigate this effect, we manually revise the options with substantial format differences. (3) Quality filtering: Finally, we distribute the problems among different co-authors, filtering out questions that exhibit semantic ambiguity, have multiple correct answers, or require domain-specific expertise.
2.3 COMPARISONS WITH EXISTING BENCHMARKS
To further distinguish the difference between MMR-Life and other existing ones, we provide de- tailed comparisons in Table 2. From the image type perspective, most existing datasets include a large proportion of symbolic images such as charts and puzzles, which creates a gap from the natural images encountered in daily life. Our benchmark excludes such images, making the evaluation more closely aligned with real-life scenarios. From the source perspective, all questions in our dataset are newly annotated rather than sampled directly from existing datasets, textbooks, or the web, which reduces the risk of data contamination.
4

---

Published as a conference paper at ICLR 2026
Table 2: The comparison between MMR-Life and other existing benchmarks. W (Web), (Text-
book), A (Annotated), E (Existing datasets), and Avg Img.# (average image counts each question).
Dataset
Images
Reason
Source
Knowledge
Avg Img.
MMF: RMI-Rennon (Yuan al., 2025)
1.2K
Symbolic
3 Types
W+T+A-B
Low
VisualPuzzles (Song al. 2025)
Symbolic
Types
W-T+A
Low
MMMU (Yue et all,, 2024)
11.5K
Hybrid
W+T+A
High
1.05
MMLU-Rcane UeUuson (Tic et al., 2025)
LIK
Hybrid
6 Types
W+T+E
Medium
1.85
MEGA-Bench (Chen et ul., 2025)
Hybrid
Medium
MME COT (Jiang al,, 2025)
1.1K
Hybrid
6 Types
E
Medium
2.10
MV-MATH (Wang et al., 2025c)
2K
Symbolic
Type
W+A
High
3.02
MMRB (Cheng al., 2025)
4.8K
Hybrid
Types
E
Medium
6.17
MMIU (Meng al., 2025b)
11.6K
Hybrid
Medium
6.64
MMR-Life (Ours)
2.7K
Natural
Types
Low
7.22
3
MAIN EXPERIMENT
3.1
EXPERIMENTAL. SETTINGS
Multi-modal Language Models without Thinking. We first evaluate the performance of SOTA non-thinking MILLMs
benchmark. These models have not undergone additional reasoning- enhancement training and lack long CoT capabilities. Open-source models include Qwen2.5 7/32/70B (Bai et al., 2025), Gemma3-12/27B (Kamath al., 2025), Jntem/VL3.5888/3BB-AA0 (Wang et al., 2025e). Closed-soure models include GPT-4.1-mini, GPT-4.1 (OpenAI, 2025a), GPT-4o (OpenAI, 2024), Seaude33.7-oome (without thinking) (Anthropic, 2025a) and Doubao- 1.5-vision (ByteDance Seed Team, 2025).
Multi-modal Language Models with Thinking. To study the effect of long CoT patterns on the reasoning abilities MLLMs, introduce several advanced thinking models into the evalu- ation. Open-source models include VL-Rethinnkrr7772BBMW
et al., 2025b), MM-Eureka-QU -weaa (Yue et al., 2025c), Keye-VL-1.5-88 (Team et al., 2025), Qreveee- (Qwen Team, 2024). Closed-source models include 04-mini (OpenAI, 2025c), Haade-Soonet--4Thiniinn (Anthropic, 2025b), Gemini-2.5-Flash (Comanici et al., 2025), Gemini-2.5-Prr (Comanici et al., 2025), GPT-5-mini and GPT-5 (OpenAI, 2025b). We provide complete experiments and results for total of 37 models in Appendix F.2.
32B (Meng et al., 2025a),
Human Level Performance. We employ 12 students with varying degrees and academic back- grounds. Then, we extract 10 questions from each task to form a mini test set of 210 unique ques- tions. From this pool, we repeatedly sample questions students, yielding a total of 600 valid human answers. These students are instructed not to use ternal knowledge sources such as the internet or books. We report the experimental results on this tiny set in Appendix F.3.
time and assign them one of 12
Implementation Details. We employ the same zero-shot CoT prompt input for all models in the main experiments to perform reasoning. To minimize random variation, we conduct five runs for every open-source model and use the average performance as the final outcome. All experiments are performed using 8 NVIDIA A100 GPUs. The detailed experimental parameters and prompts are provided in Appendix F.I.
3.2
MAIN RESULTS
Table 3 presents MILMs' performance on MMR-Life, from which draw several critical insights:
Our MMR-Life benchmark poses significant challenges for MLLMs. Despite achieving nearly 90% accuracy (OpenAI, 2025b) on complex multimodal reasoning tasks like GPQA (Rein et al., 2023) and MMMU (Yue et al., 2024), GPT-5 only achieved an accuracy of 58.69% MMR- Life, with a 14% gap compared to human performance. Moreover, almost all open-source models have accuracy rate below 40% with some of the most recent models, such Skywork-RI -IV. 38B and Intern LL355-8B, performing worse than random guessing (20%). This suggests that, although MMR-Life does not include complex knowledge requirements, real-life reasoning
5

---

Published as a conference paper at ICLR 2026
Table Performance comparison SOTA MILMs on MMR-Life. The highest and lowest scores for each model type across reasoning types are highlighted in green and red, respectively, The highest performance achieved by the model each type is indicated bold.
Model
Abd
Ana
Cau
Ded
Ind
Spa
Tem
Avg
Human
79.76
$7.65
75.00
70.59
63.41
79.76
79.76
72.28
Closed <losed-ouuce source Thinking
GPT-5
33.75
78.87
41.06
80.14
78.32
17.25
41.70
38.69
Gemini-2.5-Pro
54,40
73.77
36.99
79.43
73.66
25.10
35.79
56.86
Gemini-2.5-Flaat
46.25
75.18
34.22
71.63
73.66
23.92
30.81
53.10
04-mini
41.37
73.59
27.38
71.28
68.07
19.22
32.66
50.49
GPT-5-mini
44.95
69.72
32.32
75.18
68.76
12.16
29.52
49.77
Claude-Sonnet-4
36.96
60.92
44.11
67.02
56.64
15.69
28.23
45.32
Closed-soure
No Thinking
GPT-4.1
44.30
71.30
22.43
67.38
70.16
13.73
27.31
48.15
Claude-3.7-Somic
33.55
66.55
35.36
59.93
59.67
20.78
26.01
45.09
GPT-4o
46.91
65.67
25.86
51.42
66.20
11.37
26.01
44.75
GPT-4.1 -miii
32.90
61.62
30.80
52.13
65.27
16.47
30.63
44.10
Doubao-1.5-vviion 1.5-vision
37.13
53.70
31.18
59.57
54,31
12.16
23.06
39.98
Open-source & Thinking
VL-Rethinker:72B
36.48
50.88
33.08
56.03
$7.58
15.69
21.59
39.68
110-72B-Prreviee
31.27
41.20
38.02
47.87
31.24
14.12
16.42
31.14
MM-Eureka-Qwen
26.06
41.02
25.10
47.52
27.97
16.08
17.34
29.02
MiMo VL-7B-RL
38.76
25.88
28.14
60.99
24.94
14.12
19.19
28.68
Keye-VI.-1.5-8B
19.87
21.30
23.95
14.18
20.28
13.73
23.62
20.22
Skywork-R1V-38B
22.15
10.39
16.73
23.76
11.89
9.80
11.07
14.13
Open-source & No Thinking
Qwei2.5-VVL72BB
35.50
35,46
35.36
32.13
35,48
12.94
23.80
40.21
Gemma3-27B
35.18
57.92
36.88
31.21
60.61
12.94
18.27
38.32
Gemma3-12B
25.08
50.70
17.11
27.30
42.42
10.20
15.87
29.52
Qwen2,5-VL-32B
23.45
42.78
21.29
50.00
27.27
15.69
16.24
28.61
Qwen2.5-VL-7B
26.06
35.74
20.53
20.92
38.93
9.41
12.18
24.68
Intern BMeeWVLL.5530B-AA
45.60
19.19
33.46
36.52
14.45
12.16
14.39
23.09
Intern VL3.5-8B
35.18
11.44
18.63
34.04
11.19
14.90
16.61
18.67
scenarios still present a significant challenge for current MLLMs. Future model training and optimization should focus more on these real-world situations.
MILMSs exhibit large disparities across different types of reasoning. While current models perform well in analogical, deductive, and inductive reasoning tasks, they still have substantial room for improvement causal, spatial, and temporal reasoning tasks. We observe that all models per- form poorly in spatial reasoning, with the highest accuracy being only 25.10% %, compared to the human accuracy of 79.76%. In contrast, for tasks like analogical reasoning, most closed-source models outperform human performance. Current models can easily acquire abilities such as analogy and deductive reasoning through feature associations or by memorizing explicit reasoning paths. However, they struggle to learn more abstract world representations, such as spatial and tem- poral reasoning. This bias is one that future model training should seek to correct.
Current open-source thinking models bring limited improvement. When evaluating the effect of adding thinking mode MLLMs, we find that fooeddsouree thinking models generally out- perform closed-source source no-thinking models. However, for open-source models, the thinking mode does not show improved reasoning capabilities. achieves average accuracy of 29.01 %, whereas the thinking model achieves only 27.15% on av- erage. This implies that there is substantial potential for improving the reasoning abilities of current open-source thinking models, particularly in their ability to generalize to real-world contexts.
Table 3, the open-source no-thinking model

---

Published as a conference paper at ICLR 2026
0.5
033
(774
3
035
939
035
033
(a) Qwen2.5VVL-72
(b) GPT-4.1
Tokos
Figure 3: Response tokens vs. Accuracy.
Figure 4: Performance: without CoT vs. with COT.
(a) Gemini-2.5-Flam
(b) GPT-5-mini
(c) GPT-5
Figure 5: Performance comparison under different thinking budgets.
4 THINKING PATTERN ANALYSIS
4.1 IS LONGER THINKING ALWAYS BETTER?
From Table 3, we find that closed-source thinking models perform best MMR-Life. An important question then arises: Is this superior performance associated with the longer reasoning processes?
Reasoning Performance Scales Logarithmically With Thinking Length. To investigate the question, we first present the semi-log plot of average response token count versus average accu- racy over 14 models (see Figure 3). The overall trend shows that models with longer outputs tend to achieve higher scores, indicating that reasoning capabilities scale roughly in proportion to the loga- rithm of the reasoning length. However, there are notable exceptions. Certain open-soure thinking models, including MiMo-Vi MLMo--I--BB-RR- and QVQ-72B-Pree Previee,,, are located in the lower-right region of Figure 3, demonstrating that balaneing reasoning efficiency and model effectiveness remains a major challenge for future open-source MLLMs.
Longer Thinking Is Not All You Need. We conduct a more fine-graine analysis to investigate the relationship between model performance and thinking length across distinct reasoning types. Specifically, for no-thinking models, we follow prior work by comparing their reasoning perfor- mance with and without CoT (Li et al., 2025e; Sprague et al., 2025) (see Figure 4). For thinking models, we select those with controllable reasoning budget and vary the budget (minimal, medium, and high) to gradually increase CoT length, thereby comparing performance across different think- ing lengths (see Figure 5). From both figures, it is evident that longer thoughts do not lead to better performance for all reasoning types. For reasoning types like inductive reasoning, the performance with COT is worse in no-thinking models (see Figure 4) and using more reasoning budget does not lead to better performance in thinking models (see Figure 5). Conversely, for reasoning types such analogical reasoning. the incorporation of CoT or longer CoT results in noticeable performance improvement. We hypothesize that this is because longer CoT may only be suitable for tasks re- quiring syep-by-step reasoning, while types like inductive reasoning may benefit more from faster thinking (Li et al., 2025e).
7

---

Published as a conference paper at ICLR 2026
Table Performance across different methods. Scores higher and lower than the base model's CoT
performance are marked in green and red The highest score in each column is bold.
Model
Method
Abd
Ana
Cau
Ded
Ind
Spa
Tem
Avg (A)
CoT
26.06
35.74
20.53
20.92
38.93
9.41
12.18
24.68
Qwen2,5-VL-Ph
SC@8
28.01
39.44
23.57
25.18
45.45
10.98
13.10
27.85 (+3.17)
BoN@8
27.64
44.72
22.81
25.53
48.02
13.33
13.10
29.54 (+4.86)
GRPO
30.62
40.49
21.29
28.72
43.59
13.73
11.81
28.23 (43.55)
CoT
23.45
42.78
21.29
50.00
27.27
15.69
16.24
28.61
Qwen2.5-VI.322
SC@8
26.06
45.42
23.95
51.77
28.67
16.47
17.90
30.57 (+196)
BoN@s
25.78
44.89
19.39
55,32
30.54
16.47
19.56
30,97 (42.36)
GRPO
22.98
42.96
28.14
49.65
30.77
14.90
19.19
30.29 (+1.68)
CoT
35.50
55.46
35,36
52.13
55,48
12.94
23.80
40.21
SC@8
35.18
56.16
35,36
52.13
54.78
12.94
24.35
40.33 (+0.12)
Qwen2.5-VL-72B
BoN@8
34,20
53.35
32.70
51.77
56.88
13.73
24.72
39.80 (-0.41)
GRPO
36.48
50.88
33.08
56.03
$7.58
15.69
21.59
39.68 (-0.53)
210
215
300
215
2
250
6.0
38.5
-4
260
64
A04
Madfod
(a) Correlation heatmap
(b) Hierarchical clustering
Figure 6: Comparison of BoN and Figure 7: Analysis of correlations across different reasoning
RL performance on different models. types (averaged across all models we evaluate).
4.2 Do GENERALIZABLE REASONING ENHANCEMENT METHODS EXIST?
From the inception of CoT (Wei al., 2022) to the broad application of GRPO (DeepSeck-Al et al., 2025), the reasoning-enhancemen techniques have undergone substantial evolution. In this section, analyze and compare the generalizability of these approaches.
Failure of Enhancement Methods in Larger Models. We select four representative reasoning- enhancement methods for comparison: CoT, Seff-Consistency (SC), Best-of-N (BoN), and GRPO. To evaluate the generalizability of these methods, we directly use previously trained models for in- ference without any training on MMR-Life. Specifically, we adopt the Skywork-VI VL Reward (Wang al., 2025f) as the reward model for BoN and the VI.-Rethinker series (Wang et al., 2025b) as the GRPO-trained models. As shown in Table 4, the results demonstrate that: Across model scales from 7B to 72B, the average performance difference between other methods and CoT consistently de- creases, while an increasing number of subtypes transition from performance gains to performance drops (from green to red). Strikingly, on Qwen-2.5-VL-722
the performance of BoN and GRPO falls short of simply applying COT. According to previous works (Yue al., 2025b), hypothesize that this is because these methods primarily improve sampling efficiency towards correct reason- ing paths. For larger models, the likelihood of sampling correct paths is naturally higher, which methods.
diminishes the gains from
RL Generalizes Worse than BoN on Small Models. Reinforcement learning methods, exempli- fied by GRPO, have gained wide adoption for their strong reasoning generalization (DeepSeek-a Al et al., 2025). Nevertheless, our results in Table 4 reveal that all of the three models, GRPO exhibits weaker generalization compared to BoN. To further validate this finding, we conduct ex- periments additional small MLLMs (see Appendix G for details), comparing the performance of BoN@8 applied to base models with that of RL-trained models. The results in Figure 6 show that across different model architectures and training datasets, RL-trained models consistently underper- form compared to BoN inference on the corresponding base models. In some cases, RL models perform worse than the base models using COT. This calls for reconsideration of RL tech-
8

---

Published as a conference paper at ICLR 2026
niques: Do RL methods small models merely lead to overfitting specific datasets? We leave this question open for further exploration in future work.
4.3 Do DIFFERENT REASONING TYPES CORRELATE?
Former findings demonstrate significant differences in model performance across types. In this section, We aim capture the underlying relationships among these categories.
Correlations Between Reasoning Types. We compute the accuracy of models across reason- ing types, calculate the Pearson correlation coefficients between them, and present the results in Figure 6a. It demonstrates substantial differences correlations across these types. Some cate- gories, such as inductive and analogical reasoning, exhibit very high correlations (0.97), while some others, such as spatial and inductive reasoning, show low correlations (0.40).
Uncovering Pattern Clusters in Reasoning, Furthermore, we normalize the negative of the cor- relation coefficient the distance between categories and perform hierarchical clustering. In Figure 6b, we observe clusters formed by similar reasoning types (e.g.. Ana-Ind), suggesting the exis- tence of higher-order reasoning patterns in MILMs. For example, both analogical and inductive reasoning rely reasoning types with greater distances suggest that they involve relatively disjoint patterns. As an example, spatial reasoning is distant from all other categories, suggesting that the capabilities it re- quires (c.g., location, distance estimation) are difficult to learn from non- non-spatial tasks. In conclusion, MMR-Life enables to uncover higher-level hierarchy of reasoning patterns, facilitating deeper understanding of reasoning generalization across diverse tasks.
shared pattern of abstracting general rules from concrete features. Conversely,
5 ERROR ANALYSIS
This section focuses the errors made by GPT-5 and Gemini-2.5-Pro, the two strongest models on MMR-Life. For each model, domly select 20 incorrect examples from each reasoning type and identify the root causes of the model's erroneous responses. The distribu- tion of these errors shown in Figure 8, with selection of notable 42 cases and detailed anal- yses provided in the Appendix H. The results reveal that reasoning errors dominate at 32%, with the model frequently making basic logical mistakes such as causal inversion (24%), tem-
ane
Dreaning
Erannay Cni
Aumera Ener
Hatccaario Ove
US
(a) GPT-5
(b) Geimini-2.5-Prro
Figure 8: Error distribution over 140 errors for each model on MMR-Life.
poral confusion (42%), and missing key steps (24%) during reasoning. In addition, abstraction errors (17%), which reflect the model's short-term thinking capabilities, such as the ability to make associ- ations, are also notable. Knowledge errors (17%) and perception errors (12%) constitute substantial portions of failures, indicating challenges in recalling the correct knowledge for reasoning, well as difficulties in identifying static attributes of objects (e.g., color, shape) and dynamic changes (e.g.. movement). By systematically examining these failures, we only expose critical shortcomings in current MLLMs but also derive actionable insights that can inform the next generation of MLLMs.
RELATED WORK
Multimodal Reasoning Enhancement Methods. The development of methods in multimodal reasoning closely follows the approaches established in pure language processing. Inspired by its success in text-only settings, Co'' has recently been extended to MILMs, leading to the development of prompt-guided multimodal reasoning. Studies such as IPVR (Chen et al., 2023), CCoT(Mitra et al., 2024), and VisualSketchpad (Hu et al., 2024) combine reasoning with perception, enhancing the reliability of the reasoning process. After that, the search- search-based inference method brings reward models into the multimodal reasoning process, training scoring model to evaluate and select the best reasoning path (Wang et al., 2025d; Zang et al., 2025; Wang et al., 2025f). Recently, following the success of Deepseek-RI GRPO (DeepSeek-Al AT al., 2025), a group of thinking MILMS like
9

---

Published as a conference paper at ICLR 2026
VL-Rethinker (Wang et al., 2025b), MM-Eureka (Meng et al., 2025a), and MiMo- VL (Yue al., 2025c) have emerged. Our benchmark comprehensively evaluates different methods and MLLMs, aiming to guide their further optimization.
Multimodal Reasoning Benchmarks. There exists number of multimodal benchmarks testing MLLMs' reasoning abilities. Several studies combine world knowledge with reasoning and assess the reasoning capabilities of MILMs across various STEM fields, such as GPQA (Rein al., 2023), OlympiadBench (He et al., 2024), MME-CoT (Jiang et al., 2025), MMR-V (Zhu et al., 2025) and MMLU-Reason (Tie et al., 2025). Other works argue that reasoning should be decoupled from knowledge, using symbolic patterns to evaluate the model's logical reasoning abilities, such as Puz- zle VQA (Chia al., 2024), VisualPuzzles (Song et al., 2025), and MME-Reasoning (Yuan et al., 2025). However, both types of benchmarks exhibit deviations from real-life reasoning scenarios due to the expert-level knowledge and symbolic images. Although recent work on spatial reasoning meets real-life requirements (Yang et al., 2025a; Li et al., 2025b; Yang et al., 2025b), it covers only a limited set reasoning types. Our MMR-Lif benchmark covers seven different reasoning types and introduces real-life multi-image input, addressing former gaps.
7 CONCLUSION
We present MMR-Life, novel benchmark designed to evaluate the multimodal reasoning abil- ities of current MLLMs across seven distinct reasoning types using multiple real-life images as inputs. Through careful and diverse data curation, our dataset provides comprehensive evaluation of MLLMs' reasoning performance across various real-life scenarios, which shows that existing MLLMs still face significant challenges and exhibit notable performance imbalances across differ- ent reasoning types. We conduct a further analysis of the reasoning paradigms of these models, uncovering the relationship between the thinking length, enhancement methods, and reasoning abil- ities of MLLMs, which lays the foundation for the development of more generalizable AI systems.
ETHICS STATEMENT
In constructing benchmark, ensure strict adherence copyright and licensing regulations, explicitly avoiding data from sources that prohibit copying or redistribution. Besides, we avoid the images that contain any private information or harmful content. The data in our MMR-Life not intended to replace, nor are they capable of replacing, the original data source. Therefore, we assert that their inclusion does not affect the market value or utility of the original materials. We did not employ external crowdsourcing or paid annotation platforms. All participants volunteered, with complete understanding of the research goals, procedures, and the intended use of the data.
REPRODUCIBILITY STATEMENT
We have taken several steps to improve the reproducibility our research. Regarding the data, we provide thorough description of the data sources for each task, along with links, in Appendix C. subset of 210 items, including the questions and their corresponding images, is also uploaded in the supplementary materials. Additionally, we describe the dataset construction process and the prompts used in both $2.2 and Appendix C. On the experimental side, we offer detailed account of the model versions, parameter settings, and prompts used in the experiments, which are outlined in Appendix F.I. The full experimental code is also uploaded in the supplementary materials. We commit to making data and code open source if the paper is accepted.
ACKNOWLEDGEMENT
This work is supported by the National Natural Science Foundation of China (No. U24A20335, No. 62406321). This work is also supported by Beijing Natural Science Foundation (1.243006).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Authropic. Claude 3.7 sonnet and claude code, URL httpaa/tttt////..aaaaaaooippoppa news/ caadde- 3-7 sonnet. Accessed: 2025-09-21.
Anthropic. Introducing claude 4: Claude opus 4 and claude sonnet 4, 2025b. URL https://
www www. www.anthropic. com/ news / / cludeeeeee-- 4, Accessed: 2025-09-17.
Shuai Bai, Keqin Chen. Xuejing Liu, Jialin Wang, Wenbin Ge. Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Ming-Hsuan Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang. Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. CoRR, 120/250221192333 2025, doi: 10.48550/RXIV.2502. 13923. URL https: //tp:/ 08.44855//rxxi1 2502. 10002.13990 13923.
ByteDance Seed Team. Doubao-1.55 1.5-pro: exploring extreme balance between model performance and inference efficiency, 2025. URL https:/ //::://san /tt////eed.ee// wyt.eayce.ccom............ special/
Accessed: 2025-09-17.
Jiacheng Chen, Tianhao Liang, Sherman Siu, Zhengging Wang, Kai Wang. Yubo Wang, Yuansheng Ni, Ziyan Jiang, Wang Zhu, Bohan Lyu, Dongfu Jiang, Xuan He, Yuan Liu, Hexiang Hu, Xiang Yue, and Wenhu Chen. Mega-bench: Scaling multimodal evaluation to over 500 real-world tasks. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, OpenReview.net, 2025. URL htt.httss://openreee http://pppervviiieeee ew. net forum?id 2rWbKMmOuM.
Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Hao Zhang, and Chuang Gan. See, think, confirm: Interactive prompting between vision and language models for knowledge-basee visual reasoning. CoRR, abs/2301 005226, 2023. doi: 1044550/ARXIV:2300..55 05226.
URL
https:://oi..nn wtt.ddo.oog/.....00000 2.0...855//aaaa 2...0/axii.....
Ziming Cheng, Binrui Xu, Lisheng Gong, Zuhe Song, Tianshuo Zhou, Shiqi Zhong, Siyu Ren, Mingxiang Chen, Xiangchao Meng, Yuxin Zhang, et al. Evaluating miims with multimodal multi- image reasoning benehmark. arXiv preprint arXiv:2506.0428) 2025.
Yew Ken Chia, Vernon Toh, Deepanway Ghosal, Lidong Bing, and Soujanya Poria. Puzzlevqa: Diagnosing multimodal reasoning challenges of language models with abstract visual pat- terns. In I Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Compurational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pp- 16259-16273. Association for Computational Linguistics, 2024. doi: 10. 18653/N1/024FFINIIIGS -CD.INGGS-AAL.62222 URL https: http:://doiooo///0//// findings acl. 962.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inder- jit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpek- tor, Nan-Jiang Jiang. Krishna Haridasan, Ahmed Omran, Nikunj Saunshi, Dara Bahri, Gaurav Mishra, Eric Chu, Toby Boyd, Brad Hekman, Aaron Parisi, Chaoyi Zhang, Kornaaphop Kaw- intiranon, Tania Bedrax- Weiss, Oliver Wang, Ya Xu, Ollie Purkiss, Uri Mendlovic, Ilii Deu- tel, Nam Nguyen, Adam Langley, Flip Korn, Lucia Rossazza, Alexandre Ramé, Sagar Wagh- mare, Helen Miller, Nathan Byrd, Ashrith Sheshan, Raia Hadsell Sangnie Bhardwaj, Pawel Janus, Tero Rissa, Dan Horgan, Sharon Silver, Ayzaan Wahid, Sergey Brin, Yves Raimond, Klemen Kloboves, Cindy Wang, Nitesh Bharadwaj Gundavarapu, Ilia Shumailov, Bo Wang, Mantas Pajarskas, Joe Heyward. Martin Nikoltchev, Maciej Kula, Hao Zhou, Zachary Garrett, Sushant Kafle, Sercan Arik, Ankita Goel, Mingyao Yang. Jiho Park. Koji Kojima, Parsa Mah- moudieh, Koray Kavukcuoglu, Grace Chen, Doug Fritz, Anton Bulyenov, Sudeshna Roy, Dim- itris Paparas. Hadar Shemtov, Bo-Juen Chen, Robin Strudel, David Reitter, Aurko Roy, An- drey Vlasov, Changwan Ryu, Chas Leichner, Haichuan Yang. Zelda Mariet, Denis Vnukov, Tim Sohn, Amy Stuart, Wei Liang, Minmin Chen, Praynaa Rawlani, Christy Koh, JD Co- Reyes, Guangda Lai, Praseem Banzal, Dimitrios Vytiniotis, Jieru Mei, and Mu Cai. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality. long context, and next genera- tion agentic capabilities. CoRR, abw/2507.062265 2025, doi: 10.48550/ARXIV.5500 06261.. URL https: //o. orgg/10.0......... 48.504855//444 vxvi f 2507 06261. 26116006666.611111111

---

Published as a conference paper at ICLR 2026
Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M. Asano. Lost in time: A new temporal benchmark for videollms, 2025. URL abs/ 22100.210.000 07752.
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang. Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei. Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An. Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li. Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang. Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepscek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning, 2025. URL https: //sttps:/aate /ttp///aavvvv//a////a//ee./a.ava org/ abs 2bs/25012 2501 12948.
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai- Kei Chang. Openvlthinker: An early exploration to complex lasion-language reasoning via iterative self-imprrvement, CoRR, abs/2503. 17352, 2025. doi: 10.48550/ARXIV2550..1755 URL https: //doi.///i..e http///oo..ogg.../ 48550/ 21.50//rxiv.. 2503.103311111 17352.
Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang, Dehao Zhang. Dikang Du, Dongliang Wang. En ming Yuan, Enzhe Lu, Fang Li, Flood Sung, Guangda Wei, Guokun Lai, Han Zhu, Hao Ding, Hao Hu, Hao Yang. Hao Zhang, Haoning Wu, Haotian Yao, Haoyu Lu, Heng Wang, Hongcheng Gao, Huabin Zheng. Jiaming Li, Jianlin Su, Jianzhou Wang, Jiaqi Deng, Jiezhong Qiu, Jin Xie, Jinhong Wang, Jingyuan Liu, Junjie Yan, Kun Ouyang, Liang Chen, Lin Sui, Longhui Yu, Mengfan Dong, Mengnan Dong, Nuo Xu, Pengyu Cheng, Qizheng Gu, Runjie Zhou Shaowei Liu, Sihan Cao, Tao Yu, Tianhui Song, Tongtong Bai, Wei Song, Weiran He, Weixiao Huang, Weixin Xu, Xiaokun Yuan, Xingcheng Yao, Xingzhe Wu, Xinxing Zu, Xinyu Zhou, Xinyuan Wang, Y. Charles, Yan Zhong, Yang Li, Yangyang Hu, Yanru Chen, Yejie Wang. Yibo Liu, Yibo Miao, Yidao Qin, Yimin Chen, Yiping Bao, Yigin Wang, Yongsheng Kang, Yuanxin Liu, Yulun Du, Yuxin Wu, Yuzhi Wang, Yuzi Yan, Zaida Zhou, Zhaowei Li, Zhejun Jiang, Zheng Zhang, Zhilin Yang, Zhiqi Huang, Zihao Huang, Zijia Zhao, Ziwei Chen, and Zongyu Lin. Kimi- vi technical report. CoRR, abs/2504. 07491, 2025. doi: 10.4850/ARXIV2500.0774 /etts://doi..r /850://00....0000..///00/1 arxiv. 2504.
URL
eBird. ebird: Explore a world of birds, 2025. URL 2025-09-21.
Accessed:
fimena14. Crowd counting. Kaggle dataset, 2025. URL https: www...aggle.... com/ datasets/ fmenal 4/ crowo- count i igg Accessed: 2025-09-24.
12

---

Published as a conference paper at ICLR 2026
Nikolas Gégénava. Popular sneakers classification. Kaggle dataset. 2025. URL https: /www.
kaggle com/ dat /tt.aatss/ikooossseeeeeaaae www.ttteeeessssiiiaaiiiioooo..o. Ac-
cessed: 2025-09-24.
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang. Yuxiang Zhang, Jie Liu, Lei Qi. Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Lun-Wei Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Mceting of the Association for Computational Linguistics (Volume I: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pp. URL
Association for Computational Linguistics, 2024. doi: 10. acl -ong. 211.
https://doi org/ 18651866333
Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Ranjay Krishna. Visual sketchpad: Sketching for multimodal language models. In Amir Globersons, Lester Mackey, Danielle Bel- grave. Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL http::/ EB200000077677222000005566111111 attp://ooor....e.--------------- Cow.rraacct-Cffffeeee.ttttt html.
visual chain of thought
n ni htwppps.c//aaperffeee/////aeeeee
ikarus777. Best artworks of all time. Kaggle dataset, 2019. URL UtLhttps:://// www.le...l.....m..oo........ com/ datasets/ http://rasaaeeeeeaaaaaoa-------- time. Accessed: 2025-09-24.
Varun Jampani, Kevis-Kokitsi Maninis, Andreas Engelhardt, Arjun Karpur, Karen Truong, Kyle Sargent, Stefan Popov, André Araujo, Ricardo Martin-Brualla, Kaushal Patel, Daniel Vlasic, Vittorio Ferrari, Ameesh Makadia, Ce Liu, Yuanzhen Li, and Howard Zhou. Navi: Category- agnostic image collections with high high-quality 3d shape and pose annotations, 2023. URL https: / //arxiv. org abs/ 2306. 09109.
Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi. Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, Bo Zhang, Chaoyou Fu, Peng Gao, and Hongsheng Li. Mme-cot: Benchmarking chain-of-thoughe in large multimodal models for reasoning quality, robustness, and efficiency, CoRR, abs/2502.009211. 2025. doi: (044550AAXXV22500002222 URL https: //doi. org 10. 48550 2502. 09621.
Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Omni-reward: Towards generalist omni-modal reward modeling with free-form pref- erences. CoRR, abs/2510,23451, 2025. doi: 10,48500AR412251024411. URL https:
(/doi.org/10 48550 2510 23451.
Kaggle. Kaggle: Your machine learning and data science community, 2025. URL htttp:://www kaggle. com/ Accessed: 2025-09-21.
Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Eti- enne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Róbert Busa-Fekete, Alex Feng. Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, lain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan- Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang. Antonia Pater- son, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Char- lie Chen, Charline Le Lan, Christopher A. Choquette-Chie CCho, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng. Dimitris Paparas, Divyashree Shivaku- mar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle. Eric Noland, Erwin Huizenga, Eu- gene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna
Geoffrey Cideron,
13

---

Published as a conference paper at ICLR 2026
Heinezak-Plucinssee Harman Singh. Harsh Mchta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wi- eting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju-yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Cul- liton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu. Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willooughy, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Pöder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evei, Vedant Misra, Vincent Roseberry, Vlad Fein- berg. Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle K. Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clément Farabet, Elena Buchatskaya, Jean- Baptiste Alayrac. Rohan Anil, Dmitry (Dima) Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma 3 technical report. CoRR, abs/2503. 19786, 2025. doi: 10.48550/ARXIV.500..U0 https: /oo. .rggg0....0..................... 48550/ arxiv. 2503. 1.786.
URL
Jihyung Kil, Zheda Mai, Justin Lee, Arpita Chowdbury, Zihe Wang, Kerrie Cheng, Lemeng Wang, Ye Liu, and Wei-Lun Chao. Molm-coopbeech: A comparative reasoning bench- mark for multimodal llms. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 IS, 2024, 2024. URL http: /tap:r//paperss. nips. cc pocpppppr_ffie files ftapperrfilessppper//0000 2attaaa52222---brrccee -bstract-Datasett Datasets _nd
Neural Information
Benchmarks Track html.
Dingming Li, Hongxing Li, Zixuan Wang. Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, Weiming Lu, and Yueting Zhuang. Viewspatial- bench: Evaluating multi-perspeectve spatial localization in vision-language models, 2025a. URL https: wawaiiiivvva/..a/aaaaaaaaaaaavva org/ /bs/2505 21500.
Dingming Li, Hongxing Li, Zixuan Wang, Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqiao Zhang, Yongliang Shen, Weiming Lu, and Yueting Zhuang. Viewspatial-bench: Evaluating multi-perspective spatial localization in vision-language mod- doi: 10448500AARXIV2255521550 URL https:
els. CoRR, abs/2555522100 2025b.
///doi.org///// 48550/ 2505. 21500.
Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Daojian Zeng. Kang Liu, and Jun Zhao. Focus on your question! interpreting and mitigating toxic cot problems in commonsense reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2206-9230. 2024.
Jiachun Li, Pengfei Cao, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. Towards better chain-of-thought: A reflection on effectiveness and faithfulness. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 August 2025, pp. 10747-10765. Association for Computational Linguisties, 2025c. URL https: / / wattaaollaoo logyy org/ 2025. fi findings-aan aasssacl.55. 560/.
Jiachun Li, Pengfei Cao, Yubo Chen, Jiexin Xu, Huajjun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. Rewarding curse: Analyze and mitigate reward modeling issues for LLM reasoning. CoRR, http/2503.05888 2025d. doi: 00.48550/ARXIV.2503.000 05188. URL https: /doi. org/10.
48550/arX00 2503. 05188.
14

---

Published as a conference paper at ICLR 2026
Jiachun Li, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao. MIRAGE: evaluating and explaining inductive reasoning process in language models. The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28. 2025. OpenRe- view.net, 2025e. URL /ttp::://operreviii ew, net forum?? d=
Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang. Chunfeng Yuan, Bing Li, and Weiming Hu. Mibench: Evaluating multimodal large language models over multiple images. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL. USA, November 12-16, 2024, pp. 22417 22428.. Association for Computational Linguistics, 2024. doi: 10. 18653/V //doi. org 10. (1/88663112222
WEIEEENNNLP-MIA 1a-mllppmman 1250.
URL https:
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL. htttps://ppenre eww net/ Forum? d=
Jiayuan Mao, Xuelin Yang, Xikun Zhang, Noah Goodman, and Jiajun Wu. Clevrer-humans: De- scribing physical and causal events the human way. Advances in Neural Information Processing Systems, 35:7755-7768, 2022.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, and Wenqi Shao. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule- based reinforcement learning, 2025a. URL https: //ta/axiivv./////a// org. http:/bbs//500/ 2503. 07365.
Fanqing Meng. Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Tianshuo Yang. Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, Kaipeng Zhang, and Wenqi Shao. MMIU: multimodal multi- image understanding for evaluating large vision- laiionlanguage models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenRe- view.net, 2025b. URL https: //ppenrvv///e//evvii/eee ew net/ forum? Www.
Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA. USA. June 16-22, 2024, pp. 14420-14431. IEEE, 2024. doi: 10. (W00//CVPP55273.2224...111 CVPR52733 2024 01367.
URL https: / /ooi ott.ooorgg11...00000 1109/
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Candès, and Tatsunori Hashimoto. sl: Simple test- time scaling. CoRR, abs/2501.19393 2025. doi: 10:4850/ARXXV2250119999 URL https: 48550/8550/4
//doi. 1tl.lloogg1..485000.. aar.vv.iivvvvv.////// 2501. 19393.
OpenAl. Hello gpt-4o, 2024. URL https: htt.attppp///gee...ccmmm///////a
Ac-
cessed: 2025-09-17.
OpenAl. Introducing gpt-4.1 in the api, 2025a.
URL http:://opeai..com/nne//
gpt 4-1 Accessed: 2025-09-17.
OpenAl. Introducing gpt-5, 2025b. introducing-gpt-S5 Accessed: 2025-09-17.
URL https://ppeaal.com/neeee
OpenAl. Openai o3 and 04-mini system card, 2025c. URL UUKLhttps//ppeeii.ccm///eee
03-o4-min miiimmiiiiee mimiimiiteeennmemmmme card/. Accessed: 2025-09-17.
Paritosh Parmar, Eric Peh, Ruirui Chen. Ting En Lam, Yuhan Chen, Elston Tan, and Basura Fer- nando. Causalchaos! dataset for comprehensive causal action question answering over longer causal chains grounded in dynamic visual scenes, 2024. URL https: //ar:ii///avvv.// www.ii.org///..oeoooooooo.ooooor abs/
2404.01295
15

---

Published as a conference paper at ICLR 2026
patricklford.
Black
jack interactive card game.
Kaggle dataset,
2025.
URL https://veit http:/wwwwwww.a..aaaa//a com/ datasets sppptrickfoorrdd
black- htw.:aaccaaaetatatteeee--rr----r
Accessed: 2025-09-24.
Yi Peng, Chris, Xiaokun Wang. Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu, and Yahui Zhou. Skywork RIV: pioneering multimodal reasoning with chain-of-thought. CoRR, abs/2504.05599, 2025. doi: 10.48500ARXIV22504..000 05599.
URL https::t///eet wtt.b//..rr/1....///aaaa....
Gerald Piosenka. 100 sports image classification. Kaggle dataset, 2022. URL https://www. Accessed: 2025-
kaggle. hte//ataaseeete https//ppppppappeeeeeeeeassaaeea swwpiksaaspsssssoooeeeooooe 09-24.
Qwen Team. Qvq: To see the world with wisdom. https:://wwnlii htee//wwwwmmiiiiiime github..hhh......g. io/blog/ qvq 72b- -ebbbereeeww, December 2024. Accessed: 2025-09-24.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. CoRR, abs/2311. 1.23111.1222,, 2023. doi: 1048550/ARXIV2311.2222 URL https:
//doi.org/10 00tg//1..885000 /arr/aaxivvvv 2311 12022.
sanadalali.
Animal kingdom (90):
Masters of survival. Kaggle dataset, URL. https://wwet kaggle. com/ homm/datsettss//// aalilali/lla///aaa Accessed: 2025-09-24.
2025.
Yueqi Song, Tianyue Ou, Yibo Kong, Zecheng Li, Graham Neubig, and Xiang Yue. Visualpuzzles: Decoupling multimodal reasoning evaluation from domain knowledge. CoRR, abs/2504.10342, 2025. doi: 10.48550/ARXIV.250.. 10342. URL httptttttt.... /ttp:://ddoo.// 2504.10342
Zayne Rea Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot or not to cot? chain-of- ththugh helps mainly math and symbolic reasoning. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenRe- view.net, 2025. URL https: httpptt://ppppeeeeeeeeee /wtp//ppeeeeveeeeeeeeeeeeeeeeeee net/ forum?id:wweeee c58kkn.
Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Xiangyang Liu, Hang Yan, Yunfan Shao, Qiong Tang, Shiduo Zhang, et al. Moss: An open conversational large language model. Machine Intelligence Research, pp. 1-18, 2024.
Kwai Keye Team, Biao Yang, Bin Wen, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, Fan Yang, Guorui Zhou, Hao Peng. Haojie Ding, Jiaming Huang, Jiangxia Cao, Jiankang Chen, Jingyun Hua, Jin Ouyang, Kaibing Chen, Kaiyu Jiang. Kaiyu Tang, Kun Gai, Shengnan Zhang, Siyang Mao, Sui Huang, Tianke Zhang, Tingting Gao, Wei Chen, Wei Yuan, Xiangyu Wu, Xiao Hu, Xingyu Lu, Yang Zhou, Yifan Zhang, Yiping Yang, Yulong Chen, Zhenhua Wu, Zhenyu Li, Zhixin Ling, Ziming Li, Dehua Ma, Di Xu, Haixuan Gao, Hang Li, Jiawei Guo, Jing Wang, Lejian Ren, Muhao Wei, Qianqian Wang, Qigen Hu, Shiyao Wang, Tao Yu, Xinchen Luo, Yan Li, Yiming Liang. Yuhang Hu, Zeyi Lu, Zhuoran Yang, and Zixing Zhang. Kwai keye-vl technical report. CoRR, abs/2507.019 01949, 2025. doi: 10.48550/ ARXIV.2507.0194. .19949. URL https: doi.oroooooooooooodo 18..0....0.1.. htt.s///axiv..o 2507.2500..0 2507.01949.
Guiyao Tie, Xueyang Zhou, Tianhe Gu, Ruihang Zhang. Chaoran Hu, Sizhe Zhang, Mengqu Sun, Yan Zhang, Pan Zhou, and Lichao Sun. Mmlu-reason: Benchmarking multi- mudti- lan- guage understanding and reasoning, 2025. URL https:::: //tpxi/ivvv//////////////////wiv org
abs/2505 hwtp://ggg/bb////eeeeeee
16459.
vipoooool. New plant diseases dataset. Kaggle dataset, 2024. URL https::///// : //www//wwww///../ kaggle. com/ www.aassssvvpooooooooooooooo moreoooooo/wwwooeoooo www.///ewpppeeeee ppllttlantt-- diseases dataset. Accessed: 2025-09- 24.
Chenhao Wang. Jiachun Li. Yubo Chen, Kang Liu, and Jun Zhao. A survey of recent advances in commonsense knowledge acquisition: Methods and resources. Machine Intelligence Research, pp. 1-18, 2025a. doi: 10.1007/s1 11337//11633-221117777 1471-3.
16

---

Published as a conference paper at ICLR 2026
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. CoRR, abs/2504.08837, 2025b. doi: 10448550ARRXV2250000005 48550/arxiv 2504. 08837.
URL https::/don org/10.
Peijie Wang, Zhong-Zhi Li, Fei Yin, Dekang Ran, and Cheng- Lin Liu. MV-MATH: evaluating multimodal math reasoning in multi-visual contexts. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pp. 19541-
19551. Computer Vision Foundation IEEE, 2025c. doi: 10.1 10.1009CCVRR573332255551 URL https: /tppaa/c///ecccsss..s. theovf. wttp///ccocoooooooooomooooeooo com/ http://ooneettteeeeeeeeeee htttm /Wang_ Matt wwwaahaeaasninnninnnnoo wac_iiinnuuutttii htmm.
WwV-MATTL_EVaauat.ngggnuuu Contexts_CVPR_2ODE
Weiyun Wang. Zhangwei Gao, Lianjje Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, Lewei Lu, Haodong Duan, Yu Qiao, Jifeng Dai, and Wenhai Wang. Visualprm: An effective process reward model for multimodal reasoning. CoRR, abs/2503.10291, 2025d. doi: 1048550AARXIV2505.0029 URL https: /ttp///o.oooo//////ooo wtg./.....g..... 48550/arXiv 2503. 10291.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing. Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang. Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang. Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang. Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang. Huipeng Deng. Biqing Qi, Jiaye Ge. Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu. Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. Internv13.5: Advancing open-source multimodal models in versatil- ity, reasoning, and efficiency, 2025e. URL https: / wrwiv. org/ abs /2508.50800 18265.
Xiaokun Wang, Peiyu Wang, Jiangbo Pei, Wei Shen, Yi Peng, Yunzhuo Hao, Weijie Qiu, Ai Jian, Tianyidan Xic, Xuchen Song, Yang Liu, and Yahui Zhou. Skywork-vl An effective reward model for multimodal understanding and reasoning. CoRR, abs/2505.07263, 2025f. doi: 10. 48550/ARXIV.2500.077
URL https: /oii. org/10 48550 www...iww................
Yuqi Wang. Ke Cheng, Jiawei He, Qitai Wang, Hengchen Dai, Yuntao Chen, Fei Xia, and Zhao- Xiang Zhang. Drivingdojo dataset: Advancing interactive and knorledge-enriched driving world model. Advances in Neural Information Processing Systems, 37: 17:10300013034, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of-thouuht prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agar- wal, Danielle Belgrave, K. Cho, and A. Oh (eds.). Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL http://papers: nips. http://eeeeppppeeeeeeeeeeeeeeee html. 0a5000011522eccc1155aaaooooo
Zhiheng Xi, Guanyu Li, Yutao Fan, Honglin Guo, Yufang Liu, Xiaoran Fan, Jiaqi Liu, Jingchao Ding, Wangmeng Zuo, Zhenfei Yin, Lei Bai, Tao Ji, Tao Gui, Qi Zhang, Philip Torr, and Xuanjing Huang. BMMR: A large-scale bilingual multimodal multi-discipline reasoning dataset. CoRR, doi: 10.4850/ARXIV.250000555 48550/arxiv 2507 2507.03483.
fbs/2507.034483 2025.
URL /ttt:s://doi.oratt
Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Skizler-Cinbii Recipequ: A challenge dataset for multimodal comprehension of cooking recipes, 2018. /tatttps///aai.....///
abs/ /tas/1809. 00812.
Jihan Yang, Shusheng Yang, Anjali W. Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Think- ing in space: How multimodal large language models see, remember, and recall spaces. In
17

---

Published as a conference paper at ICLR 2026
IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN. USA, June 11-15, 2025, pp.
URL Computer Vision Foundation / IEEE, 2025a.
doi: 00.1099CCVRS373342222..00999
theevf. com/ moddll
Thin Large__aanguage_Mddeee MoteeeeeMMddlsscc Se____eee___eeeeeeeeerrr. http://wwggggpooooooeeeeeeeeeeea
Sihan Yang. Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, Dahua Lin, Tai Wang, and Jiangmiao Pang. Mmsi-bench. A benchmark for multi-image spatial intelligence. CoRR, (bss/2505.23764 2025b. doi: 10.48550/
ARXIV.2505.2376 URL www.t:ttp//.// http:///ddo.oo.o///./e./oo/ httg//...855//rxiii 2ttp://.a..v...5....... 2575.23764....
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang. Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. R1-onevision: Advancing general- ized multimodal reasoning through cross-modal formalization. CoRR, abs/2503. 10615, 2025c. doi: 10.48550/RRXV.25005 10615. URL https://doi..... www.iiiorg111.. 4854048555511 2000/ 10615.
Jiakang Yuan, Tianshuo Peng, Yilei Jiang, Yiting Lu, Renrui Zhang, Kaituo Feng, Chaoyou Fu, Tao Chen, Lei Bai, Bo Zhang, and Xiangyu Yue. Mme-reasoning: A comprehensive benchmark for logical reasoning in milms. CoRR, abs/2505.21327, 2025. doi: 10.48550AAXXV225522137... URL hwtph:tp/////to //oii www.oo.../.... 48550/ar5iiii. 2505. 21327.
Xiang Yue, Yuansheng Ni, Tianyu Zheng. Kai Zhang, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang. Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A massive multi-discipline multimodal understanding and reason- ing benchmark for expert AGI. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. 9556-9567. IEEE, 2024. doi: C0.110//CVRR5733.22222.2. 2024. 00913.
URL https: pestps://oii. /twii//iiio//////////// www/oo...ooooooooooooooo....o..e 10. 1009///P2/22233
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilchvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July August 1, 2025, pp. 15134-15186. Association for Computa- tional Linguistics, 2025a. URL https: wwt.aaaaanthhtcoooecc
2025 acl-lcllonnnl 736/.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in 1lms beyond the base model? CoRR, abs/2504.13837, 2025b. doi: 1044850/ARXIV.250..10 13837. URL https:
//doi. www. 48554/8555//// 2504 13837.
Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Xiao-Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye. Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan. Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song. Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai, and Bingquan Xia. Mimo-vl technical report. CoRR, abs/2500.00369, 2025c. doi: 10.48550/ARXIV.5566.00 doi www. 48550/85044 arviv 2506. 03569.
URL https://
Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Ziyu Liu, Shengyuan Ding, Shenxi Wu, Yubo Ma, Haodong Duan, Wenwei Zhang, Kai Chen, Dahua Lin, and Jiaqi Wang. Internim- xcomposer2.5-rewain A simple yet effective multi-modaa reward model. CoRR, abs/2501.113668, 2025. doi: 00.4850//RXII.25011122 2501. 12368.
URL https: / /oii. wrw..o.gg..... org/10.48550/ar2k.
18

---

Published as a conference paper at ICLR 2026
Zhicheng Zheng, Xin Yan, Zhenfang Chen, Jingzhou Wang, Qin Zhi Eddie Lim, Joshua B Tenen- baum, and Chuang Gan. Contphy: continuum physical concept learning and reasoning from videos. In Proceedings of the 41st International Conference on Machine Learning, pp. 61526- 61558, 2024.
Kejian Zhu, Zhuoran Jin, Hongbang Yuan, Jiachun Li, Shangqing Tu, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. MMR-V: V: what's left unsaid? A benchmark for multimodal deep reasoning in videos. CoRR, abs/2506.04141, 2025, doi: 1048550/ARXX:225000044444 URL https:://ds wrgg/0..48500000... /awvvv 2506.00144
19

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
In this study, a large language model (LLM) was employed as a tool to assist in the refinement and enhancement of the manuscript's language. The specific usages of the LLM include:
Grammar and Syntax Improvement: The LLM helped to correct grammatical errors and improve sentence structures, contributing to greater clarity and fluency in the writing.
Conciseness and Precision: provided suggestions for more concise and precise wording, aiding in the refinement of certain sections without altering their meaning.
It important note that while the LLM contributed to the refinement of the manuscript's language, the research ideas, data analysis, and conclusions were independently conceived and developed by the authors. The LLM's contributions were exclusively related text refinement and did not extend to the conceptual aspects of the study.
B KEY CONCEPTS IN MMR-LIFE
We begin by discussing key concepts in the benchmark to clearly define the core problems and design principles that our work addresses.
B. REASONING IN REAL-LIFE SCENARIO
As real-life reasoning is a fundamental design principle of our benchmark, we provide a brief defi- nition of it:
Definition 1 (Reasoning in Real-life Scenarios). Reasoning in real-life scenarios refers to the process of applying diverse reasoning capabilities to solve problems from everyday situations, which defined by a images and textual descriptions that satisfy the following conditions:
(i) Multiple natural images: The inpul must contain multiple images, each depicting objects events that either objectively exist in real life are realistically simulated resemble real-world conditions. Purely abstract diagrams or symbolic renderings are excluded.
(ii) Commonsense solvability:
The answer to the problem must not rely on complex domain- specific knowledge. Instead, it should be solvable using only basic human commonsense reasoning and general logic.
As mentioned in $1, the two existing benchmark types do not fully adhere to the above definition, they often incorporate unnatural images, such charts and synthetic puzzles, and may require specialized domain knowledge. In contrast, MMR-Life is constructed in strict accordance with the above definition, emphasizing the evaluation of reasoning in real-life scenarios from the outset. It should be noted that this definition is not intended to be broadly applicable but serves as the guiding principle for the design of this study.
B.2 MULTI-IMAGE VS. VIDEO
In $1, we noted that real-life images are continuous, which led us to adopt multi-image input. How- ever, a natural question arises: why not use continuous videos instead? In the following, we compare and discuss this choice. Overall, we opt not use video as input format for the following rea- sons:
Low Reasoning Types Coverage: The relationship between multiple images in video is typically limited to temporal sequencing. In this context, it is difficult to design reasoning tasks, such as analogy inductive reasoning, since these tasks often require parallel relationship between the images, which cannot be fully captured by video input.
Low Data Diversity: From a data perspective, as discussed in $2.2, real-world videos are only subset of our image sources. If all inputs were required to be videos, would lose a significant variety data sources, such as natural photographs, thereby reducing data diversity.
20

---

Published as a conference paper at ICLR 2026
Table 5: Data sources and image types of different tasks in MMR-Life
Reasoning Type
Task Type
Image Type
Data Source
Human Activity Attribution
Domestic Life
TVhench (Cores el al., 2025)
Abductive
Character Interaction Atribution
Hurnan Animation
Tom Jerry Cartoon (Purmar el al., 2024)
Multi Holt Collision Attribution
Physical Phenomenon
CLEVRER CLEVRER-Human, Humans (Mao al., 2022)
Animal Relation Inference
Natural Creutures
Kaggle (sanadalalii 2025)
Analogical
Product Similarity Inference
Product Shots
Koggle (Gégénava, 2025)
Artwork Style Inference
Human Artwork
Kaggle (ikarus77,, 2019)
Character Interaction Prediction
Hurnn Animation
Tom & Jerry Cartoon (Parmar et al., 2024)
Causal
Multi-Hop Collision Prediction
Physical Phenomenon
CLEVRER-Human, (Mao al., 2022)
Counterfactual Fluid Prediction
Physical Phenomenon
ComPhy (Zheng 2024)
Material Composition Deduction
Everyday Objects
MatVisa (Lu 2024)
Deductive
Card Winner Deduction
Game Symbols
Kaggle (patricklfford, 2025)
Recipe Step Deduction
Daily Dining
RecipeQA (Yagcioglu ct, 2018)
Bird Migration Induction
Migration Map
eBird (cBird, 2025)
Inductive
Plant Disease Induction
Pathology Photos
Kaggle (vipoooool, 2024)
Sport Feature Induction
Sports Activities
Kaggle (Piosenka, 2022)
Relative Position Estimation
Interior Views
ViewSpatial -anchal-Benhhhhh ct 2025a)
Spatial
Camera Rotation Estimation
Everyday Objects
NAVI (Jampani al., 2023)
Navigation Roule Planming
Interior Views
ViewSpatial- Bench (Li et al., 2025a)
Crowd Timeline Reconstruction
Crowd Surveillance
Kaggle (fmena14, 2025)
Temporal
Driving Sequence Prediction
Traffic Scene
Drivingdojo (Wang et al., 2024)
Human Activity Localization
Domestic Life
TVBench (Cores al., 2025)
High Noise in Input: In video-based benchmarks, frames typically sampled from videos and input into the model, which can introduce many irrelevant frames that inter- fere with reasoning. While this setup is closer real-world scenarios, our benchmark aims to directly assess the model's reasoning abilities, minimizing interference from other factors.
C DETAILS OF ANNOTATION PROTOCOLS
This section presents additional details of task annotation pipeline and protocols, providing complete details for $2.2 of the main paper.
C.I DATA SOURCES OF DIFFERENT TASKS
Table 5 presents the data sources for all the tasks included in MMR-Life, During the data collection phase, all annotators strictly adhere to copyright and licensing regulations the source sites or datasets. Moreover, following Definition I, we limit the dataset strictly to natural images, explicitly excluding symbolic diagrams and other non-photographic forms.
C.2 ANNOTATION GUIDELINES
During the annotation of questions and golden answers, all annotators were given the following guidelines:
All questions must contain multiple images (at least two images).
All questions should be written in English.
All questions should be solvable without complex domain-specific knowledge.
The question should not be ambiguous and be answered with one option.
The questions should adhere to the definitions of the respective reasoning types (see $2.2), ensuring clear differentiation between tasks of different reasoning types.
C.3 PROMPTS FOR NEGATIVE OPTION GENERATION
We list negative option generation prompts from Figure 11 to Figure 17.
21

---

Published as a conference paper at ICLR 2026
D DATA DIVERSITY OF MMR-LIFE
We demonstrate the diversity of data in MMR-Life in this section, where Figure 9 visualizes the variety of image types and Figure 10 presents the distribution of input image counts. The various types tasks included in our study are illustrated Appendix
E
TASK DETAILS
In this section, we give detailed description of each task presented in MMR-Life.
E.1 ABDUCTIVE REASONING
E.1.1 HUMAN ACTIVITY ATTRIBUTION
Task Description. This task tests a model's reasoning about human behavior motivations. By observing people's s behavior in given context, the model must analyze environmental clues and behavior cues to select the most plausible motivation among candidate explanations.
Examples.
See Figure 19, 20.
E.1.2 CHARACTER INTERACTION ATTRIBUTION
Task Description. This task requires the model understand causal relationships between charac- ters (e.g., in Tom & Jerry). Given scene of interaction, the model must analyze character behaviors and situational factors to infer the most reasonable cause for specific event outcome.
Examples.
See Figure 21, 22.
E.1.3 MULTI-HOP COLLISION ATTRIBUTION
Task Description. This task assesses model's causal reasoning in complex physical collision chains. In scenes involving multiple objects colliding, the model must trace the collision chain and identify the root cause triggering event for given outcome.
Examples.
See Figure 23, 24.
E.2 ANALOGICAL REASONING
E. ANIMAL RELATION INFERENCE
Task Description. This task requires models understand visual analogical relationships between animals. Given three animal images, the model must recognize the relational pattern between the first two animals and then select fourth animal from the options so that the relation between the third and fourth animals matches the original pattern.
Examples.
See Figure 25, 26.
E.2.2 PRODUCT SIMILARITY INFERENCE
Task Description. This task assesses a model's reasoning about product style preference. Based person's ''s owned or disliked product samples, the model must analyze design features and style preferences to recommend, from candidate options, product that best suits their intentions or tastes.
Examples.
See Figure 27, 28.
22

---

Published as a conference paper at ICLR 2026
E.2.3 ARTWORK STYLE INFERENCE
Task Description. This task evaluates model's understanding and recognition of artistic style. Given multiple sample works from the same artist, the model must learn the distinctive stylistic features and then identify which candidate option is most likely also created by that artist.
Examples.
See Figure 29, 30.
E.3 CAUSAL REASONING
E.3.1
CHARACTER INTERACTION PREDICTION
Task Description. This task tests model's ability to predict outcomes of interactions between animated characters. Given a specific behavior event by character, the model must use contextual understanding and character relations to predict the most likely follow-up reaction result.
Examples.
See Figure 31,32. 32.
E.3.2 MULTI-HOP COLLISION PREDICTION
Task Deseription. Given a sequence of consecutive images capturing object motion from initial to current time, the model must reason about the underlying physics and simulate possible multi-stage collision propagation, ultimately predicting the most likely next collision event or chain reaction.
Examples.
See Figure 33, 34.
E.3.3 COUNTERFACTUAL FLUID PREDICTION
Task Description. This task examines model's counterfactual reasoning ability in fluid dynam- ics. The model must analyze how a fluid flows and, if a barrier is removed, predict how the flow would change (i.e., determine the altered flow paths) and final positions under the new condition.
Examples.
See Figure 35, 36.
E.4 DEDUCTIVE REASONING
E.4.1 MATERIAL COMPOSITION DEDUCTION
Task Description. This task requires complex combinatorial reasoning about material composi- tion. Given different types and quantities of material components and the material requirements for certain products, the model must calculate how many products can be produced under the current material constraints.
Examples.
See Figure 37,38, 38.
E.4.2 CARD WINNER DEDUCTION
Task Description. This task examines model's understanding of Texas Hold 'em poker rules and logical reasoning. multiplayer poker game, each player has hole cards and there are community cards the board; based these, the model must analyze the best possible hand for each player and determine the winner.
Examples.
See Figure 39, 40.
E.4.3 RECIPE STEP DEDUCTION
Task Deseription. This task requires understanding the logical order of cooking processes. Given a dish name and a set of unordered images depicting stages of preparation, the model must deduce the correct cooking sequence based ingredient states, tool usage, and causal relationships.
23

---

Published as a conference paper at ICLR 2026
Examples.
See Figure 41, 42.
E.5 INDUCTIVE REASONING
E.5.1 BIRD MIGRATION INDUCTION
Task Description. This task requires the model to analyze temporal distribution changes of birds. By observing how bird distributions change over past years, the model must infer migration patterns and predict the likely distribution in the upcoming year.
Examples.
See Figure 43, 44.
E.5.2 PLANT DISEASE INDUCTION
Task Description. This task evaluates model's ability to learn disease patterns in plants. Given samples of leaves afflicted with particular disease, the model must learn the visual features and then identify which candidate leaves also suffer from the same discase.
Examples.
See Figure 45,46. 46.
E.5.3 SPORT FEATURE INDUCTION
Task Description. This task tests model's ability to induce patterns in sports characteristics. Given series of images depicting sports with certain patterns rules, the model must understand the characteristic relationships and choose the next sport that best matches the pattern.
Examples.
See Figure 47, 48.
E.6 SPATIAL REASONING
E. RELATIVE POSITION ESTIMATION
Task Description. This task model's spatial relationship reasoning. Given the relative positions of some objects in an indoor scene, the model must infer the relative positions of others and judge directional relationships (e.g. east, west, north, south).
Examples.
See Figure 49,500 50.
E.6.2 CAMERA ROTATION ESTIMATION
Task Description. This task requires the model to analyze viewpoint changes between consecutive images. By comparing the same scene from different angles in the image sequence, the model must accurately estimate the camera's 's rotation angles and directions at each step.
Examples.
See Figure 51, 52.
E.6.3 NAVIGATION ROUTE PLANNING
Task Description. This task tests a model's spatial reasoning and path planning capability. A robot must navigate given indoor environment from start point goal point. Only 90 or 180" turns and forward moves are allowed, and obstacles must be avoided. The model must plan the correct sequence of moves.
Examples.
See Figure 53, 54.
24

---

Published as a conference paper at ICLR 2026
E.7 TEMPORAL REASONING
E.7.1 CROWD TIMELINE RECONSTRUCTION
Task Description. This task assesses model's understanding of temporal sequences in complex scenes. Given a set of unordered images crowd activities, the model must use cues from people's positions, actions, and environmental changes infer the correct chronological order.
Examples.
See Figure 55, 56.
E.7.2 DRIVING SEQUENCE PREDICTION
Task Description. This task evaluates a model's ability to predict time-varying driving scenes. Given sequence of images from front-facing cockpit (driver's perspective) view, the model must integrate road geometry, vehicle motions, traffic participants, and environmental cues to predict the most likely next frame.
Examples.
See Figure 57, 58.
E.7.3 HUMAN ACTIVITY LOCALIZATION
Task Description. This task asks the model to locate when in a video sequence particular human activity occurs. Given a video and description of an activity, the model must precisely predict which time segment (start, middle, end, throughout) the activity takes place.
Examples.
See Figure 59, 60.
F DETAILS OF MAIN EXPERIMENT
F.1 DETAILED EXPERIMENTAL SETUP
Multimodal Language Models. Here, we list all the models used our experiment and pro- vide the corresponding version (if available): gpt-5-2025-08-07 (OpenAI, 2025b), gpt-5-mini-200
08-07 (OpenAI, 2025b), 1pt-4.1-2025-04 1-2025-04-1444 (OpenAI, 2025a), gpt-4. mimm-mm05-------1111111 (Ope- nAI, 2025a), 2p1-40-2024-11-22 (OpenAl, 2024), mmn--4--mii-2222--007 mini-2025-04- (OpenAI, 2025c), 2aude-somnee-4222255015 (Anthropic, 2025b), claude-3-7- sommet-20250219 (Anthropic, 2025a), gemini-2.5-flast (Comanici et al., 2025), gemini-2.5-pro (Co- manici et al., 2025), doubao-I-5-viise -----iisiiin-----22 (ByteDance Seed Team, 2025), Thinking-2505 2506 (Du et al., 2025), Eey5-VL-1 (Team et al., 2025), (Yue et al., 2025c), al., 2025a), MM-Fareka-QWenn-2 (Meng al., 2025a), OpenVL.Thinkerr7B-vV.... (Deng al., (Deng et al., 2025), -.ven2.5-VL-7B-Inmse
(OpenAl, 2024), 04-
Mo-oo-V--7--L---55- ---0-------- (Yue et al., 2025c). OMM-Eurka--ween (Meng
2025), OpenVt.Tiinker77B11..-//// (Bai al., 2025), Qween2.5-11-72---ssrrad (Bai et al., 2025).R1-
(Bai et al.,
2025), Swee2.5-VVL-32B--mme Onevision-7B (Yang et al., 2025c), R1-Onevision-7B-R (Yang et al., 2025c), Skywork-R/V--3 (Peng et al., 2025), VL-Rethinker--75 (Wang et al., 2025b), VL-Rethinker-32B (Wang al., 2025b), VL-Rethinker-722 (Wang et al., 2025b), Intern VL3.5-8B (Wang et al., 2025e), Intern VL3.5 V03.5V3B33--B- 3-3B--BBB (Wang et al., 2025e), InternVL3.5-388 (Wang et al., 2025e), gemma-3-4b-t (Kamath et al., 2025), gemmu-3-12b-it (Kamath et al., 2025), gemma-3-27b-it (Kamath et al., 2025), QVQ-72B-Previee (Qwen Team, 2024).
Parameters.
For parameters during the model's inference. We set the temperature to 0.5, top p
0.5, and seed to 17.
Prompts.
The prompt used in the main experiments are illustrated in Figure 18.
F.2 FULL EXPERIMENTAL RESULTS
We demonstrate full evaluation results on 37 MILMss in Table 6.
25

---

Published as a conference paper at ICLR 2026
F.3 EXPERIMENTAL RESULTS ON TINY SET
We present the model performance comparison on the mini test set in Table 7.
G DETAILS OF THINKING PATTERN ANALYSIS
For the base setting, we use MiMo-VL-7B-STD RL-Onevision, Qwen-2.5-VL-32B and Qwen-2.5- VL-7B (with CoT prompting). For the RI. setup, the model corresponding to the RL training version for CoT: MiMo-VI--BBRI. RI--Onevision-RL. MM-Fureka-32B, and VL--Rethinker-7B. These models are trained on various datasets to illustrate the generalizability of our conclusions.
H CASE STUDY
We further provide additional case studies as shown from Figure 19 Figure 60, showing both correct and incorrect responses by GPT-5 and Gemini-2.5-Pro.
Table 6: Full performance comparison of SOTA MLLMs on MMR-Life.
Model
Abd
Ana
Cau
Ded
Ind
Spa
Tem
Avg
gpt-5
53.75
78.87
41.06
80.14
78.32
17.25
41.70
58.69
gemini-2.5-pro
54.40
73.77
36.99
79.43
73.66
25.10
35.79
56.86
gemini-2.5-flash
46.25
75.18
34.22
71.63
73.66
23.92
30.81
53.10
04-mini
41.37
73.59
27.38
71.28
68.07
19.22
32.66
50.49
gpt-5-mini
44.95
69.72
32.32
75.18
68.76
12.16
29.52
49.77
gpt-4.1
44.30
71.30
22.43
67.38
70.16
13.73
27.31
48.15
claude-somet---ttiiiingg
36.96
60.92
44.11
67.02
56.64
15.69
28.23
45.32
claude-3.7-sonnet
33.55
66.55
35.36
59.93
59.67
20.78
26.01
45.09
gpt-4o
46.91
65.67
25.86
51.42
66.20
11.37
26.01
44.75
gpt-4.1-mini
32.90
61.62
30.80
52.13
65.27
16.47
30.63
44.10
claude-sonnet-4
35.50
57.22
38.02
64.89
55.71
14.51
25.83
42.82
Qwen2.5-VL-72B
35,50
55.46
35.36
52.13
55.48
12.94
23.80
40.21
Noubao-1.5-vision
37.13
53.70
31.18
59.57
54.31
12.16
23.06
39.98
VL-Rethinker-72B
36.48
50.88
33.08
56.03
57.58
15.69
21.59
39.68
Gemma3-27B
35.18
57.92
36.88
31.21
60.61
12.94
18.27
38.32
gpt-4o-mini
23,45
55.28
20.15
31.21
62.00
12.55
16.42
34.54
QVQ-72B-Previeer
31.27
41.20
38.02
47.87
31.24
14.12
16.42
31.14
VL-Rethinker-32B
22.98
42.96
28.14
49.65
30.77
14,90
19.19
30.29
MM--Eureka-Qwen--77
31.27
42.25
20.91
36.17
37.53
13.73
17.34
29.59
Gemma3-122
25.08
50.70
17.11
27.30
42.42
10.20
15.87
29.52
Ma-Euureka-Qwen--33
26.06
41.02
25.10
47.52
27.97
16.08
17.34
29.02
RI-Onevision-7B
28.34
37.85
23.57
25.89
42.66
14.51
19.37
28.80
MiMo-VL-7B-BL
38.76
25.88
28.14
60.99
24.94
14.12
19.19
28.68
Qwen2.5-VIL-2B
23.45
42.78
21.29
50.00
27.27
15.69
16.24
28.61
VL-Rethinker-7B
30.62
40.49
21.29
28.72
43.59
13.73
11.81
28.23
MiMo-VL-7B-SFT
36.81
23.24
27.00
63.48
23.31
12.16
18.08
27.36
Qwen2.5-VL-7B
26.06
35.74
20.53
20.92
38.93
9.41
12.18
24.68
RI R1-Onvvision-7B-RL
24.76
35.04
22.05
25.53
30.30
12.16
12.73
24.00
Intern VL3 5-30B-A3B 5
45.60
19.19
33.46
36.52
14.45
12.16
14.39
23.09
Intern
46.25
15.67
26.24
41.13
5.59
14.51
18.27
21.77
Gemma3-4B
14.66
29.75
20.91
26.60
23.31
12.16
17.34
21.50
Keye-VL-115--5
19.87
21.30
23.95
14.18
20.28
13.73
23.62
20.22
Intern 5-8B
35.18
11.44
18.63
34.04
11.19
14.90
16.61
18.67
Simi-VLLASBB-TThnnnn--25566
24.10
12.85
18.25
37.23
10.96
12.94
18.08
18.07
OpenWVLThiinee=77B-v11.
16.94
19.37
20.91
12.06
18.18
17.25
18.27
17.84
OpenV\LThinneer7-B-11222
16.29
19.19
21.29
13.83
18.41
17.65
17.16
17.80
22.15
10.39
16.73
23.76
11.89
9.80
11.07
14.13
26

---

Published as a conference paper at ICLR 2026
Table 7: Performance comparison of SOTA MLLMs MMR-Life mini
Model
Abd
Ana
Cau
Ded
Ind
Spa
Tem
Avg
Human
79.76
57.65
75.00
70.59
63.41
79.76
79.76
72.28
gpt-5
63.33
66.67
53.33
73.33
66.67
30.00
50.00
57.62
gemini-2.55pra
63.33
80.00
60.00
76.67
50.00
30.00
36.67
56.67
gemini-2.5-flash
53.33
63.33
50.00
53.33
50.00
50.00
53.33
53.33
o4-mini
46.67
73.33
43.33
73.33
50.00
23.33
43.33
50.48
gpt-5-mini
50.00
56.67
40.00
83.33
50.00
6.67
33.33
45.71
claude-sonnet-4
40.00
66.67
60.00
66.67
30.00
16.67
56.67
48.10
gpt-4.1
50.00
56.67
36.67
16.67
63.33
6.67
26.67
36.67
claude-3.7-sonnet
30.00
73.33
46.67
70.00
40.00
13.33
43.33
45.24
gpl-4o
43.33
50.00
40.00
16.67
46.67
20.00
26.67
34.76
gpt-4.1-mini
33.33
63.33
36.67
53.33
50.00
10.00
26.67
39.05
doubao-1.5-vision
50.00
50.00
36.67
63.33
40.00
13.33
23.33
39.52
VL-RRethinker722B
26.67
50.00
40.00
63.33
33.33
13.33
23.33
35.71
QVQ-72B-Preview
26.67
53.33
50.00
36.67
26.67
3.33
16.67
30.48
MMM-areka-QQwen-2222
20.00
46.67
43.33
50.00
33.33
16.67
36.67
35.24
MiMo-WL--BBRR.
36.67
16.67
40.00
70.00
16.67
13.33
26.67
31.43
VI.-Rethinker-7B
23.33
36.67
13.33
13.33
33.33
0.00
16.67
22.78
Keye-VL-1.5-8B
13.33
23.33
33.33
20.00
26.67
13.33
26.67
22.38
Skywork-RI-388B
30.00
6.67
33.33
16.67
30.00
16.67
13.33
20.95
Qwen255-VL-72B
36.67
40.00
40.00
20.00
36.67
10.00
30.00
30.48
Gemma3-27B
20.00
36.67
33.33
30.00
40.00
6.67
30.00
28.10
Gemma3-12B
30.00
50.00
16.67
33.33
20.00
6.67
23.33
25.71
Qwen2.5-VL VL-32B
23.33
30.00
30.00
20.00
20.00
6.67
23.33
21.90
Qwwen.5-VIL-775
13.33
0.00
23.33
20.00
0.00
6.67
10.00
14.67
InternVL3.55 5..
33.33
13.33
40.00
26.67
16.67
10.00
16.67
22.38
Intern VL.3.58B
30.00
6.67
20.00
16.67
20.00
13.33
10.00
16.67
27

---

Published as a conference paper at ICLR 2026
300
250
Count
200
120
150
100
50
Physical Pheremeentt
Natural Creatures
Sport Actavites
Everytay cejerts
Traffec Scorm
Prrduct stino
Human Arteork
Petholopy Photos
crone Surveillacee
Intertar Vieaa
Human Animaaior
Game Spmboots
Daily Dining
Higatten Map
Domentic
Image Types
Figure 9: Image type distribution in MMR-Life.
662
600
544
500
407
400
Value
300
251
200
52
43
0
2
10
Image Counts
Figure 10: Image counts distributions in MMR-Life.
28

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt
[Task]
Please act as a reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer:
Let's think step by step before answering.
Figure 11: Negative option generation prompt.
29

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt II
[Task]
Please act as a reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer: (Event 1, Event 2)
Let's think step by step before answering.
Figure 12: Negative option generation prompt.
30

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt III
[Task]
Please act as a reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer: X-X-X-X-X.., where 'x' is the number corresponding to the image.
Let's think step by step before answering.
Figure 13: Negative option generation prompt.
31

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt IV
[Task]
Please act as a reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer: Rotate clockwise/counterclookwsss about x
degrees(, then chockwiss/couuterlookviis about x
degrees)(, and finally counterclockwise about x degrees), where represents the degrees.
Let's think step by step before answering.
Figure 14: Negative option generation prompt.
32

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt V
[Task]
Please act as reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer: DIRECTION, where DIRECTION represents the eight common directions.
Let's think step by step before answering.
Figure 15: Negative option generation prompt.
33

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt VI
[Task]
Please act as a reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
where ACTION is limited to the following actions: Turn Left, Turn Right, Turn Back, Go forward until the xxx. You should add numbering (e.g., 1., 2., 3. ...) before each action in the answer. 1. ACTION 2. ACTION ..., where ACTION is limited to the following actions: Turn Left, Turn Right, Turn Back, Go forward until the xxx. You should add numbering (e.g., 1., 2., 3. ...) before each action in the answer.
Answer: 1. ACTION 2. ACTION
Let's think step by step before answering.
Figure 16: Negative option generation prompt.
34

---

Published as a conference paper at ICLR 2026
Negative Option Generation Prompt VII
[Task]
Please act as a reasoning expert to answer the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer: COLOR stick/container(, COLOR stick/container)(, COLOR stick/container), where COLOR represents different colors.
Let's think step by step before answering.
Figure 17: Negative option generation prompt.
35

---

Published as a conference paper at ICLR 2026
Question Answer With CoT Prompt
[Task]
Please act as a reasoning expert to choose the correct option of the given multimodal reasoning questions based on multiple images. The last line of your response should be of the following format:
[Output]
Answer: A/B/C/D/EE
Let's think step by step before answering.
Figure 18: Prompt used in the main experiment.
36

---

Published as a conference paper at ICLR 2026
Abductive: Human Activity Attribution
Question: Why does the man look to the left after shaving? Please choose the best explanation based the given images.
Options:
(A) To check his shave.
(B) He looks to the left after shaving to ensure thorough grooming.
(C) likely looks to the left after shaving because he preparing to leave area.
(D) He looks to the left to grab his shirt to put on ofter shaving.
(E) He likely looks to respond to something in his environment.
<input images>
Gemini-2.5-Pro: Step 1: Analyze the initial action. The first four images show person sitting in front of a mirror, engaged in what appears to be grooming activity, possibly shaving
applying something to their face. They are focused their reflection. Step 2: Identify the key change in action. In the fourth image, the person turns their head to left, away from the mirror. The question for this specific action. Step 3: Observe the subsequent actions. The fifth and sixth images show what happens immediately after the person looks to the left. They get up, and in the fifth image. they are seen grabbing light-colored shirt piece of clothing that was located to their left. The sixth image shows them walking away with this item of clothing. Step Connect the look to the subsequent action. The person's look to the left directly precedes the action of getting and grabbing the shirt from that same direction. This indicates clear couse-and-effect relationship: they looked towards the object they intended to interact with next. Step 5: Evaluate the given options based on evidence. A. check his shave: This unlikely, as turned his head away from the mirror, which is necessary for checking one's shave. B. He looks to the left after shaving to ensure thorough grooming: Similar to looking away from the mirror would not help with grooming. C. He likely looks to the left after shaving because he preparing to leave the area: This is partially true, but it's not the most specific reason. His immediate action not just leaving, but grabbing something first. D. He looks to the left to grab his shirt to put on ofter shaving: This is the most precise explanation. The images clearly show him looking left and then immediately grabbing a shirt from that location. This action is direct follow-up to his look. E. He likely looks to respond something
for the reason
environment: While possible, there no evidence in the images of an external stimulus (like sound onother person). The most direct evidence points to his own planned action. Step Conclude the best option. Option D provides the most accurate and specific explanation for why the man looked to the left. as it is directly supported by the visual sequence of events shown in the images.# # Answer:
Ground Truth: (D) looks to to grab his shirt to put on after shaving.
Figure 19: A correct example of Human Activity Attribution task.
37

---

Published as a conference paper at ICLR 2026
Abductive: Human Activity Attribution
Question: Why does the man reach for the table? Please choose the best explanation
based on the given images.
Options:
(A) To set his drink down on the table.
(B) To use it as surface to hold or manage iterss.
(C) To either obtain detergent for mopping
(D) To pick up the cup with milk.
(E) To gain balance while handling the broom.
<input images>
GPT-5: The sequence shows the man sweeping with a broom.- Then he appears holding drink small table with the drink in his hand, consistent with placing it down so he can continue sweeping.- There's no sign of detergent, balancing, or milk specifically # Answer: A"
hand while still handling the broom.- He leans toward the
Error Category: Perception Error (Motion and State Confusion)
Error Reason: Instead of perceiving the man picking up the cup (the correct action), GPT-5 incorrectly perceives him as placing the cup down.
Ground Truth: (D) To pick up the cup with milk.
Explanation: From the next three frames we can see that the man reaches into the table because he wants to pick up glass of milk not put the drink down.
Figure 20: An error example of Human Activity Attribution task.
38

---

Published as a conference paper at ICLR 2026
Abductive: Character Interaction Aftribution
Why did Tom cover its eyes and start smoking in the end? Please choose the best explonattin based on the given images.
Question:
Options:
(A) Tom covered his eyes and started smoking simply to imitttt cowboy stereotype.
not because of bull's attack
(B) covered his eyes because got
sand them during the scuffle, smoking was just on unreloted gog oddeed afterword.
(C) Because
upset embarrassed (defeated Jerry). he covered eyes shame and lit cigarette calm himself.
(D) Because Tom had way and wanted calmlyy wait for impoct//Tm covered his and started smoking due to being overwhelmed and stressed after narrowly escaping the charging bull.
(E) Tom covered his eyes and smoked because he thought the show over and wanted make dramatic, theatrical exit like an old Western movie character.
<input images>
Gemini-2.5-PP Step 1: Anolyze the sequence of events in the images. Image 1 shows o very angry bull charging. This establishes a clear and immediate danger. Image shows Tom running away in extreme fear. His expression and posture indicate being chased by bull from the first image. Image shows Tom hiding behind a wooden fence, peeking over anxxiety.
look continued fear likely present Image
temporary refuge, but the Tom's behavior. is now dressed as cowboy, smaking simultanecusly
shows cigarette feigned cool nonchalant covering eyes with his other hand. 2: Interpret Tom's final action Image The oct of smoking. especially cowboy outfit, trope for appearing cool and unfazed. The of covering one's eyes clossic gesture fear, indicating an inabilify or unwillingneas watch terrifying event unfold. Combining these two controdictory actions crectes comedic effect. Tom is trying to pretend he is colm and control, but his underlying terror from the bull chase is so overwhelming that he can't even bear to look. He has narrowly escaped the immmediate danger by getting behind the fence, but the stress and the lingering threat (the bull is likely about to smash through the fence) have pushed him to this state, =Step 3: Evaluate the given options based on the analysis. A: This option is incomplete. While he is imitating cowboy stereotype, it's direct reaction to the bull's attack, which is the primary motivator for his octions. B: There is no visual evidence to suggest dust or sand his eyes. The gesture is a clear V"I can't look'" pose, not one of irritation. conflict shown solely between Tom and the bull. Attributing his actions to Jerry evidence. D: This option accurately captures the duclity Tom's state, He has narrowly escaped but immense danger Nooeewweelmed and stressed The smoking anticipates the bull's impact covering eyes reveals true simply unable process the stressful situation. This aligns perfectly with the visual narrative. This character's actions driven by the in-story events (the bull), not an awareness of being in cartoon. Step Conclude the best explanation. Option D provides the most logical and comprehensive explanation for Tom's behavior. It correctly identiffes the couse (the charging bull). the emotional state (ovenwhelmed and stressed), and the meaning behind his contradictory actions (feigning calm while being terrified). # Answer: D
However,
The images provided do not include Jerry. The
on assumption not supported
suffering from extreme attempt at calm facade, while
the
intertiiterrotttioon that breaks the fourth wall. The
Ground Truth: (D) Because Tom had no way out and wanted to calmly walt for the cow's impact./Tom covered his eyes and started smoking due to being overwhelmed and stressed after narrowly escaping the charging bull.
Figure 21: A correct example of Character Interaction Attribution task.
39

---

Published as a conference paper at ICLR 2026
Abductive: Character Interaction Attribution
Question: Why did the appearance of the yellow cat change? Please choose the best explonation bosed on the given imoges.
Options:
(A) The yellow cat's appearance changed becouse got tangled toy plane's propeller, twisting body into distorted shape.
(B) The yellow cat's appearance changed because paint slime from toy splotrered onto making look different and exaggerated.
(C) Because
squashed/hit by falling (plone/balloon), which flattened and deformed its oppearance.
(D) The yellow cat's appearance changed because wropped inside the toy plane's parachute, puffing
cloth-like covering.
(E) It swallowed the watermelon that was used to attack them, and its body transformed into the shape of watermelon.
<input images>
Gemini-2.5-Pro. Step Analyze the image. The image shows Tom cat toy airplane, flying indoors large green watermelon attached battom the plane, positioned goggles and Analyze the second image. The second image shows three kittens and Jerry the the floor, looking expressions and postures indicate that something coming down towards them from obove. This links directly the first image. where Tom flying above.Step 3: Analyze the third image. The third image shows of the yellow/orange kittens from the second image. Its body has been drastically altered; It now extremely flat and elongated, almost like pancake. It has dazed expression. The background is the some room, confirming it's the same scene. Step 4: Synthesize the information and evoluate the options. We can infer sequence of events: Torn, in his plane (Image 1), drops the watermelon on the kittens below. The kittens see it coming and try to flee (Image 2). One the kittens is hit by the falling object, resulting in it being squashed flat (Image 3). Option A The cat is flattened, not twisted as if by a propeller. This is incorrect. Option B: The cat's shape is changed, it's not just covered in paint or slime. This is incorrect. Option C: This option states the cat was squashed/hit by folling toy (the watermelon from the plane), which flattened and deformed its appearance. This perfectly matches the logical sequence of events inferred from the images. The cause (falling object) and the effect (flattened cat) are consistent with the visual evidence. Option D: The cat is incorrect. Option The
it's bomb be dropped. Tom wearing mischievous look, suggesting he
mission.
alarm and scrambling get away. Their
flattened, not puffed up or wrapped in parachute. is flat, not round like a watermelon. clearly didn't swallow the watermelon. This is incorrect. Step Conclude best explanation. analysis. most logical occurate explanation yellow change appearance that toy plane
step-by-step
flattened the watermelon dropped from Tom's
Category: Perceptual (Color Visual Confusion)
Error Reason: The cat ends up being round like watermelon, flot. However, image information that Gemini-2.5-PP didn't analyze says the opposite.
Ground Truth: (E) It swallowed the watermelon that was used attock them. and its body transformed into the shope of watermelon.
Explanation: We can see in Figure 1 that Tom wonts to attack Jerry with watermelon, and then in Figure 3 Jenry's body becomes an oval (the shape of watermelon).
Figure 22: An error example of Character Interaction Attribution task.
40

---

Published as a conference paper at ICLR 2026
Abductive: Multi-Hop Collision Attribution
Question:
What event responsible for the silver cylinder moves to the left?
Options:
(A) The green ball collides with the cyan cylinder
(B) The gray cylinder bumps into the brown ball
(C) The brown ball was pushed to the west
(D) The green ball came from the right
(E) The blue ball collides with the silver cylinder
<input images>
GPT-5: Initially, the silver cylinder is in the center, the brown ball is on its left, and the green ball is on the far right- A cyan cylinder oppears to the right and pushes the green ball leftward.- The green ball approaches from the right and presses against the silver cylinder.- This contact causes the silver cylinder to shift left, later nearing the brown Thel.fhheeeooe, the event responsible is the green ball coming from the right and pushing the silver cylinder leftward. # Answer:
Ground Truth: (D) The green ball came from the right
Figure 23: A correct example of Multi-Hop Collision Attribution task.
41

---

Published as a conference paper at ICLR 2026
Abductive: Multi-Hop Collision Attribution
Question:
What event was the catalyst for the green cylinder moves?
Options:
(A) The red cylinder bumps into the gray ball
(B) The purple sphere slows down a bit
(C) The red cylinder collides with the purple ball
(D) The red cylinder moves away.
(E) The purple cylinder bumps into the gray ball.
<input images>
GPT-5: Initially, the objects are static along diagonal: purple sphere (matte) at left, then the red cylinder, gray ball, and finally the green cylinder. The red cylinder and gray ball are already near each other, but the green cylinder doesn't move. When the purple sphere reaches and collides with the red cylinder, pushes the red cylinder, which in turn pushes the gray ball into the green cylinder. This chain reaction causes the green cylinder to start moving, # Answer: C*
Error Category: Reasoning Error (Flawed Causal Reasoning)
Error Reason: When the purple sphere arrives and collides with the red cylinder, the green cylinder has already started moving, confusing cause and effect.
Ground Truth: (A) The cylinder bumps into gray Explanation: As be seen from the picture, after the red cylinder comes out from the upper right corner, it first collides with the gray ball, and then the gray ball collides with the green cylinder, causing the green cylinder to start moving.
Figure 24: An error example of Multi-Hop Collision Attribution task.
42

---

Published as a conference paper at ICLR 2026
Analogical: Animal Relation Inference
Question: You are presented with three animal pictures, Select the most appropriate fourth animal so that the relationship between the first pair is analogous To the relationship between the second pair.
Options:
<input images>
coptions images (A-E)
Gemini-2.-Pro. Step 1: Analyze the relationship between the first pair of images. The first image shows sharks, which are large, predatory, vertebrate animals living in marine environment. They high the food chain. The second image shows oysters, which are small, invertebrate (mollusk) onimals. They are fliter feeders, on food chain, and often consumed as food. They also live marine environment. The core relationship is between a large, predatory vertebrate and a small, invertebrate food source from the same general habitat (marine), Step Anolyze the third image. The third image shows brown bear, which large, omnivorous (often predatory). vertebrate animal living in terrestrial environment. the food chain. Step 3: Apply the relationship from Step 1 to the third image to find the fourth image. We are looking for an animal that has similar relationship to the bear as the oyster has to the shark. Fallowing the analogy. invertebrate food source from terrestrial environment. Step Evaluate the given options. Option A (Caterpillar): (insect larva). Bears are omnivores and are known to eat insects, Including caterpillors and grubs. This fits the analogy perfectly: large terrestrial vertebrate (bear) small terrestrial invertebrate food source (caterpillar). Option B (Tmpala): An impala large herbivorous vertebrate. While it could prey some predators, it does not fit the invertebrate or small size criteria established by the first pair. Option C (Badger): A badger is a medium-sized omnivorous vertebrate. It is not an invertebrate and does not fit the Vsmall food sourceN" role relative to bear. Option D (Bat): bat small flying vertebrate. It not invertebrate. Option E (Bear): This onother bear. The relationship in the first pair is between very different types animals, not the same type. Step 5: Conclude the best The relationship is: Large Vertebrate Predator Small Invertebrate Food Source, Pair 1: Shark (Large Marine Vertebrate) Oyster (Small Marine Invertebrate). Pair 2: Bear (Lorge Terrestrial Vertebrate) Caterpillar (Small Terrestrial Invertebrate). Option is only one that mointains this analogous relationship.# # Answer:
high on
need small,
caterpillar is small, terrestrial invertebrate
Ground Truth: (A)
Figure 25: A correct example of Animal Relation Inference task.
43

---

images. Published as a conference paper at ICLR 2026
Analogical: Animal Relation Inference
Question: Consider sequence of three animal images. Identify which animal should appear os the fourth image to maintain the same relational pattern observed between the first two and the last two images. Options:
<input images>
coptions images (A-E)
Gemini-2.5-Pro: Step Analyze the relationship between
first image shows a wolf. A wolf is carnivorous predator. The second image shows mice. Mice are small mammals that are often prey for various predators, including wolves (though not their primary food source, the predator- pereotor--reyyy relationship exists). Therefore,
relationship between the first and second images Predator: Prey. Step Apply this relationship to the third and fourth images. The third image shows bald eogle. A bald eagle is bird of prey. predator. Following the established pattern, the fourth image should depict common prey animal for
eagle. Step Evaluate eagle. Bald eagles are known for their diet, which primarily consists fish. They also eat other birds, small mammalls and carrion. Option A (Springbok): This is medium-sized antelope found Africa. It is not prey for bald eagle, which is native North America. Option (Pigeons): While eagles might occasionally hunt pigeons, they are not their primary food source. Option C (Badger): badger is a formidable mommal and is not prey for bald eogle. Option (Bat): Bats can be prey for some roptors. but they not typical primary food source for bald eagles. Option E (Bear fish (likely a salmon). Fish, especially salmon, are the primary and most iconic food source for bald eagles. Although the image also shows another predator (a bear). it the only option that clearly features the main prey of bald eagle. The logical connection between Conclude the best fit. The relationship is Predator (Wolf) Prey (Mice). To maintain this relationship, we need Predator (Eagle) Prey. The most fitting prey for an eogle among the options is the fish shown in image E.# Answer:
options as patential prey a bald
fish): image depicts bear that has caught a
eagle (predator) and the fish (prey). Step
Enror Category: Perceptual Error (Color and Visual Confusion) Error Reason: Gemini-25-Pro may focus only on local details (e.g. fish) while ignoring broader image relationships
Ground Truth: (B)
Explanation: The relationship between wolf and mouse, hawk and dove is essentially one predator prey.
Figure 26: An error example of Animal Relation Inference task.
44

---

Published as a conference paper at ICLR 2026
Analogical: Product Similarity Inference
Since my friend already owns these shoes but wants a comparable pair, which of the following should I give him?
Question:
Options:
<input images>
HIGHS
<options images (A-E) >
Gemini-2.5-Pro. Observe the original shoes: black, low-top skate style with white star and two white side stripes the Converse One Star/Acaderny style.- Compare options: A: White tennis shoe (Stan Smith) different silhouette and branding. B: Retro runner different style and sole. C: Adidas Samba soccer-inspired with three stripes, not a star. D: High-top Nike Dunk different cut and look. E: Black low- top with the same star-and-baa design essentially the same/contemporary comparable pair.Therefore, the most comparable choice is E.# Answer:
Ground Truth: (E)
Figure 27: A correct example of Product Similarity Inference task.
45

---

Published as a conference paper at ICLR 2026
Analogical: Product Similarity Inference
My friend already owns these pairs of shoes, but he prefers other styles. Which the following shoes would be the best gift for him? Options:
Question:
<input images>
coptions images (A-E) >
The first image shows the pair he already owns: Nike Dunk-style low-top leather sneaker. He prefers other styles, as suggested by the subsequent images
GPT-5:
(running shoe, high-top Jordan, minimalist Stan Smith). Among the options: A is
another Dunk (same style he already has) not ideal. D is Jordan 1
silhouette is very similar to a Dunk. E is a white leather Stan Smith with velcro closer in feel to his current low-top leather style and also bit off due velcro. B
is Adidas performance running shoe clearly different style he prefers. C (New Balance) isn't among the shown preferences. So the best gift that matches a different preferred style the Adidas running shoe. # Answer: B
Error Category: Understanding Error (Rule Question Misunderstandinn) Error Reason: GPT-5 limits the scope of interpretation by narrowing "other styles" to "preferences shown only in the figure," without considering that option (New Balance) is actually the correct "other styles" given in the question.
Ground Truth: (C)
Explanation: We can see that the shoes in the first and fourth pictures in the options are similar to the friend's first and third pairs shoes, while the second and fifth pairs of shoes in the options are similar to the friend's second and fourth pairs of shoes respectively, so only the third pair of shoes meets the different styles.
Figure 28: An error example of Product Similarity Inference task.
46

---

Published as a conference paper at ICLR 2026
Analogical: Artwork Style Inference
Here are paintings from one artist. Choose the option that you think was painted by the same artist. Options:
Question:
<input images>
<options images (A-E)
The given set includes ballet dancers and jockeys on horseback, both hallmark subjects of Edgar Degas. The loose. sketch-like brushwork, unusual cropping, and muted palette further point to Degas, who also painted occasional historical scenes.- Among the options, shows dancers backstage with the same impressionist handling and composition typical of Degas.- The other choices match different artists: (Modigliani), B (Magritte), C (Klee), (Delacroix).f # Answer: E*
GPT-5:
Ground Truth: (E)
Figure 29: A correct example of Artwork Style Inference task.
47

---

Published as a conference paper at ICLR 2026
Analogical: Artwork Style Inference
Question: You are given set of paintings from particular artist. Based the following choices, select the artwork that best matches this artist's style. Options:
<input imagess
<options images (A-E)
The set of reference images share characteristics of Paul Gauguin: post- Impressionist color, simplified forms with dark contours, Breton/Tahitian themes (e.g. riders and horses), and early-period interior/landscape. Among the options: A shows the elongated, stylized portraît style of Modigliani. B is a Surrealist still is geometric abstraction, unlike Gauguin. depicts a garden with impressionistic brushwork and palette, consistent with Gauguin's early landscapes. Therefore, the artwork that best matches the artist's style is E. # Answer: E
GPT-5:
life linked to Magritte. Orientalist scene in the manner of Delacroix.
D is an
Error Category: Abstraction Error (Incorrect Abstraction and Analogy)
Error Reason: GPT-5's misidentification of "E" as Gauguin likely stems from a style match based on superficial similarity: it's drawn to Impressionist style natural landscapes and localized tones and textures, while overlooking Gauguin's key stylistic elements: Synthetism//loossonniiss which emphasizes the flattening of large blocks of color, heavy outlines, and a weakening of perspective and gradients.
Ground Truth: (C)
Explanation: Option C demonstrates Gauguin's core grammar: replacing volumetric modeling with flattened large blocks of color, separating color fields with clear (often dark) outlines. simplitying/decorating forms, and weakening perspective and light and shadow gradients-thest are exactly the stylistic points he established in Synthetism/Cloosonniis..
Figure 30: An error example of Artwork Style Inference task.
48

---

Published as a conference paper at ICLR 2026
Causal: Character Interaction Prediction
The yellow cat was attacked by the watermelon. Based on the given images, what is the most likely consequence of this event?
Question:
Options:
(A) The cat is likely to be involved scuffle with other cats.
(B) The result after the cat being attacked by the watermelon would be that the cat is
very full and feeling bit sluggish or satisfied.
(C) The orange cat is now tired, upset, or in state of defeat.
(D) The result that the cat is left alone and distressed or tired due the
commotion.
(E) The appearance of the yellow cat changed.
<input images>
GPT-5: tom is flying toy plane carrying green watermelon-like object- the prompt says the yellow cat attacked by the watermelon.- after the attack, the third image shows the yellow cat looking swollen and deformed compared to its normal shape.- therefore, the most direct consequence is a noticeable change in the yellow cat's appearance due the impact.# onswer:
Ground Truth: (E) The appearance of the yellow cat changed.
Figure 31: correct example Character Interaction Prediction task.
49

---

Published as a conference paper at ICLR 2026
Causal: Character Interaction Prediction
The little mouse has higher emotional intelligence, but the big mouse knows how to survive. Based on the given images, what is the most likely consequence of this event?
Question:
Options:
(A) The big mouse (Tom) will maintain control and benefit from the situation.
(B) The cat and mouse form truce temporary friendship, with the mouse's
emotional intelligence and the cat's survival instincts creating mutual respect.
(C) The little mouse tricks the cat into wearing the bell, so the mouse stays safe while the cat is foolishly burdened.
(D) The little mouse will likely outsmart and continue to manipulate the big mouse due to its higher Emotional Intelligence.
(E) Tom had completely different attitudes towards the two mice.
<input images>
GPT-5: In the first images, Tom is hostile toward Jerry and tries to catch him. Then smaller mouse (Nibbles) offers Tom a gift. Inside is bell with ribbon. Using charm and innocence (emotional intelligence), the little mouse convinces Tom to put the on himself. With the bell on Tom's neck, movements will be noisy and predictable, keeping the mouse safe while Tom is handicapped. Therefore, the likely consequence is that the little mouse tricks the cat into wearing bell so he stays safe. # Answer:
Error Category: Hallucination Error Fabrication without Basis) Error Reason: GPT-5's answer fully constructs a causal chain of the "bell plot": the mouse coaxes + Tom wears bell + mouse safe.
Ground Truth: (E) Tom had completely different attitudes towards the two mice. Explanation: Because Jerry used the traditional confrontation method to get along with Tom, while the little mouse used gifts to show his friendship to Tom, Tom's attitude towards them was completely different.
Figure 32; An error example of Character Interaction Prediction task.
50

---

Published as a conference paper at ICLR 2026
Causal: Mulp Collision Prediction
Question:
After the blue cylinder came from the right, what will be the result?
Options:
(A) The green cube collides with the yellow cube.
(B) The orange cylinder moved west.
(C) The orange cylinder collides with the green cube.
(D) The yellow cylinder moved west.
(E) The yellow cube comes from the side.
<input images>
Gemini-2.5-Pro: Step 1: Analyze the initial scene. The first image shows five objects on grey surface: green cube, yellow cylinder, a cyan cylinder, a purple cylinder, and blue cylinder. Step 2: Observe the sequence of events. The images show progression of actions: In the second image, the blue cylinder, which was the upper right, has moved closer to the yellow cylinder. In the third image, the blue cylinder collides with the yellow cylinder. In the fourth image, the impact has pushed the yellow \'westX" in this 2D projection),
cylinder to the left (which causing it to collide with the green cube. In the fifth image, new object, yellow cube, appears from the bottom left of the frame. This appears be seporate event, not directly caused by the previous collisions. Step 3: Evaluate the question. The question asks for the Vresulty of the blue cylinder coming from the right. This implies we need to identify consequence of the blue cylinder's movement and subsequent impact. The direct consequences are movement the yellow cylinder and its collision with the green cube. Step 4: Analyze the given options based the observed events. The green cube collides with the yellow cube: This incorrect. The green cube collides with the yellow cylinder, not yellow cube. The orange cylinder moved west: This is incorrect. There is no orange cylinder in the scene; the cylinder in question is yellow. C. The orange cylinder collides with the green cube: This is incorrect for the same reason B; the cylinder is yellow, not orange. D. The yellow cylinder moved west. This is correct. After being struck by the blue cylinder, the yellow cylinder is pushed to the left, which is westward direction in this context. This is direct result the action described the side: While yellow cube does appear in the last image, it's not presented direct physical result of the blue cylinder's action. It seems to be an independent event. The question asks for the result of the collision. making the movement of the yellow cylinder a more direct and logical answer. Step 5: Conclude the best option. Based the analysis, option D the only statement that both factually correct according to the images and describes a direct consequence of the action mentioned in the question. The blue cylinder's movement causes the yellow cylinder move west.# Answer: D
be interpreted
question. The yellow cube comes from
Ground Truth: (D) The yellow cylinder moved west.
Figure 33: A correct example of Multi-Hop Collision Prediction task.
51

---

Published as a conference paper at ICLR 2026
Causal: Multi-Hop Collision Prediction
Question:
In the wake of the gray ball comes, what transpires?
Options:
(A) The cyan cylinder tagged the cyan cylinder.
(B) The green ball hit the cyan cylinder.
(C) The cyan ball hit the cyan cylinder.
(D) The green cylinder moved.
(E) The blue ball moved.
<input images>
GPT-5: The gray ball rolls forward and makes contact with the shiny green ball. This pushes the green ball toward the nearby cyan cylinder: In the last frames, the green ball reaches and hits the cyan cylinder, while the other objects (cyan ball, blue cylinders) do not move notably.# Answer: B
Error Category: Perceptual Error (Color and Visual Confusion)
Error Reason: GPT-5 mistook the cyan sphere for a green sphere.
Ground Truth: (C) The cyan ball hit the cyan cylinder.
Explanation: We can see that after the gray ball appears from below, it first hits the cyan metal ball, and then the cyan metal ball hits the cyan cylinder. There is no green ball the whole scene.
Figure 34: An error example of Multi Hop Collision Prediction task.
52

---

Published as a conference paper at ICLR 2026
Causal: Counterfactual Fluid Prediction
Question:
If the pink stick were removed, which container would pink fluid flow into?
Options:
(A) Orange container, Green container
(B) Green container, Red container
(C) Pink container
(D) Green container
(E) Green container, Pink container
<input images>
The pink fluid drops from the right spout. Some of it always falls along the right side guided by the black bar and ends up the right green container.- The rest runs left along the gray ramp. If the pink stick removed, the stream from the gray ramp drops directly onto the green ramp below.- The green ramp launches this stream leftward into the left pink container.So with the pink stick removed, pink fluid goes into both the green container and the pink container.t # Answer:
GPT-5:
Ground Truth: (E) Green container, Pink container
Figure 35: A correct example of Counterfactual Fluid Prediction task.
53

---

Published as a conference paper at ICLR 2026
Causal: Counterfactual Fluid Prediction
Question:
If the black stick were removed, which stick would green fluid poss?
Options:
(A) Pink stick, Blue stick
(B) Blue stick, Yellow stick
(C) Blue
(D) Pink stick
(E) Yellow stick
<input images>
Gemini-2.5-Pre Step Analyze the initial flow of the green fluid. The images show sequence of fluids being poured. We need to trace the path of the green fluid The first green source the leffmost one In the second image, the green fluid released and lands the pink stick It flows down the pink stick and, seen in the third and fourth images. it then falls onto the blue stick From the blue stick, the fluld is directed towards and hits the black stick After the black stick, hits the red stick and then folls into the containers below So, the original path for the green fluid is: Pink stick -> Blue stick -> Black stick -> Red stick. Step 2: Hypathesize the new flow with the black stick removed. The question asks what would happen the black stick were removed The initial port of the path would remain the same. The green fluid would shill fall from its source. land on the pink stick, and then flow onto the blue stick The change occurs after the fluid leaves the blue stick. In the original setup, hits the black stick the black stick is removed, the fluid continue to fall from the end of blue stick Looking at the positions of other sticks, the yellow stick located directly below the black stick in path the fluid falling from intercept
stick Therefore, without the blue directly onto yellow stick Step 3: Determine complete patn and evalucte the options. The path for green fluid would Pink stick
would
-> Yellow stick question asks Vwhich stick would green fluid pass?X". We need choose the best option from the list The fluid posses the pink, blue, and yellow sticks this scenario Let's look of options: Pink stick, Blue stick: This true, but incomplete. Blue stick, Yellow stick: This is also true, but incomplete.
Blue stick: True, but incomplete. D. Pink stick: True, but incomplete. Yellow stick: This is also true Since multiple options are technically correct subsets of the full path, must infer the question's intent. The question about the consequence of removing the block stick. The interactions with the pink and blue sticks crucial change is that the fluid, instead of hitting the block stick, now hits the yellow stick. Therefore. the most significant and defining part of the new path is the interaction with the yellow stick. This makes \"Yellow stick\" the most logicol and informative onswer Answer:
unchanged. The
Error Category: Hallucination Error Misinformation and Confusion)
Error Reason: if the block stick removed, the green liquid will flow through the yellow stick but the model's analysis wrong, causing Gemini-25-Pro to justify itself and exclude the correct onswer, even though the model's previous analysis correct.
Ground Truth: (A) Pink stick, Blue stick
Explanation: We can see that after the gray ball appears from below, it first cyan metal boll, then the cyan metal ball hits the cyan cylinder. There no green ball the whole scene.
Figure 36: An error example of Counterfactual Fluid Prediction task.
54

---

Published as a conference paper at ICLR 2026
Deductive: Material Composition Deduction
Question: You are tasked with building toy robots. Each robot requires 2 spheres, 2 cubes, and 1 cylinder. You can source spheres from both Image and Image 2, cubes only from Image 2, and cylinders only from Image What the maximum number of complete robots you build?
Options:
(A) 1
(B) 2
(C) 3
(D) 4
(E) None of the above
<input images>
Gemini-2.5-Pro: Here is step-by-step breakdown of the reasoning:1. Identify the requirements for one robot: spheres cubes cylinder2. Count the available parts according the sourcing rules: Spheres (from Image and Image 2): Image has spheres (one large gold, small silver). Image 2 has 2 spheres (one large gold, one small green). Total available spheres 2 2 4. Cubes (only from Image 2): Image has 6 cubes (one large cyan. two small red, one small brown, one small blue, one small grey). Total available cubes Cylinders (only from Image 1): Image has 3 cylinders (one green, one purple, one brown). Total available cylinders 3.3. Calculate the maximum number of robots that can be built based on each part type (this identifies the limiting factor): Based on Spheres: You have spheres, and each robot needs 2. So, / 2 robots. Based Cubes: You have 6 cubes, and each robot needs 2. So, 6 / 2 3 robots. Based on Cylinders: You have cylinders, and each robot needs 1. So, 3 / 3 robots.4. Determine the overall maximum: The number of complete robots you can build is limited by part that runs first. In this case, you only have enough spheres to build robots. Even though you have enough cubes and cylinders for 3 robots, you will run out of spheres after completing the second one. Therefoof, the maximum number of complete robots you can build 2.# Answer:
Ground Truth: (B) 2
Figure 37: A correct example of Material Composition Deduction task.
55

---

Published as a conference paper at ICLR 2026
Deductive: Material Composition Deduction
Question: Each explorer-bot bot requires sphere, cube, and cylinders. Spheres be taken only from Figure 1, cubes only from Figure 2. and cylinders from either figure, After building as many complete explorer-bots as possible, how many cylinders left unused?
Options:
(A) 1
(B)
(C) 3
(D) 4
(E) None of the above
<input images>
GPT-5: from figure 1: spheres (teal and brown). cylinders 5. from figure 2: cubes 4 (yellow, two blue, green). cylinders 3. total cylinders available 5 3 8
each explorer-bot requires 1 sphere, cube, and 2 cylinders. max bots possible = min(2 spheres, cubes, 8/2 cylinders) min(2. 4, 4) bots. cylinders used 2 # answer:
bots x 4. cylinders left unused
Error Category: Perceptual Error (Color and Visual Confusion)
Error Reason: GPT-5 correctly identified that there were 5 cylinders in Figure 1, but it misidentified the number of cylinders in Figure 2. There were actually cylinders, but the model only identified 3, resulting an incorrect answer.
Ground Truth: (E) None of the above
Explanation: We can that there are 5 and cylinders Figure 1 and Figure 2 respectively. Since at most 2 explorer-bots can be made, and each explorer-bot requires two 2 cylinders, there are 5 cylinders left at the end.
Figure 38: An error example of Material Composition Deduction task.
56

---

Published as a conference paper at ICLR 2026
Deductive: Card Winner Deduction
Question: We have players in a poker game. Each player has 2 hole cards, and there are 5 community cards on the table. Determine the winner based on the best 5-card hand formed using any combination of hole cards and community cards. Hand rankings (from highest to lowest): 1. Royal Flush: Five consecutive cards of the same suit, from 10 to Ace (10-J-Q-K-A). 2. Straight Flush: Five consecutive cards of the same suit. 3. Four of Kind: Four cards of the same rank 4. Full House: Three cards one rank and two cards of another rank. Flush: Five cards the same suit that are not in sequential order. 6. Straight: Five consecutive cards of different suits. 7. Three of a Kind: Three cards of the same rank. 8. Two Pair: Two cards of one rank and two cards of another rank. 9. One Pair: Two cards of the same rank High Card: A hand that doesn't fit into any of the above categories. The input images are in the following order: the community cards, Player I's hole cards, Player 2's hole cords, Player 3's hole cards, Player 4's hole cards, and Player 5's hole cords. Please evaluate each player's hand strength determine winner.
Options:
(A) Player 1
(B) Player
(C) Player
(D) Player
(E) Player
<input images>
GPT-5: community cards: q 64 4+
3+
3 player (9+ 54 ): best hand is
one pair 3s with q. 9, kickers.- player (7 7 ): two pair - 7s and 3s with q kicker. player 3 (a* 4w two pair and 3s with kicker. player 4 (3+ 3* four of kind four 3s with kicker. player 5 (a+ j+ one pair 3s with
kickersthe strongest hand is player 4's four of a kind (3s). answer: D
Ground Truth: (D) Player
Figure 39: A correct example of Card Winner Deduction task.
57

---

Published as a conference paper at ICLR 2026
Deductive: Card Winner Deduction
Question: We have players poker game. Each player has 2 hole cards, and there are community cards on the table. Determine the winner based on the best 5-cord hand formed using any combination of hole cards and community cards Hand rankings (from highest to lowest):1 Royal Flush: Five consecutive cards of the same suit, from 10 to Ace (10-J-Q-K-A) --.--. Straight Flush. Five consecutive cards of the same suit.3. Four of Kind: Four cards of the same ronk4. Full House: Three cords of rank and two cards of another rank5. Flush: Five cards of the
suit that not in sequential ordenó, Straight. Five consecutive cards different suits.7. Three of Kind: Three cards of the same rank.8. Two Pair: Two cards of one rank and two Pair: Two cards the
rank10. High Card: hand that doesn't into any of the above categories. The Input images are in the following order: the community cords, Player I's hole cards, Player 2's hole cards, Player 3's hole cards, Player 4's hole cards, and Player 5's hole cards Please evaluate each player's hand strengtth and determine the winner.
cards of another rank9.
Options:
(A) Player
(B) Player 2
(C) Player 3
(D) Player
(E) Player 5
<input images>
GPT-5: "Let's read the cards: Community cords: A Q+ 4t
2
94
Player 1:
7+
9* Makes pair of 9s using 94
99 with kickers A+
Q+
7
Player 2:
3 No pair, best high card (A+
4v Player
10*
No pair: best is high card A (A+ Q+
104
94 4 ). Player 4: 6
84 - No
pair: best high card (A+ Q+ 94 84 6 ). Player 5: 9
K Makes pair
9s using
9w with higher kickers
Q+ Compare the best hands:
Player and Player both have pair of 9s, but Player 5's kickers (A, K, Q) beat
Player I's (A, 7). Therefore, Player wins. Answer: E"
Error Category: Perceptual Error (Color Visual Confusion)
Error Reason: GPT-5 misidentified the suit and value the playing cards.
Ground Truth: (D) Player
Explanation: Although Player 1 also has a pair of 6s, Player has a stronger kicker (8) and therefore beats the other player (high card).
Figure 40: An error example of Card Winner Deduction task.
58

---

Published as a conference paper at ICLR 2026
Deductive: Recipe Step Deduction
Question: I want to make Bundt Bread With Nuts And Chocolate, please generate the correct order of the images to make complete recipe.
Options:
(A) 9-3 9-3-5-2-7-1-4-8
(B) 8-1-3-2-5-7-4-6
(C) 8-3-2-5-9-7-1-4
(D) 8-3-5-7-2-1-4-6
(E) 8-3-2-5-7-1-4-6
<input images>
Gemini-2.5-PP Here is step-by-step analysis of the images to determine the correct order for making Bundt bread with nuts and chocolate Image 8: Gather all ingredients. This is the clossic \'mise en placeX step, where all necessory ingredients and tools are laid out before starting. We can see bogs of flour, sugar. chocolate, nufs, eggs, and the Bundt pon itself. This is logically the first step. Therefore, the correct sequence must start with 8. This eliminates option A.2. Image Prepare the baking pan. The Bundf pon is shown greased, floured, and with chopped walnuts sprinkled on the bottom. Preparing the pan an early step, often done right after gathering the ingredients, ensure it's ready when batter/dough mixed... Image 3: Measure dry ingredients. yellow bowl contains a pile of white powder, which likely flour and sugar, key dry ingredients for the bread. This is another preparatory step. The order preparing pan and measuring ingredients (3) can sometimes be swapped, but both are done before mixing begins. The sequence 8-1-3 represents a very organized start4. Image 2: Mix the initial dough. stand mixer with dough hook is kneading a plain, smooth dough white bowl. This the base dough before any solid additions like nuts or chocolate are mixed in. This step must follow the measurement ingredients
Image 5: Add chocolate to dough. some shown, now dork chunks. clearly identifiable as chacolate, are being incorporated into dough. This step logically follows the creation of dough (2).6. Image The final mixed dougi. The dough (mixer amm removed). con
mixer
the mixing lighter brown nuts were complete. This step
chocolate
pieces nuts distributed throughout dough. indicates odded after chocolate, and mixing process follow step Image Tronsfer dough the pan. The fully mixed dough from step now being placed
prepared Bundh pan from step The dough not yet fill pan completely.8. Image 6: Let the dough rise. The final image shows Bundf pon, but has significantly,
filling pon almost the top. This the final procfing stage before the bread goes into the oven. This step must follow the transfer of dough into the pan (4) Based this logical progression. 6.Let's check this sequence against the given options: 9-3-5-2-7-1-4-8 Incorrect stort and contoins invalid image number (9). 8-1-3-2-5-7-4-6 This matches derived logical sequence perfectly. 8-3-2-5-9-7-1-4 Contoins invalid image number (9). D. 8-3-5-7-2-1-44- Incorrect mixing order (5 before 2). E. 8-3-2-5-7-1-4-66 This
the dough
the correct sequence is:
sequence suggests preparing the (1) after the daugh completely mixed (7). While possible, It's less conventionall and less efficient workflow than preporing the part of the initial setup (as in option B). The V'mise en place\ approach shown is more standard. Therefore, option B provides the most logical and stondard order for the baking process shown the images # Answer:
Ground Truth: (B) 8-1-3-2-5-7-4-6
Figure 41: correct example of Recipe Step Deduction task.
59

---

Published as a conference paper at ICLR 2026
Deductive: Recipe Step Deduction
Question: wontt to make Boeuf Bourguignon Modern Fusion, please generate correct order of the images to make complete recipe.
Options:
(A) 5-4-2-6-3-7-1-8
(8) 5-4-2-6-3-1-7--8
(C) 5-4-2-1-6-337-88
(D) 4-5-2-6-1-3-7-8
(E) 4-5-6-2-1-3-7-8
<input images>
Gemini-2.5-PP Step 1: Ingredient Preparation. The first step in any recipe to gather prepare the ingredients. Image shows shank, also known onions, potato, tomatoes, mushrooms) prepped preparing both meat and vegetables. So the sequence begins with and 5, Step 2: Searing the Meat. To build flavor, the meat is seared hot Image shows piece of meat being browned in the pan. This meat and leaves browned bits (fond) the pan. This must happen after the meat prepped (4) and before other ingredients are cooked in the So, the sequence 4-5-2. Step Soutéing the Vegetables. After the meat is seared and removed from the the vegetobles are typically soutéed in the remaining fat and fond. Image shows the chopped corrots, onions, potatoes, and mushrcoms being cooked in the same pan. This step follows the searing of the meat. The sequence is now 4-5-2-6. Step 4: Deglazing and Adding Liquid. After the vegetables are softened, liquid (often wine in Boeuf Baurguignen) is added to deglaze the pon, lifting the flavorful fond from the bottom. The seared meat is then returned to the pon. Image shows the meat back in the pon, simmening in a red liquid, which represents this stage. The sequence becomes 4-5-2-6-1 Step 5: Combining and Simmering. All the ingredients are combined to cook together. Image shows the meat, all the sautéed vegetables. plus the chopped tomatoes and a bay leaf, simmering in the braising liquid. This is the beginning of the long, slow cooking process. The sequence is now 4-5-2-6-1-3, Step 6: Reducing and Finishing. The stew is simmered for an extended period. allowing the flavors to meld, the meat to become tender, and the sauce to reduce and thicken. Image shows the dish ofter this process. The liquid level is lower than in Image 3, colors are deeper, and the ingredients look fully cooked tender. This the final stage cooking before serving. The sequence 4-5-2-6-1-3-7, Step Plating. The final step to serve the finished dish. Image shows Boeuf Bourguignon plated and ready to eat. This is the lost image in the sequence. Conclusion: The complete and logical order the images to make recipe 4- 5-2-6-1-3-7-6 This matches option D. Answer:
meat (a cut beef buco) bowl. Image shows the vegetables (carrots.
cutting boord. The logical stort is
creates flavorful crust the
Error Category: Other Error
Error Reason: Gemini-2.5-0 sometimes overlook crucial information necessary to solve a problem when analyzing recipe, fails to grosp the cues variations in the appearance of the same side dish, such as a steak changing from rare to well-done
Ground Truth: (C) 5-4-2-1-6-3-7-8
Explanation: The reason for choosing C is that it follows the complete cooking process ingredient preparation - frying meat - adding vegetables - stewing and reducing the souce serving.
Figure 42: An error example of Recipe Step Deduction task.
60

---

Published as a conference paper at ICLR 2026
Inductive: Bird Migration Induction
Looking the observed yearly changes bird distribution shown images, predict the most likely distribution pattern for next year. Options:
Question:
<input images>
coptions imoges (A-E)
Gemini-2.5-Pro. Step 1: Analyze the input images to identify the pattern of bird distribution over time. The four provided images show the distribution of bird species over what is implied to be four consecutive years. all four images, the distribution is widespread across North America. The birds are consistently found high concentrations consistent, though less dense, presence along
eastern half of the United States and southern
Canada. There is mouniaainous regions trend from the observed patterns. By companing that the overall geographic ronge the birds remorkably stable. While there might minor year-to-year fluctuation occupied the density within those squares. the fundomental distribution pattern does not change. There is clear evidence of large-scale migration shift. expansion,
Coast
western US. Step Defermine images,
which specific grid squares
contraction of the overall range. The pattern is one of consistency. Step 3: Formulate prediction based the identified frend. Since the observed trend over the post four years is stability. the most logical prediction for the next year is that the distribution will continue to be stable and will look very similar to the patterns seen in the input images. Step 4: Evaluate the given options against the prediction. Option A: Shows a significant contraction of the range. The birds are almost entirely absent from the westemn half the continent. This contradicts the stable, widespread distribution seen in the input images. Option Shows distribution pattern that highly consistent with the four input images. diaplays a widespread presence across the eastern southern Canada, and parts of the western This matches the prediction noticeable northword westward
stability. Option concentration of birds, with denser population represents significant change, than
Canadian prairies and
sporser the southeastern contraction of range, even stability. Option Shows Option with birds mainly the northeasterm southeastern US. This drastic change. Option Shows the most extreme contraction, with the population limited to small area in the northeastern US and Great Lakes region. This completely inconsistent with the histarical data. Step 5: Select the best option. Option B the only one that represents continuation of the stable distribution pattern observed in the input images. Therefore, it is the most likely prediction for the next year Answer: B*
Ground Truth: (B)
Figure 43: A correct example of Bird Migration Induction task.
61

---

Published as a conference paper at ICLR 2026
Inductive: Bird Migration Induction
Question: Looking at the observed yearly changes in bird distribution shown the images, predict the most likely distribution pattern for next year. Options:
<input images>
coptions images (A-E) >
Gemini-2.5-Pro: First, compare the four yearly maps from earliest to latest (top-left to bottom-right). There is a clear monotonic increase the number purple grid cells (occupied/ /bbeevvddd areas), with the strongest growth in the south (Argentina-Chile) and along coastlines. The fourth year shows the highest density, especially expanding into Patagonia and around the coasts, while the Amazon interior remains relatively less occupied. Extrapolating this trend next year suggests further densification, with even more cells filled across the southern cone and coastal areas, while still keeping the Amazon comparatively lighter. Among the options, D best matches this continued expansion and densification pattern.** # Answer: D
Error Category: Knowledge Error (Over-reliance Internal Knowledge) Error Reason: Gemini-2.5-Pre does not perform analysis based on image information, but instead reasons based on internal geographical knowledge. And without any quantitative verification comparison with the candidate images, was finally selected based on the inertial cognition of "continuous expansion', while ignoring the actual inflection point changes the image.
Ground Truth: (E)
Explanation: Because the four-year input images show that although the distribution of birds has gradually expanded, it has reached saturation by the fourth year, with only slight increase in marginal areas such as the southern and eastern coasts; E just continues this "slight expansion" trend, rather than over- extrapolating it into full-scale outbreak like
Figure 44: An error example of Bird Migration Induction task.
62

---

Published as a conference paper at ICLR 2026
Inductive: Plant Disease Induction
The leaves in the images are affected by a disease already recognize. Which the option leaves share the same infection? Options:
Question:
<input images>
coptions images (A-E)
Gemini-2.5-Pro: Step 1: Andlyze the initial set of images. The three images at the top show leaves, which oppear symptoms. The common symptoms across these three leaves are a mottled or mosaic pattern of yellow and green patches. The yellowing (chlorosis) is not uniform but oppears as irregular blotches spots across the leaf surface. image, some of these yellow areas are starting to turn brown (necrosis). This pattern is characteristic of viral disease, most likely Tomato Mosaic Virus (ToMV) similar virus. Step Evaluate each option against the identified symptoms. Option This leaf is uniformly green and appears healthy. It does not show any mottling. yellowing, spots. It also has different shape from the tomato leaves the initial images. This does not match. Option B: This leaf shows large, whitish-gray patches, which are typical of fungal disease like powdery mildew. While there is some yellowing and necrosis, the initial images. This does not match. Option C: This leaf is dark green but has severely puckered and burnpy texture. It does not exhibit the characteristic yellow and green mottling, This symptom different from the one not match. Option D: This leaf is clearly tomato leaf. It displays distinct yellow and green mottled pattern, very similor to the leaves in the initial images. There are irregular yellow patches and some developing brown/necrotic spots. is strong match for the symptoms of the same disease. Option E: This leaf is mostly green with some slight discoloration and curling. The symptoms not pronounced clear as in the initial images or in Option It does not clearly show the same mosaic pattern. Step 3: Conclude the best match. Comparing all the options, the leaf in Option D exhibits the most similar and characteristic symptoms the leaves in the initial set of images. The yellow and green mosaic pattern is the key feature, and is clearly present in both the initial images and Option D.# Answer: D
from a tomato plant, exhibiting specific disease
the third
primary symptom is different from mosaic pattern in
initial images. This does
Ground Truth: (D)
Figure 45: A correct example of Plant Disease Induction task.
63

---

Published as a conference paper at ICLR 2026
Inductive: Plant Disease Induction
Question:
The images include leaves that show the disease I've noticed earlier. From the options, which leaf is infected by the same disease?
Options:
<input images>
coptions images (A-E)
Gemini-2.5-Prev Step 1: Analyze the symptoms the initial set of four images. The leaves in the first four images share very distinct primary symptom: they severely wrinkled, crinkled, and have a bumpy or blistered texture. This is often referred to \'savoying\.. The leaves are also distorted in shope and appear curled or misshapen. Some a masaic pattern of light and dark green (e.g. image 2 and the blurry image 3), but the most consistent and dramatic feature the severe textural abnormality. Step Evaluate each option against the identified symptoms. Option A: hos textured surface and serrated edges, but it locks the severe, deep wrinkling and distortion seen in the initial images. Its overall shape relatively normal. Option This leaf's primary symptoms are yellowing (chlorosis) and small, dark, necrotic spots. The leaf surface is mostly flat. This represents spotting disease, which is different from the disease the initial images. Option C: This leaf shows large patches of what appears to be fungal growth or blight (dark, fuzzy areas). This is completely different symptom. Option D: This leaf somewhat misshapen has slightly uneven, leathery texture, but does not deep. blister-like crinkling the initial set. Option E: This leaf perfectly matches the symptoms from the initial set. It is dark green, severely wrinkled, crinkled, and blistered. identical
leaves show discoloration,
yellowing or
the characteristic
distorted. The texture bumpy shown in the first four images. Step 3: Conclude based on the comparison. The disease shown in the initial images is characterized by severe leaf wrinkling and distortion. Option E the only image that displays this same set of characteristic symptoms. # Answer: E*
disease pattern
Error Category: Abstraction Error (Lack of Abstraction)
Error Reason: Gemini-2.5-PP P failed to correctly generalize key features or patterns in an image and thus failed to identify key characteristics of disease.
Ground Truth: (B)
Explanation: We can see that all four leaves in the input have yellowing symptoms, but only has obvious yellowing.
Figure 46: An error example of Plant Disease Induction task.
64

---

Published as a conference paper at ICLR 2026
Inductive: Sport Feature Induction
Given the sports displayed, determine which sport from the provided options should appear in the next position. Options:
Question:
NNTCKY
<input images>
<options images (A-E) >
Gemini-2.5-Pro: Step Analyze the initial set of images to identify a common theme. The first image shows person on snowmobile, which motorized vehicle used for racing snow or ice. This is form of motorsport. The second image shows Formula 1 race car on track This is high-speed, open-wheel motorsport. The third image shows a stock car, typical of NASCAR racing, on racetrack. This is another form of motorsport. The unifying theme across these three images is ""motorsports". which are competitive sporting events involving motorized vehicles. Step 2: Evaluate each option to see which one fits the identified theme. "OOption A:** This image shows several people racing on motorcycles. Motorcycle racing is well-known category of motorsport. This option fits the theme. "Option B:** This image shows amputee football (soccer). This is team sport and a parasport, but it does not involve motorized vehicles. It does not fit the theme "*Option C:*" This image shows fencing. This is a combat sport and does not involve motorized vehicles. does not fit theme. **Option D This image shows field hockey. This is team sport played with sticks and ball, not motorized vehicles. It does not fit the theme. "Option E:** This image shows wheelchair basketball. This is a team sport and a parasport, but does involve motorized vehicles. It does not fit the theme. Step 3: Conclude which option is the correct choice. Based on the analysis, only Option A continues the theme of motorsports established by the first three images.# Answer. A
Ground Truth: (A)
Figure 47: A correct example of Sport Feature Induction task.
65

---

Published as a conference paper at ICLR 2026
Inductive: Sport Feature Induction
Question: Considering the sports shown, which of the following should be chosen as the next figure? Options:
TYPES OFBOXIG BETS
<input images>
<options images (A-E)
GPT-5: Look at the four given images: boxing, judo, arm wrestingg and sumo. These are all combat/strength- the options, only steer wrestling (rodeo) is another direct physical combat/strengthh contest. The others (tug of war, golf, gymnastics, car racing) not match this pattern. # Answer: E'
sports, typically one-on-one physical contests. Among
Error Category: Abstraction Error (Incorrect Abstraction and Analogy)
Error Reason: The four inputs (boxing, judo, arm wrestling, sumo) are all human-vs- human strength contests based on direct contact, leverage, and control. It generalized this to "any intense struggle" and chose steer wrestling because it looks like grappling. But steer wrestling changes key element: it is human animal, usually with clear animal/rodeo cues, which are absent in the images. So the choice comes from surface resemblance ("looks like wrestling") rather than the shared structure ("human human").
Ground Truth: (A)
Explanation: Tug of war(A) matches the key pattern in the inputs: humans competing directly through strength and leverage. Even though it's usually team-based, it remains purely human-vs-human physical contest, unlike sports that rely vehicles, judged routines, or precision skills.
Figure 48: An error example of Sport Feature Induction task.
66

---

Published as a conference paper at ICLR 2026
Spatial: Relative Position Estimation
The washing machine is south of the microwave. In which direction the carpet relative to the electric kettle?
Question:
Options:
(A) North
(B) Southeast
(C) East
(D) Southwest
(E) South
<input images>
GPT-5: From the statement, the washing machine is south of the microwave. In the images, the washing machine is below the microwave, so "down" corresponds to south and "right" to east.- The electric kettle is on the left side of the counter (northwest area of the scene).- The carpet is on the floor near the door, to the right of the washing machine and lower in the scene.- Relative to the kettle, the carpet is both to the right (east) and lower (south).# Answer: B
Ground Truth: (B) Southeast
Figure 49: A correct example of Relative Position Estimation task.
67

---

Published as a conference paper at ICLR 2026
Spatial: Relative Position Estimation
of Hello Kitty plush toy. Where is Minion plush toy positioned nelative the pony plush toy?
Question:
The chair
Options:
(A) Southeast
(B) North
(C) Northwest
(D) Northeast
(E) West
<input images>
\The chair Step 1: Establish the cardinal directions. The problem states that east of the Hello Kitty plush toy. Image shows black chair with a wooden frame on the left side the scene. Image 4 shows the Hello Kitty plush toy the bed, inside purple tent, which is to the right of the chair's initial position. The camera pans from left right, moving from the chair towards the Hello Kitty toy. This means chair is physically the chair is to the EAST of the Hello Kitty toy. we can establish our directional mapping: East Left (in the direction of the chair from Hello Kitty) West Right, North Towards the back wall, South Towards the foreground/camerc Step Locate the Minion plush toy and the pony plush toy. The Minion plush toy visible in Image 3. It a yellow, one-eyed character sitting on the red bed, near the white headboard (which is against the back wall). The pony plush toy (a large white and unicom) is visible Image Determine the relative position the Minion the pony. We need find the position of the Minion relative to the pony. The Minion is near the back wall, making it NORTH of the pony, which is further out the floor. The Minion on the bed, which
Gemini-2.5-Pro:
left
Hello Kitty toy. Given that
the floor, to the right the bed. Step
the floor. Since \'left' EAST, the Minion is to the EAST the pony. Combining these two directions, the Minion is to the NORTH and to the EAST of the pony. Step 4: Canclude the final direction. position that is both North and East is Northeast. Therefore, the Minion plush toy positioned the Northeast
left where the pony
pony plush toy Answer: D
Error Category: Knowledge Error ( Directional and Positional Confusion) Error Reason: Geminn-2255Prr lacks common sense of orientation and reverses left to west and right east.
Ground Truth: (A) Southeast
Explanation: Since the chair east of Hello Kitty, the Minion plush toy next the chair, and pony plush toy is next Hello Kitty, the Minion plush toy are southeast of pony plush toy.
Figure 50: An error example of Relative Position Estimation task.
68

---

Published as a conference paper at ICLR 2026
Spatial: Camera Rotation Estimation
From the continuous sequence of images, what were the degrees of rotation the camera mode?
Question:
Options:
(A) Ratation clockwise about 90 degrees.
(B) Rotate clockwise about 180 degrees.
(C) Rotate clockwise about 90 degrees.
(D) Rotate counterclockwise about 145 degrees.
(E) Rotate clockwise 120 degrees.
<input images>
Observe the toy bunny on the scanner bed. In the first image, the bunny faces the carnera (front with the "3" visible). In the second image, the camera view shows the back of the bunny. Seeing the front then the back implies the camera moved to the opposite side of the object, a rotation of about 180 degrees around (direction doesn't matter for 180%; clockwise equivalent). Answer: B*
GPT-5:
Ground Truth: (B) Rotate clockwise about 180 degrees.
Figure 51: A correct example of Camera Rotation Estimation task.
69

---

Published as a conference paper at ICLR 2026
Spatial: Camera Rotation Estimation
Given the continuous images, what were the successive rotation angles of the camera?
Question:
Options:
(A) Rotate clockwise about 30 degrees, then clockwise about 45 degrees
(B) Rotate clockwise about 30 degrees, then clockwise about 30 degrees
(C) Rotate clockwise about 30 degrees, then counterclockwise about 150 degrees
(D) Rotate counterclockwise about 30 degrees, then counterclockwise about 60
degrees
(E) Rotate clockwise about 45 degrees, then clockwise about degrees
<input images>
We compare the shark toy's orientation relative to the frame and the car's edges.- From the first to the second image, the scene rotated slightly clockwise: the shark's head points more downward-right and window edge tilts bit, indicating small clockwise roll (~30 ).- From the second to the third image, the rotation continues in the same (clockwise) direction and by a larger amount: the shark is now viewed closer to frontal-right angle, roughly an additional ~45 Answer:
GPT-5:
Error Category: Knowledge Error (Rotation and Viewpoint Confusion)
GPT-5 misinterpreted counterclockwise rotation of approximately 150* as a continued clockwise rotation of small angle, ignoring the direction reversal evidence provided by the absolute reference.
Error Reason:
Ground Truth: (C) Rotate clockwise about 30 degrees, then counterclockwise about 150 degrees
First, from the right side of the shark to the shark's tail, you need to rotate clockwise about 30 degrees, and then to the shark's head, you need to rotate counterclockwise 150 degrees.
Explanation:
Figure 52: An error example of Camera Rotation Estimation task.
70

---

Published as a conference paper at ICLR 2026
Spatial: Navigation Route Planning
Question: You are robot beginning the refrigerator and facing the refrigerator. You want to navigate to the recycling bin. You can only make 90-degree 180- degree turns and go forward, and you have to go around obstacles. What actions should you perform?
Options:
(A) 1. Turn right 2. Go forward until the round table 3. Turn left 4. Go forward until the recycling bin
(B) 1. Turn left 2. Go forward until the round table with red chairs 3. Turn right 4. Go forward until the recycling bin
(C) 1. Turn back 2. Go forward until the round table 3. Turn left 4. Go forward until the recycling bin
(D) Turn left 2. Go forward until the table 3. Turn left 4. Go forward until the recycling bin
(E) 1. Turn left 2. Go forward until the suitcase 3. Turn left 4. Go forward until the
printer 5. Turn right 6. Go forward until the recycling bin
<input images>
GPT-5: Start at the refrigerator facing it; the recycling bin is across the room near the printers and the round table in the back corner.- First, turn left to align with the counter area and move forward along it until you reach the green suitcase the floor.- Turn left again to head across the room toward the printers against the opposite wall.- Go forward until you reach the printer, then turn right to face the corner with the blue bin.- Move forward to the recycling bin.# Answer:
Ground Truth: (E) Turn left 2. Go forward until the suitcase 3. Turn left Go forward until the printer 5. Turn right 6. Go forward until the recycling bin
Figure 53: A correct example of Navigation Route Planning task.
71

---

Published as a conference paper at ICLR 2026
Spatial: Navigation Route Planning
Question: You are a robot beginning at the door, facing the window. You want to novigate to the refrigerator. You can only make 90-degree or 180-degree turns and go forward, and you have to go around obstacles. What actions should you perform?
Options:
(A) 1. Turn right Go forward until the refrigerator
(B) 1. Turn left Go forward until the stove Turn right Go forward until the refrigerator
(C) 1. Go forward until reaching the gas cooker. 2. Turn left. 3. Go forward until reaching the fridge.
(D) 1. Turn right 2. Go Forward until the counter Turn right 4. Go Forward until the
refrigerator
(E) 1. Go forward until the end of the counter. 2. Turn right. 3. Go forward until the refrigerator
<input images>
From the doorway you're facing the window with the counter and stove along your left side.- Move straight ahead down the aisle beside the counter.- When you reach the end of the counter, the refrigerator is positioned to the right- Turn right and go straight to the fridge.# Answer: E*
GPT-5:
Error Category: Knowledge Error (Directional and Positional Confusion)
Error Reason: GPT-5 had poor understanding of the kitchen's spatial perception in the image. It made the wrong choice because it ignored obstacles like the stove and counter, mistakenly assumiin could go straight and then turn right the refrigerator, leading to oversimplification of spatial reasoning.
Ground Truth: (B) 1. Turn left 2. Go forward until the stove 3. Turn right Go forward until the refrigerator
Explanation: Since you are at the door and facing window at beginning, you need to turn left first instead of going directly to the kitchen counter.
Figure 54: An error example of Navigation Route Planning task.
72

---

Published as a conference paper at ICLR 2026
Temporal: Crowd Timeline Reconstruction
Question: what order these images appear chronologically?
Options:
(A) 3-1-4-2
(B) 1-3-2-4
(C) 4-1-2-3
(D) 3-1-2-4
(E) 4-3-1-2
<input images>
Gemini-2.5-Pro: Step 1: Identify Key Moving Subjects. To determine the chronological order, we need to track the movement of people across the four images. Several individuals and groups can be used as reference points: 1. A group of four people (a plaid shirt pushing a stroller, and woman in a brown coat) walking away from the camera the right side of the mall. 2. man in yellow t-shirt walking towards the camera from the camera on the left side. A woman in a dark outfit who walks from right to left in front of the people sitting around the palm tree. Step 2: Track the Group Four with the Stroller. This group walking away from the camera. Their position further down the hall indicates later time. In Image 3, this group is the closest to the camera. The stroller is just beginning to pass the circular seating In Image: the group Image 4. the group has moved even further down the hall. In Image 2, the group is the furthest away from the camera among the four images. This tracking gives Step Track Man
dark jacket, woman in light top.
in
the right side. A woman
red coat walking away
moved slightly further away from the camera. In
Yellow T-shirt. This man walking towards the camera. His position closer to the bottom of the frame indicates later time. In Image 3, he is the furthest away, near the top right of the frame. Image 1, has walked closer to the camera. Image closer than in Image In Image 2, he the closest to the camera, near the bottom right corner. This tracking also gives the order: -> -> 4 2. Step 4: Track Woman
order: 3 ->
even
Red Coat. This woman walking away from on the left. Her position further down the hall indicates a later time. In Image she is closest to the carnera, near the bottorn left. In Image 1, she has moved Image 4,
camera
further up walkway. furthest away of the four images. This tracking confirms the order: >> 1 -> 4 -> Step 5: Track the Woran in the Dark Outfit. This woman walks from right to left in front the seating area. Image she is not yet visible in that area. Image 1, she appears on the right side of the seating area. In Imoge 4, she is in the middle, directly in front of the seated people. In Image 2, she has passed the seated people is on left side. This tracking also supports the order.
further still. Image 2,
>> 2. Step 6: Conclude the Order. All key subjects' movements consistently point to the same chronological sequence. The images. in order, show the progression of these people through
moll. The correct sequence then 1, then and finally 2. Step 7: Match with the Options. The determined order is 3-1-4-2, which corresponds to option A# Answer: A"
Ground Truth: (A) 3-1-4-2
Figure 55: A correct example of Crowd Timeline Reconstruction task.
73

---

Published as a conference paper at ICLR 2026
Temporal: Crowd Timeline Reconstruction
Question: What the correct chronological sequence these images?
Options: (A) 5-3-2-4-1
(B) 5-3-1-4-2
2-4-1-3-5
1-2-3-4-5
4-1-2-3-5
<input images>
Gemini-25-Pret Here is step-by-step analysis to determine the chronological order of the images: Identify Key Moving Elements: The images depict a busy shopping mall with many people moving around. To establish a timeline, we can track the positions of specific individuals or groups across the different fromes. Let's focus on a few distinct groups Group (Right Side): A group of four people, including a womon holding the hand of small child in a red coat. They are walking away from the comera, down the right-hand comidor Group (Center): A couple, a man in a dork top and woman in light top, walking towards the camera in the upper-middle part of the scene Person (Left Side). woman a light-colored jacket and dark pants, walking from the upper left towards the bottom left the frome. 2. Track the Movement Group (Child in Red): Image This group is the furthest away from camera, just entering the circular area around the central planter Image 3: They have moved forword. woman holding the child's hand now roughly level with right side the plonter Image They have moved further shill They are now past the planter longer visible, indicating they have walked Images and must Track Movement of Person (Woman in Light Jacket): Image This yet visible Image She first oppears Image She moved further along path, towards the bottom left Image She even further along, closer the battom left corner Image She is the very edge of the frome, about exit. This tracking confirms the sequence: (5) Track the Movement of Group B (Central Couple): Image This couple the furthest away. the upper middle of the scene Image They have closer Image They have
battom right of frame. Images and 2:
This group tracking establishes the sequence: ->
the frame. This
after these.
upper leftt beginning
walked closer to the comero Image They walked past the central planter and now moving towards the klosk the left Image 2: They
even closer to the klosk. This tracking also confirms the sequence: 5 -> 3 -> 1 >> 4
5. Conclusion: By consistently trocking the movement of multiple people and groups. we can confidently determine the chronological order. The people in the scene are furthest away in Image 5 and progressively move closer or across the frame through Images 3, 1, 4, and finally 2. Therefore. the correct chronological sequence 5-3-1-4-2. # Answer: B
Error Category: Reasoning Error Temporal and Sequential Confusion)
Error Reason: Gemini-2.5- 2.5-Pro can use certain reference point to sort the events, but there are too many reference points. In this example, the model sorting result is completely opposite to the correct onswer.
Ground Truth: 2-4-1-3-5
girl in surrounding behavior reference point. holding parent's hand and be seen moving further and further away from camera.
Explonation: images, we
Figure 56: An error example of Crowd Timeline Reconstruction task.
74

---

Published as a conference paper at ICLR 2026
Temporal: Driving Sequence Prediction
Question:
Which of the images in the options is most likely to be the next one to occur?
Options:
<input images>
coptions images (A-E)
GPT-5: Observe the sequence: the dashcam car inches forward while the scooter on the left moves out from under the tent toward the street (rightwards each frame). The pedestrians the lower-right remains visible. The next frame should therefore show the left scooter further into the road and the right-side pedestrians bit farther right:Option E best matches this continued motion: the left scooter advances further right and the pedestrians are slightly more to the side, consistent with the progression.#* # Answer:
the right shift slightly toward the sidewalk and the small child
Ground Truth: (E)
Figure 57: A correct example of Driving Sequence Prediction task.
75

---

Published as a conference paper at ICLR 2026
Temporal: Driving Sequence Prediction
Question:
Which image in the options is most likely to appear in the next moment?
Options:
<input images>
<options images (A-E)
We have five consecutive frames showing o girl on the left starting to move away to the left, while the boy and the robot remain nearly stationary.- From frame to frame, the girl's position shifts progressively leftward; by the last provided frame she has begun running and nearing the left edge.- The most likely next moment should show her even farther left (still partly in frame). with the rest of the scene largely unchanged. Among the options:- A and B show her still standing near the robot (earlier state). C shows her already almost out of frame (likely later moment).- E shows minor movement only.- shows continuing to run left, slightly farther than the last frame, matching the immediate next step.# # Answer:
GPT-5:
Error Category: Reasoning Error ( Temppral and Sequential Confusion)
Error Reason: GPT-5 did not correctly analyze the little girl's movement trend. She was almost out of the frame.
Ground Truth: (C)
Explanation: We can see from the last image input that the little girl is almost running out of the picture, only option C fits the sequence of events.
Figure 58: An error example of Driving Sequence Prediction task.
76

---

Published as a conference paper at ICLR 2026
Temporal: Human Activity Localization
Question:
Can you identify when the action 'a person walks through the doorway' happens in the sequence?
Options:
(A) Throughout the entire sequence.
(B) At the end of the sequence.
(C) In the middle of the sequence.
(D) the beginning the sequence.
(E) Not shown in the sequence.
<input images>
GPT-5: First frame: door is open with no person present.- Second frame: person appears, stepping out the doorway stands near the doorway and drinks, not walking further.Therede the walking through the doorway occurs right at the beginning of the sequence. # Answer: D."
a mug.- Subsequent frames: the person
Ground Truth: (D) At the beginning of the sequence.
Figure 59: correct example of Human Activity Localization task.
77

---

Published as a conference paper at ICLR 2026
Temporal: Human Activity Localization
Question:
During which part of the sequence does the action 'a person puts coffee cup on shelf occur?
Options:
(A) Throughout the entire sequence.
(B) At the end of the sequence.
(C) In the middle of the sequence.
(D) At the beginning of the sequence.
(E) Not shown in the sequence.
<input images>
GPT-5: Look at the sequence: the person opens kitchen cabinet and reaches toward items inside.- The shelves contain jars and possibly packets; the person then holds packet or tray-like item.- At no point is a coffee cup being placed onto a shelf. the actions are opening, reaching, and holding items, likely taking them out # Answer: E.
Error Category: Perceptual Error (Color and Visual Confusion) Error Reason: GPT-5 fails to capture key visual information, such as the coffee cup in the first few frames.
Ground Truth: (D) At the beginning of the sequence. Explanation: We can the first 5 frames, where the little boy puts the coffee cup on shelf.
Figure 60: An error example of Human Activity Localization task.
78


Reviews:
[{"review_id": "1n1prpIkn4", "reviewer": "Reviewer_5uk2", "summary": "The paper introduces MMR-Life, a benchmark for evaluating multi-image and multimodal reasoning in realistic scenarios. It builds 2,676 multiple-choice questions from 19,367 real-world images, covering seven reasoning types and 21 tasks. The authors construct the dataset through automatic question generation plus human filtering, then evaluate 37 vision–language models to compare with human performance. Results show a clear gap between models and humans, especially on spatial and temporal reasoning, so the benchmark is intended to serve as a diagnostic testbed for future multimodal reasoning model", "questions": "Please refer to weakness section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The benchmark uses 2,676 MCQ questions built from 19,367 real-world images, with many questions requiring information integration across several images, so models cannot rely on single-image shortcuts.\n2. Reasoning tasks are organized into seven well-defined types (abductive, analogical, causal, deductive, inductive, spatial, temporal) and 21 tasks, providing a structured view of where current models fail.\n3. The analysis across reasoning types shows differentiated difficulty profiles, which makes the benchmark useful as a diagnostic tool rather than only a leaderboard.", "weaknesses": "1. The paper analyzes the effect of longer thinking traces only at the overall level, lack discussion by break down reasoning type, it is unclear whether “longer thinking ≠ better” holds uniformly across tasks.\n2. Human-level performance is based on a subset (validation questions), whereas model results are reported on the full benchmark, which makes the human–model comparison not fully aligned and may overstate the human gap.\n3. Error analysis stays relatively shallow and mostly descriptive, could reveal deeper insights, guiding future model improvements.\n\n### **Minor comments**\n1. Figure 6 font size is a bit small\n2. In Figure 48 the chosen option seems defensible from the question phrasing.\n3. Table 2 lists MMMU as a comparison point, but MMMU is not a single-image benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hTalUJdvVm", "reviewer": "Reviewer_Bdkt", "summary": "The paper introduces MMR-Life, a large-scale benchmark designed to evaluate multi-image reasoning abilities of multimodal large language models (MLLMs) in real-world scenarios. It includes 2,676 multiple-choice questions derived from 19,367 real-life images, spanning seven reasoning types. Unlike previous benchmarks that rely on domain-specific knowledge or synthetic geometric objects, MMR-Life emphasizes integrating information across multiple real-world images. Experiments with a broad range of open and proprietary models reveal certain insights and gaps relative to human performance, underscoring the need for further progress in multi-image reasoning.", "questions": "1. Table 2: Why is the source of MMR-Life marked solely as A (Annotated)? Given the benchmark heavily draws from existing datasets, shouldn’t it also be labeled as E (Existing datasets)?\n2. Can the authors clarify why human performance falls below GPT-5 in the Analogical, Deductive, and Inductive categories? Does this reflect genuine model advantages, human weaknesses, or potential biases in the evaluation setup?\n3. How can the insights derived from the benchmark inform future model architecture design or prompting strategies to enhance multi-image reasoning?\n4. Were the authors able to verify whether the benchmark’s source datasets were not included in the training data of the evaluated open models?\n5. As noted, Keye-VL-1.5-8B and InternVL3.5-8B perform worse than random guessing. Can the authors explain this behavior? Does it stem from output formatting issues, context-length limitations, or other factors?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper tackles an important and timely problem - evaluating whether MLLMs can effectively handle multi-image, vision-based reasoning tasks, a topic of clear relevance to the ICLR community.\n2. The MMR-Life benchmark covers a broad range of reasoning types across diverse real-world image contexts, enabling fine-grained analysis of model strengths and weaknesses and offering a well-structured evaluation framework.\n3. The evaluation is thorough and well-rounded, including a diverse set of both open and proprietary models. Importantly, the authors also examine the impact of different training and inference methods, such as Chain-of-Thought, Self-Consistency, Best-of-N, and GRPO, adding an additional analytical dimension.\n4. The paper provides insightful analyses of performance patterns across reasoning categories and includes an error taxonomy that helps characterize model limitations and failure modes.\n5. The paper is clearly written, logically structured, and comprehensive in its literature review, making it accessible and informative to readers.", "weaknesses": "1. The benchmark largely repurposes existing datasets (Appendix C.1), which limits its novelty. The primary contribution lies in the reorganization and categorization of these datasets by reasoning type rather than in introducing new data or task formulations.\n2. The evaluation omits several important baselines, including representative MLLMs (e.g., LLaVA, InstructBLIP) and non-MLLM approaches such as supervised CNNs or few-shot/meta-learning methods (Prototypical Networks, SNAIL, MetaBaseline).\n3. The human performance evaluation (Section 3.1) appears shallow and exhibits high variance. Each of the 600 validation questions seems to have been solved by only one annotator, making it unclear how results would generalize across individuals. Moreover, the comparison to models could be more balanced if model performance were reported on the same subset of questions used for human evaluation.\n4. The claims of a substantial human–model performance gap (e.g., Figure 1) seem overstated. As shown in Table 3, GPT-5 actually outperforms humans on three reasoning categories (Analogical, Deductive, Inductive), with humans leading in the remaining four.\n5. The Spatial reasoning category, identified as the most challenging, has already been explored in prior work - particularly in the SPACE benchmark [Ramakrishnan, 2025]. This overlap somewhat limits the unique contribution and potential utility of MMR-Life for advancing spatial reasoning evaluation.\n\nRamakrishnan, Santhosh Kumar, et al. \"Does spatial cognition emerge in frontier models?\" International conference on learning representations. 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4UPbH7LohP", "reviewer": "Reviewer_YUHP", "summary": "This paper introduces MMR-Life, a multimodal multi-image reasoning benchmark focusing on real-life scenarios. The benchmark contains  2,676 multiple-choice questions covering seven reasoning categories. Evaluation of state-of-the-art models shows deficiency in performance, and current reasoning enhancement strategies do not work too well on the benchmark.", "questions": "The paper says that \"We generate question-answer pairs using either automatic synthesis or manual annotation, depending on the task type\". What is the percentage of machine- and human-generated problems? I see there is a task description section in the appendix, but I think it is also important to clearly outline sources of questions (i.e., human/LLM-generated) for each task.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-written and easy to follow, with an abundance of examples.\n- The benchmark contains interesting challenges and performs a pretty thorough evaluation of sota models, showing their deficiency.\n- The exploration of reasoning enhancement strategies provides some interesting insights.", "weaknesses": "- I appreciate the rich appendix section and accompanying analysis. But these examples are most likely cherry-picked, which is totally fine for presentation. It would be good to include a link to an anonymously hosted repository containing the full dataset so reviewers can get a more unbiased, holistic view of data quality.\n- Related to above: Some of the problems in the appendix are of low quality. \n    - For example, on page 65, the ground truth explanation states that the answer should be \"water sport/racing\". Not all images in the question are water sports (e.g., there is horse racing), so it is perfectly fine to select E as the correct answer as well. \n    - On page 45, the question states \"My friend already owns this pair of shoes\", but four pairs of shoes are shown. This is rather confusing. Also, I don't think this question makes any sense. The ground truth, C, looks similar to the first pair in terms of color and the second in terms of style.\n    - On page 77, I do not even understand the question. It seems to contain two completely separate questions?\n    - On page 37, even after zooming in, I do not see where there is milk on the table.\n\nThe point I want to make is that the selected questions probably represent some of the higher-quality ones, yet some of them are still not totally reasonable for models to answer, so I think giving reviewers access to the full dataset is essential for quality control.\n- The paper mentions a 600-question validation set. But I think it only appears once in the paper in the context of human performance. Model performance is reported on the whole set, but human performance is only on the validation set. They may not be directly comparable. What is the process for constructing this validation set? What is model performance on this subset?\n- The paper claims the questions cover real-life scenes, but many are in fact pretty artificial or unlikely to come across in real-life (e.g., pages 40, 41 -- synthetic images; pg 60, 61 -- scientific and domain-grounded; pg 48, 49 -- cartoon scenes). These still carry relevance and assess multimodal reasoning, but weakens the paper's focus on real-life scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)