ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
Authors: Yongyuan Liang
Wei Chow
Feng Li
Ziqiao Ma
Xiyao Wang
Jiageng Mao
Jiuhai Chen
Jiatao Gu
Yue Wang
Furong Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Unified Multimodal Model
Generation Benchmark
Cross-modal Reasoning
Primary area: datasets and benchmarks
TL;DR: A benchmark for evaluating cross-modal reasoning in unified multimodal models

Abstract: Unified multimodal models (UMMs) have emerged as a powerful paradigm for seamlessly unifying text and image understanding and generation. However, prevailing evaluations treat these abilities in isolation, such that tasks with multimodal inputs and outputs are scored primarily through unimodal reasoning, i.e., textual benchmarks emphasize language-based reasoning, while visual benchmarks emphasize reasoning outcomes manifested in the pixels. We introduce ROVER to address this pressing need to test reciprocal cross-modal reasoning, the use of one modality to guide, verify, or refine outputs in the other, an ability central to the vision of unified multimodal intelligence.
ROVER is a human-annotated benchmark that explicitly targets reciprocal cross-modal reasoning, which contains 1,312 tasks grounded in 1,876 images, spanning two complementary settings. Verbally-augmented reasoning for visual generation evaluates whether models can use verbal prompts and reasoning chains to guide faithful image synthesis. Visually-augmented reasoning for verbal generation evaluates whether models can generate intermediate visualizations that strengthen their own reasoning processes for question answering.
Experiments on 17 unified models reveal two key findings:  (i) Cross-modal reasoning determines visual generation quality, with interleaved models significantly outperforming non-interleaved ones; notably, combining strong unimodal models fails to achieve comparable reasoning.  (ii) Models show dissociation between physical and symbolic reasoning: they succeed at interpreting perceptual concepts literally but fail to construct visual abstractions for symbolic tasks, where faulty reasoning harms performance. These results highlight reciprocal cross-modal reasoning as a critical frontier for enabling true omnimodal generation. Homepage: https://roverbench.github.io

Published as a conference paper at ICLR 2026
ROVER: BENCHMARKING RECIPROCAL CROSS- MODAL REASONING FOR OMNIMODAL GENERATION
Yongyuan Liang*A, Wei Chow" Feng Lio, Ziqiao Ma Xiyao Wang
Jiageng Mao*, Jiuhai C Gu Yue Wang** Furong Huang
University of Maryland, College Park University of Pennsylvania
UUiiersiity of Michigan * UUnieersty of Southern California The Hong Kong University of Science and Technology
ABSTRACT
Unified multimodal models (UMMs) have emerged as a powerful paradigm for seamlessly unifying text and image understanding and generation. However, pre- vailing evaluations treat these abilities in isolation, such that tasks with multi- modal inputs and outputs are scored primarily through unimodal reasoning, i.e., textual benchmarks emphasize language-based reasoning, while visual bench- marks emphasize reasoning outcomes manifested in the pixels. We introduce ROVER to address this pressing need to test reciprocal cross-modal reason- ing, the use of one modality to guide, verify, or refine outputs in the other, an ability central to the vision of unified multimodal intelligence. ROVER is a human- annotated benchmark that explicitly targets reciprocal cross-modal reason- ing, which contains 1,312 tasks grounded in 1,876 images, spanning two comple- mentary settings,
reasoning for visual generation evalu- ates whether models can verbal prompts and reasoning chains guide faithful image synthesis. Visually-augmented reasoning for verbal generation evalu- whether models can generate intermediate visualizations that strengthen their own reasoning processes for question answering. Experiments on 17 unified mod- els reveal key findings: (i) Cross-modal reasoning determines visual genera- tion quality, with interleaved models significantly outperforming non-intenn non-interieaved ones; notably, combining strong unimodal models fails to achieve comparable rea- soning. (ii) Models show dissociation between physical and symbolic reasoning: they succeed at interpreting perceptual concepts literally but fail to construct vi- sual abstractions for symbolic tasks, where faulty reasoning harms performance. These results highlight reciprocal cross-modal reasoning as a critical frontier for enabling true omnimodal generation. Homepage: roverbench, roverbench.gittub.i..
Temperaa
ROVER-Image Gen
ROVER-Text Gen
Pizza
Spatial
Input Image
Instruction
Question
Input
Causat
Causli
Spabil
ofter 3 seconds
Find BEFC?
Logio & Mata
Genmatr
Art
Science
Tampora
Unified Multimodal Models
Imagia.
Image Gen
ROVER
Logic
Commetry
Verbal Reasoning
Visual Retsoring
World
ROVER Twxt Gen
ink diffuce....
Quant
Tumporii n
Common Sensa
Pazzle
Plysics
Visual Parcaption
Mfllivien
Spatial
Quioa
28v3
2/3
Output Image
Ground Truth
Ground Truth Output
Figure 1: The ROVER benchmark. ROVER evaluates UMMs through reciprocal cross-modal rea- soning: ROVER-IG (left) requires generating images with language-augmented reasoning, while ROVER-TG (right) requires generating text answers with visually-augment reasoning.
Authors contributed equally this work. Advisors contributed equally this work.

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
The development of unified multimodal models (also referred omnimodal models) has drawn significant attention to their potential for unified understanding and generation across text and im- ages (Comanici et al., 2025; Hurst et al., 2024; Tong et al., 2024; Deng et al., 2025; Xu et al., 2025b). However, prevailing evaluations treat these abilities in isolation, such that tasks with multimodal in- puts and outputs are scored primarily through unimodal reasoning: textual benchmarks emphasize language-based reasoning. while visual benchmarks emphasize reasoning outcomes manifested in the pixels. On the language side, evaluation focuses on generating text in response to an image and an accompanying question, thereby testing perceptual understanding (Chen et al., 2024; Liu et al., 2024: Yu et al., 2024) and reasoning (Lu et al., 2023; Yue et al., 2024; Wang et al., 2024; Hao et al., 2025; Gao et al., 2025). On the vision side, evaluation centers generating images conditioned on either instructions or text-image pairs, thereby testing direct image generation (Ghosh et al., 2023; Ma et al., 2024; Niu et al., 2025) image editing (Kawar al., 2023; Zhang al., 2023; Ma et al., 2024; Sheynin et al., 2024; Yu et al., 2025a; Liu et al., 2025b; Wu et al., 2025e).
Unlike earlier multimodal systems that specialize in either visual perception generation, UMMs are designed to reason seamlessly across modalities and produce outputs that span both. This cre- ates a pressing need for benchmarks that evaluate their ability use one modality to guide, verify, or refine outputs in the other. We refer to this capability as reciprocal cross-modal reasoning (Fig- ure 1). To benchmark such capability in current unified multimodal models, we present ROVER, a human- annotaten and rigorously verified benchmark with over 1,312 tasks grounded in 1,876 im- ages. ROVER targets two complementary settings: (i) verbally-augmmented reasoning for visual generation, including 4 conceptual domains (natural science, culture & art, common sense, and logic & math) with high complexity are instantiated across reasoning subtasks: temporal, spa- tial, causal, synthetic, quantitative, abstract, and mathematical. Each instance provides textual prompt with an initial image and a chain constraints that a correct output image must satisfy. (ii) disually-augmenn
reasoning for verbal generation, including 6 subtask variants spanning 3 problem domains: visual perception, world modeling for robot manipulation & physical dynam- ies prediction, and logical reasoning for geometry & puzzle solving. Instances interleave turns of text and images, requiring the model to emit visual intermediates that make downstream reasoning auditable.
A key challenge is that evaluating reciprocal cross-modal reasoning requires assessing both the ra- tionales and the output. Text-only metrics overlook visual fidelity, while image-only metrics cannot verify whether the image reflects valid reasoning. Human evaluation provides accurate judgments but prohibitively expensive at scale. To address this, we adopt a multi-dimensional protocol that combines an automated VLM judge (based on GPT-4.1 (Hurst et al., 2024)) with expert valida- tion on stratified samples. The judge is supplied with rubric cards and reference assets and scores along three reasoning-spedt processes, (ii) the alignment of generated outputs with target descriptions or ground-truth answers, and (iii) the consistency between intermediate reasoning steps and the final images answers. For visual generation tasks, the framework additionally incorporates established image consistency and quality metries (Hu al., 2023; Wu et al., 2023; Kirstain et al., 2023; Xu et al., 2023; Brooks al., 2023). The judge is calibrated with expert explanations, and its agreement with expert evaluations reported, following recent LLM-as-judge methodologies (Kim et al., 2023; Hu al., 2023).
dimensions: (i) the logical coherence of domain-specific reasoning
Through extensive evaluation of 17 unified multimodal models, our experiments reveal significant gaps in cross-modal reasoning capabilities. Specifically, unified models exhibit: (1) mross-modal reasoning capabilities largely affect reasoning-dependentt visual generation quality, with interleaved image-text generation models significantly outperforming con-interleaved counterparts; critically, strong unimodal models cannot replicate this cross- reasoning on ROVER, even when com- bined; (2) Striking dissociation between physical and symbolic visual reasoning. While unified models excel at generating visual reasoning steps for perceptual and physical world concepts through literal interpretation of visual elements, they fundamentally struggle to reason about visual abstrac- tions as symbolic representations (Hsu et al., 2025), causing poor visual reasoning to degrade down- stream performance rather than improving it. These findings reveal fundamental capability limita- tions in current unified models, underscoring the critical role of reciprocal cross-modal reasoning for holistic omnimodll generation, where independent optimization of constituent modalities proves insufficient.
2

---

Published as a conference paper at ICLR 2026
Table I: Summary of Multimodal Reasoning Benchmarks. We compare existing works from aspects including: 'interleave: supports multi-image or multi-turn inputs; pprocess evaluation: eval- uates intermediate reasoning steps; "vision necessity: requires reasoning grounded in visual under- standing: " 'multidimensional evaluation: scores models along multiple dimensions; "hybrid evalu- ation: uses GPT-based judgments instead of purely visual metrics; manual annotations: whether manual annotations and filtering are applied; scale: dataset scale; stypes: data categories.
Benchmark
Venue
Inter.
Process
Eval
Vision Necess.
Multi.
Hybrid
Eval
Manual
Anno.
#Types
ReasonPix2Pix (Jin et al., 2024)
arXiv'24
X
ReasonEdit (Huang et al., 2024)
CVPR'24
X
X
X
X
EditWorld (Yang et al., 2024)
MM'25
X
X
Reason5OK (He al., 2025)
arXiv'25
MeIl-BBench (Wu al., 2025e)
NeurIPS'25
X
X
RISEBench (Zhao et al., 2025)
NeurIPS'25
X
X
V
V
J
V
WorldGenBench (Zhang ct al., 2025)
arXiv'25
X
Unified- (Yan al., 2025)
MetaQuery (Pan al., 2025)
arXiv'25
V
ROVER
Ours
V
23
Our main contributions are summarized as follows:
We introduce ROVER, the first benchmark that explicitly targets reciprocal cross-modal reason- ing for visual generation and interleaved multimodal reasoning.
We provide principled task taxonomy and a verification-ready instance design with process targets and visual artifacts, together with multi dimensional protocol that scores coherence, alignment, and step to output consistency.
We evaluate 17 unified models, uncovering significant limitations in cross-modal reasoning and providing the community with insights on unified model development toward truly omnimodal generation.
2 RELATED WORK
Unified Multimodal Models (UMMs). UMMs represent a paradigm of architectures designed seamlessly integrate multimodal comprehension and generation capabilities within singular, cohesive framework. To achieve this unified objective, seminal works (Karypis et al., 1999; Wu al., 2025b; Chen al., 2025b) leverage image tokenization strategies, employing autoregressive next-token prediction paradigms to generate visual tokens. Building upon these foundations, Show- o2 (Xie et al., 2025) introduces discrete diffusion scheduling mechanisms to enhance the token prediction process and improve generation fidelity. Subsequent developments, driven by the pursuit of enhanced image synthesis quality, incorporate diffusion- based or flow-matching heads (Lipman et al., 2022) integrated with shared transformer architectures (Deng al., 2025; Ma et al., 2025; Zhou et al., 2024). Alternative approaches within the UMM paradigm maintain powerful pretrained backbone in frozen state for reasoning tasks, while routing their intermediate feature representa- tions through learnable query mechanisms to external image generation modules (Pan et al., 2025; Wu al., 2025d). However, the comprehensive evaluation of synergistic relationships between multimodal understanding, reasoning, and generation UMMs remains largely unexplored, with existing benchmarks inadequately assessing whether these capabilities exhibit mutual enhancement or coordination deficiencies.
Reasoning-Guiier Image Generation. With the emergence of UMMs, multimodal reasoning has garnered increasing attention from the research community. However, the majority of existing work remains focused on instruction comprehension, namely leveraging input images perform instruction translation and subsequently generate corresponding visual outputs (Jin et al., 2024; Huang et al., 2024; Yang et al., 2024; He et al., 2025; Wu al., 2025e; Yu al., 2025b). Unified- Bench (Yan et al., 2025) employs iterative image-text generation to measure the degree of unifica- tion between comprehension and generation models. RiSEBench (Zhao et al., 2025) extends beyond prior work by introducing LMM-as-a-judge to evaluate visual rationality in addition to assessing im- age consistency, yet remains limited to computing similarity scores against numan-provide ground truth. However, these benchmarks lack comprehensive evaluation beyond image consistency, par- ticularly overlooking the intermediate reasoning processes, such as whether the rationale sound and aligns with the generation outcome. In contrast, ROVER represents the first benchmark in-
3

---

Published as a conference paper at ICLR 2026
Temporal
Spatial
Causal
Imaginative
Quantitative
Natural Science
happane
bapasta con bett
pow--VV asgoezne
speciran Jotng pant
pepuictionddz
Temporal
Spatial
Causal
Imaginative
Quantitative
Culture &
printee
Ainorgg -epprrtggggtttttttttt Ima-gpprneette workpioee
price
Temporal
Spatial
Causal
Imaginative
Quantitative
Sense
Capturs molkan Stamse
wader perapeciivs ride
lutting wandow
family manoo
orange citon,
Puzzle
Geumetry
Logic &
Complee
Gentrate siew
Salee www.eiii.diioeooo
Tabel the witth the
200 2422 in this squzne
Figure 2: Overview of ROVER-IG, the benchmark for evaluating how unified multimodal models generate images under intensive verbal reasoning. The benchmark spans domains (natural science, culture and art, common sense, and logic), instantiated across 7 reasoning subtasks.
Agent-Informed State Change
Physics- Informed State Change Vissal Reasoning
Iniital
Farret
Vieaa Ressinng
(388) sotl
(2233, 7085
Inpot
World
the action
pat
(389), 5080
Inter?
middle
Geometry
Puzzle
with
Elighlight Irtangte
Starting
gowen and
Exllow
ABC shord 1K1- and
Logic
Since equal angien, equal BC B0 =1
point ii List the petno you pans in
The path goos
&
length
ot:
Inpul
Ressemning
Iapai Jigsaw
Reasaning
Multi-view Reasoning
Image vilao strt image2
The full gancreeed image.
Combised ronm
Visual Percept.
teccaam moving att werght
camer anvve right.
Piace A complete the Hip
Ingut
Vissa. Reasoning
Ingrut
Vosuel Ressoning
Figure 3: Overview of ROVER-TG, the benchmark for evaluating visually- augmented reasoning verbal generation. The benchmark spans 3 scenarios and subtasks: physical world modeling, logical assistance, and visual perception enhancement.
vestigate the interplay between reasoning and generation. A detailed comparison can be found in Table A more detailed discussion about interleaved reasoning can be found Appendix C.
ROVER BENCHMARK
3.1 VERBALLY-AUGMENTEE REASONING FOR VISUAL GENERATION
We introduce ROVER-IG, benchmark designed evaluate how UMMs generate images when jointly guided not only by visual understanding but also by intensive language reasoning. Taxon- omy. It spans domains and reasoning subtasks, each demanding complex text-driven reasoning chains to direct image generation and test models' ability to integrate text text-augmented reasoning with visual synthesis. Figure provides a visual overview of our benchmark taxonomy and representative examples.
Domains. We categorize tasks across 4 distinct areas: Nature Science encompasses scientific phenomena, experimental processes, and fundamental laws nature; Culture & Art includes artistic creation, cultural artifacts, humanities, and aesthetic principles; Common Sense covers
4

---

Published as a conference paper at ICLR 2026
everyday scenarios requiring intuitive understanding and practical reasoning: Logic & Math fo- cuses on abstract visual puzzles, mathematical relationships, and general pattern discovery.
Reasoning subtasks. We define 5 core reasoning capabilities: Temporal involves sequence pre- diction, progression analysis, and time-based changes; Spatial requires understanding geometric relationships, perspective changes, and spatial visualization; Causal connects cause-effect rela- tionships and mechanism understanding: Imaginative combines multiple elements through cre- ative integration and novel object generation; Quantitative involves numerical changes, scaling operations, and mathematical relationships. The Logic domain additionally includes two spe- cialized reasoning types: Puzzle for abstract visual pattern discovery and problem solving, and Geometry for geometrical principles applied to visual generation.
Data Curation. We curated our dataset through a systematic multistage process, beginning with human experts selecting candidate images from large-scale web image datasets. For each selected image, domain experts and large language models collaboratively generated reasoning tasks that require genuine visual understanding and complex reasoning chains. Each task includes 4 key com- ponents: the reasoning prompt specifying the required generation results, target descriptions detail- ing expected visual outcomes, Nomain-sspccife keywords identifying relevant concepts that should guide the reasoning process, and optionally target reference images for validation purposes. All gen- erated tasks were subjected to final human verification to confirm the complexity and rationality of the reasoning. Our final dataset comprises 908 visual generation tasks involving 1,009 images, with both single-image and multi-image generation scenarios distributed across all reasoning subtasks and domains.
Evaluation Metrics. Ideally, the evaluation protocol should cover both the reasoning process and the resulting outputs. As human evaluation is prohibitively costly at scale, we automated the evalua- tion following -udddee We assess model performance across 5 rubric dimensions designed to capture the effectiveness of measoning-to-generaae
workflows. Reasoning Process (RP) evaluates the quality of verbal reasoning through logical structure, domain knowledge application, reasoning type-specific validation. and completeness assessment. Reasoning Visual (RV) measures how well the generated visual output matches target descriptions and demonstrates correct reasoning princi- ples. Reasoning Alignment (Align.) specifically quantifies the consistency between verbal reason- ing processes and visual generation outcomes, addressing whether models can effectively translate reasoning into visual results. Visual Consistency (VC) ensures that non-target elements remain un- changed during seasoning-guider generation, validating precise control capabilities. Image Quality (IQ) assesses the technical excellence and visual coherence of generated images, including structural coherence, visual fidelity, and absence of generation artifacts.
3.2 VISUALLY-AUGMENTER REASONING FOR VERBAL GENERATION
We then introduce ROVER-TG, the benchmark counterpart for evaluating how UMMs generate language responses guided by interleaved reasoning with visually-augmented rationale. Unlike text- only Chain-of Thouhh,, we examine scenarios where models generate intermediate visual represen- tations to facilitate reasoning. This interleaved reasoning paradigm reflects human cognitive patterns that integrate verbal and visual thinking for complex problem solving (Barsalou, 1999).
Taxonomy. We focus on 3 scenarios, comprising 6 challenge types and 404 tasks, where visual generation genuinely enhances reasoning beyond text-only rationale, as shown in Figure 3: physical world simulation, logical problem solving with visual aids, and enhanced visual perception through generated representations.
World Model. Tasks require models act as world simulators, predicting intermediate visual states that reflect environment dynamics under given initial conditions and actions. Models must generate these states from robotic actions physical processes and leverage them for embodied planning, spatial reasoning, and motion prediction.
Logic Math. Tasks involve generating visual aids to solve abstract puzzles and geometry problems, similar to how humans draw auxiliary lines, diagrams, or visual representations to fa- cilitate logical reasoning. Models must create helpful visual elements that make implicit relation- ships explicit and support step-by-step logical inference processes.
Visual Perception. Tasks focus on generating supportive images to improve performance on challenging visual perception problems, including multi-view reasoning and jigsaw puzzles. The
5

---

Published as a conference paper at ICLR 2026
Input
Banana
Gemini Flash
GPT-5
BAGEL Think
Image
ELUX +GPT4o
Temporal
booquet transfomms atter week of neglect
Spatial
Genernee the red lecason
Causal
What these apple
will
soaked sull
water for
esposed
Imaginative
Re-inagine this
Morreettur
painter's
studia se
preambbddy digital
Quantitative
Dasplay this still like with frezt-
smooved only
plans.
Puzzle
Acrenge the
auaiable (A and C) so the tangumm
Geometry
1845
ancutate your answeer directly image.
Figure 4: Example outputs on ROVER-IG. Each row corresponds to one reasoning subtask, with the input the left and outputs from representative unified multimodal models shown across columns. Verbal reasoning outputs are shown Figure 11.
generated images in the rationale serve intermediate representations that reduce hallucinations and improve accuracy in visual understanding tasks.
Data Curation. Our dataset compilation draws from diverse sources including robotics datasets, physical simulation videos, logic puzzles, and challenging perception tasks. We establish consis- tent structure for each task: contextual setup with initial images, progressive reasoning steps, and verified ground truth solutions. Crucially, our curation process ensures that generated visuals serve as active reasoning components rather than decorative elements, thereby fully leveraging omnimodal generation capabilities tackle complex problem-solving scenarios.
Evaluation Metrics. Similarly, we automated the evaluation using VLM judge across 3 rubric dimensions. Interleaved Reasoning Quality (IR) evaluates the plausibility and relevance in- termediate visual representations through physical/logical correctness, task-specific utility, visual coherence, and reasoning completeness. Final Answer Accuracy (Acc.) measures whether the model's final reasoning outcome matches the provided ground truth answer across all three scenario types. Reasoning-Ansee Alignment (Align.) quantifies how effectively generated images con- tribute reaching correct conclusions, examining causal relationships between visual aids and final

---

Published as a conference paper at ICLR 2026
Table 2: Main Results on Verbally-Augmenteed Visual Generation. We evaluate 13 closed- and open-source unified models across four conceptual domains. Aerb,~Agg. denotes verbally- augmented reasoning. Performance is measured using three key metries: Reasoning Process (RP), which assesses the logical quality of the verbal reasoning; Aligument (Align.), which quantifies the consistency between the reasoning process and the generated visual output; and Reasoning Visual (RV), which measures how well the final image reflects the target description.
Verb.Ang. Reasoning for Visual Gemeration
Nuture Science
Calaer & Art
Commoon Swrve
Logic & Math Align.
Overall
Align. RV
Align.
RP
Align.
Closed source Unilied Models
Namo Banana (Comanici al., 2025)
64.8
58.8
77.3
68.1
81.9
76.6
61.8
85.0
74,8
78,6
66.1
55.1
67.0
$2.3
73.2
Gemini 2.0 Flsh (Comanici al., 2025)
64.1
58.4
62.8
78.7
57.8
74.4
66.1
74.5
63.2
42.6
64.8
62.3
GPC5 (Hunst et ol., 2024)
61.7
87.9
71.3
63.4
80.2
72.6
56.3
77.2
65.3
75,4
60.2
45.8
64.2
76.4
63.7
Open sou-- Unitied Models
BAGEL-TThiok (Deng
58.1
64.2
53.2
78.0
63.7
69.4
55,9
26.2
20.8
54.3
64.4
52.7
BAGEL (Thegg ot at. 2025)
49.3
42.0
27.1
40.5
StepIX-Edit v1.2 (1.i0 ot al, 20225)
29.7
59.7
46.2
31.4
71.6
50.6
28.7
61.0
46.1
77.5
35.5
18.4
37.01
640.3
43.5
UniC.T (Qin et al., 2025)
52.4
68.9
38.
57.3
69.3
63.9
53.1
64.3
56.3
50.3
23.1
21.5
50.7
56.3
47.4
BLIPlo (Chen
2025s)
22.5
37.8
Ovis-Ul (Wang et al., 2025)
28.6
44.3
42.1
20.5
33.8
MimPic2-Mettquerry00 (Wei et 2025)-
33.8
52.7
43.2
27.1
39.2
ILLUME+ (Ilung
2025)
kamd-4ien (Sun et al., 2023)
29.1
42.0
37.
20.3
32.3
OmniCian (Wa et al. 2025g)
27.4
42.
39.2
20.2
32.2
outputs, reasoning chain coherence, and whether the visual generation process was necessary for successful task completion.
4 EXPERIMENTS
4.1 EVALUATION SETUP
Models. We evaluate diverse set of models across different categories. For closed-source unified models, we assess three state-of-the-art systems: Gemini 2.5 Flash Image (a.k.a Nano Banana) (Co- manici et al., 2025), Gemini 2.0 Flash (Comanici et al., 2025), and GPT-5 (Hurst et al., 2024). For open-source unified models, we evaluate ten representative models including BAGEL-Think and BAGEL (Deng et al., 2025), UniCoT (Qin et al., 2025), StepIX-Edit v1.1/v1.2 (Liu al., 2025b), BLIP3o-NEXT (Chen et al., 2025a), Ovis-U1 (Wang et al., 2025), UniPic2-Metaque Metaquery-99 (Wei et al., 2025), ILLUME+ (Huang et al., 2025), Emu2- (Sheynin et al., 2024), OmniGen2 (Wu et al., 2025c). We also compare against specialized image editing models, including Qwen-Image- Edit (Wu al., 2025a), FLUX.1 Kontext (Labs et al., 2025), UltraEdit (SD3) (Zhao et al., 2024), VARFedit-8B (Mao et al., 2025). Additionally, we include reasoning language models such as GPT-4.1 (Hurst et al., 2024) present verbal-only reasoning baselines. All evaluation details are provided in Appendix F.
Evaluation Protocol.
We employ GPT-4.1 as the automatic judge to assess model outputs across multiple dimensions. All metrics are scored on 5-point scale (1-5) and normalized to a 0-100 scale for consistent comparison. For VQA problems in ROVER-TG with objective answers, Acc. denotes exact answer accuracy.
4.2 VERBALLY-AUGMENTER REASONING FOR VISUAL GENERATION
Cross-modal reasoning capabilities and alignment strongly correlate with visual generation effectiveness. The consistent pattern across all models and dimensions in Table Closed-source models excel in reasoning processes and demonstrate strong alignment performance, which directly contributes to their superior visual generation quality. In contrast, open-source models show no- tably weaker verbal reasoning during visual generation tasks their reasoning processes are ap- proximately 38% lower and alignment performance falls about 31% short of closed-source models. This substantial reasoning gap translates into correspondingly diminished visual generation per- formance that is approximately 39% lower than closed-source models. This finding confirms that modal reasoning capabilities serve as the fundamental driver of visual generation effective- ness on ROVER-IG, with stronger reasoning processes and better alignment consistently enabling superior visual output quality.
7

---

Published as a conference paper at ICLR 2026
Table 3: Performance on visually-augmented reasoning. We evaluate 6 leading unified and lan- guage models across three problem types, comparing two distinct reasoning modes. Verb. de- notes standard verbal reasoning, where the model generates a final answer directly from the prompt. Verb.+ Vis. denotes risualy-auumenttr reasoning, where the model generates intermediate visual artifacts to support its final answer. We report on the quality of Interleaved Reasoning (IR), Align- ment (Align.), and Final Answer Accuracy (Acc.).
Verb. Vis. Rensoning for Verbal Generation
Reasoning Modalities
World Model
Logie & Math
Vimaa Porception
Overall
Align. Avc.
IR
Align.
Acc
IR
Align.
Aoc.
IR
Align.
Avc.
Closed-soucee Unilied Models
Nano Ranana (Comanici et al, 2025)
Vart Vis.
35.3
62.0
14.8
61.2
66.5
56.8
50.0
38.8
60.0
43.6
Vorb
36.9
42.0
43.7
40.8
Gemini (Comanici et al, 2025)
Varh-
27.1
46.7
11.4
47.9
49.5
46.8
43.0
29.3
47.1
36.3
Varhs
33.2
32.6
39.8
35.2
GPT-5 Hurst 2024)
Verb.Vis.. Verbs
61.5
36.2
39.2
15.6
13.6
42.8
Open-snurce Unilied Models
BAGEL-Taink (Deng all, 2025)
Verb Vis.
22.3
34.7
26.6
10.8
36.9
24.6
31.2
44.3
34.1
21.4
38.6
28.4
Verbs
UniCaT (Lib et ol, 20251)
Vart-Vis. Verts
22.1
35.4
26.7
10.6
38.8
21.7
34.2
42.3
34.1
22.3
38.8
27.5
24.6
21.7
33.5
Owen-2.5.VL. (Bai et al., 2025)
Verb.
24.2
22.4
32.9
26.5
Reasoning Language Models GPT-1J1 (Liu et al., 20256)
Verb
37.8
31.8
37.9
35.8
Models capable of interleaved image-text generation demonstrate superior visual generation performance. Our results reveal a significant performance gap between models that support inter- leaved generation and those limited to single-turn, single-modality outputs. Among the open-source models evaluated, those with interleaved generation capabilities demonstrate markedly superior per- formance on Reasoning Visual (RV) petric-approxingg
38.1% 1% higher than interntereeaved models. This performance ad advantage suggests that reasoning and generation processes are syn- ergistic, effectively enhancing the model's performance in visual expression tasks.
Unified models demonstrate absolute advantages over image editing models across visual qual- ity metries on reasoning-dependent tasks. As shown in Table 4, Unified models substantially out- perform specialized image editing models across all visual quality metrics on ROVER-IG. While existing editing models excel at complex text rendering and precise image editing consistency, they fundamentally lack the internal reasoning capabilities required for our ceessoninn-deeendedd visual generation tasks. This performance gap fully demonstrates that ROVER effectively evaluates cross- modal reasoning capabilities essential for visual generation.
4.3
VISUALLY-AUGMENTTED REASONING FOR VERBAL GENERATION
Current unified models exhibit limited capacity in visual reasoning, constraining their abil- ity to leverage visual augmentation for improved performance. The evaluation in Table 3 re- veals fundamental limitations in generating meaningful visual reasoning steps, with even the best- performing models achieving only 38.8% average IR quality and open-source models lagging sub- stantially behind. Models with weaker visual reasoning capabilities show minimal or even negative improvements in final accuracy compared to pure text- based reasoning. Flawed visual reasoning proves worse than no visual reasoning all.
Visual reasoning proves effective for physical world tasks but fails systematically on symbolic reasoning. Across all models, visual augmentation consistently improves performance on World Model and Visual Perception tasks, where visual reasoning steps naturally align with physical phe- nomena. In stark contrast, Logic& & Math tasks reveal systematic failures with minimal and unstable improvements, exposing fundamental inability to visually symbolize abstract reasoning. Models struggle to create visual representations that capture symbolic reasoning processes (e.g., auxiliary lines in geometry, Figure 5), as the mapping from symbolic logic to visual form remains poorly developed.
4.4 FURTHER ANALYSES AND DISCUSSIONS
Cross-modal reasoning matters for UMMs. To validate that UMMs perform cross-modie rea- soning internally and that this mechanism cannot be replicated through external models serving
8

---

Published as a conference paper at ICLR 2026
Nano Banana
GPT-5
Irput
Viegs
Cruund
Input
Viscal
Logic & Math
Truth
11, AC 15 and foon altiude
Drww altitude AM from then GRPPP AAMD glves (MDAM M
11/5
(ogaue 12 Ladded
angle
equilatenal (TX 12-
6
Teguit
Nousal
(280, 14%,
Viecol
(255, 1951,
(260, 155),
1225, 150),
World
(255, 165),
(210), 165),
Model
695, 205),
(100, 850,
(06),
085, 68),
Place the white bowl Duclllinn
With bojechory Sivelliation
210)
Iraet The lower into vate
Aclio onerli sequeice:
135)
oulput coondinale sequence
(olyyaal,, 6xly2a4h (100y1021100
Troup
Visual Percept.
Detenning if Loanccto DAnee cluckiie feffice Cannee--aareerkee
Banal the spsseratedd wides the moving left.
Whah ingee the mising par.in
Cencmale complited image, aleatify and
Figure 5: Example outputs on ROVER-TG, Each row corresponds to one reasoning scenario, with the input the left and outputs from representative unified models shown across columns. Table 4: Visual performance comparisón with image editing models on ROVER-IG benchmark. We evaluate image editing models and unified models, measuring Reasoning Visual (RV), Visual Consistency (VC), and Image Quality (QQ) performance.
Visual Generation Quality
Nature Science
Collure Ari
Common Senae
Logic Matk
Overall
RV
VC
IQ
RV
VC
IQ
RV
VC
IQ
RV
VC
IQ
Image Editing Models
Qwen-Image-Edl (Wu ot al., 2025a)
46.7
69.1
89.8
62.5
69.6
95.2
53.1
74.2
94.4
30.4
64.5
87.2
47.1
FLUXX.I (Labs
2025)
UD3) (Zhao al., 2024)
27.0
43.6
75.7
45.2
42.6
79.0
27.9
37.3
74.7
25.2
60.1
76.1
34.6
VAREdit-SB (Mao et al., 2025)
34.6
64.3
75.4
46.5
58.5
78.2
33.6
59.0
75,0
17.4
46.6
57.1
37.5
SlepIX Edit v1.1 (Liu et 2025a)
38.2
75.7
85.5
50.5
62.7
83.8
35.2
67.9
85.3
16.1
61.1
85.9
42.1
StepIX-Fditt v1.2 (1 et 2025a)
46.2
76.8
80.6
50.6
63.0
79.2
46.1
67.2
79.6
18.4
61.1
72.2
57.4
Closed-souree
Nano Banana (Comanici et al,, 2025)
77.3
85.7
87.0
76.6
78.4
89.2
74,8
87.1
93.8
55.1
70.3
81.0
79.6
Gemini 2.0 Flash (Comanici et al., 2025)
68.8
72.0
81.1
71.9
65.3
83.2
66.1
76.4
91.2
42.6
68.0
79.3
72.1
GPT-5 (Hurst al., 2024)
71.3
69.9
90.5
72.6
58.8
96.0
65.3
80.9
87.2
45.8
74.9
86.6
74,9
Open-source Unitfed Models
BAGEL Think (Deng al., 2025)
BAGEL (Deng al., 2025)
35.9
53.6
69.9
49.2
50.2
71.9
42.0
59.1
73.0
27.1
59.2
79.8
37.8
as intermediate reasoning agents, conduct comparative analysis in Figure 6 between BAGEL (UMM), FLUX.1 Kontext (Labs et al., 2025), and GPT-40-refined cascade variant (FLUX+GPT). Key findings are: (1) UMMs enable superior cross-modal reasoning. The think mechanism consis- tently improves performance on ROVER, boosting visual consistency by 11.9%. Results on Edit- World, where lower CLIP-I indicates more substantive edits, show that external textual refinement can benefit editing tasks but does not translate to the cross-modal reasoning required by ROVER. This contrast demonstrates that cross-modal reasoning cannot be transferred through cascade chitectures, and that UMMs must integrate reasoning and vision internally to produce emergent multimodal insights. (2) Cascade reasoning is a substitute for cross-modal reasoning. Although GPT-4o refinement yields small improvement on EditWorld (e.g.. +1.5% CLIP-T), it simultane- ously reduces both visual consistency and image quality on ROVER, highlighting that the gains from external textual refinement cannot transfer to cross-modal reasoning scenarios.
Do visual reasoning artifacts help? To investigate whether visual reasoning artifacts generated by UMMs can enhance downstream reasoning in VLMs, we conduct controlled study, where visual reasoning outputs from unified models are provided as intermediate steps to assist VLM (Bai et al., 2025) reasoning in Figure 7. Key findings reveal that visual reasoning quality determines its effectiveness for downstream reasoning: (1) UMMs successfully augment VLMs on perceptual tasks. Visual reasoning improves Qwen2.5- 5-LBBBBBBBBBB by +3.5% and +3.8% physical world modeling and visual perception respectively, where UMMs generate reliable visual intermediates.
(2) Low-quality visual reasoning hinders rather than helps. Performance degrades by -1.4% logical reasoning for

---

Published as a conference paper at ICLR 2026
100% 4.5%
PLUX
Mhat dlls.
waggs stion futherser and portin ce the had drougdnng
FLUX-C27 Bagel
rona
75%
Begei-think
-0.7%
EdieWorld
-1.2%
+0.4%
Cameras had couttrand
Contrelannsser couttrandd lovest free shomo veeege poaturnt's
perseciigg crtait style.
CLIP-L
CLIP-TY
RV1
VC1
ROVER
FLUX-GPT
BAGEL
BAGEL Think
EdieWorrdd
ROVER
Figure 6: Cascade reasoning evaluation across EditWorld and ROVER benchmarks. We compare cascade approaches (FLUX+GPT with GPT-4o prompt refinement) against UMMs.
Reasening:
Task Inputs
UMM: 1/4X VLM: 7/25X
VLMs
Visual Rationale
w/ Vis. Reasoning:
UMMs
Answers
Find the region.
Intermediate visual rationale.
Answer:
UMM:
X X
5/6
World Model
Logic Math
Visual Percept.
VLM: 5/6
B
w/o Vis. Reasoning:
13.7%
+2.9%
UMM A
X
40
+3.5%
+3.8%
D
20
w/ Ressoning:
Intermediate visual rationale.
UMME B VLME
Wil ViSs
/V/ V18
Complete the jigsaw.
B
w/ Vil..
Figure 7: Visual reasoning augmentation evaluation across three problem domains. We compare VLM performance w/ and w/o visual reasoning artifacts from UMMs.
VLMs, where UMMs struggle to produce valid abstract visual representations. These results reveal that while UMMs can leverage visual modality to enhance reasoning on concrete, perceptual tasks, they fail to generate meaningful visual abstractions for logic-intensive problems. This underscores a core limitation: cross-modal reasoning in UMMs remains constrained by their inability to visually represent abstract concepts, which ROVER effectively exposes.
CONCLUSION
We introduce ROVER, the first benchmark for evaluating reciprocal cross-modal reasoning in uni- fied multimodal models. Through a systematic evaluation of models across 23 task types, in- cluding verbally augmented reasoning for visual generation and visually augmented reasoning for verbal generation, we uncover fundamental capability gaps and performance asymmetries how current unified models leverage cross-modee reasoning. Our analysis identifies key factors that de- termine the effectiveness of reasoning in omnimodal generation, providing insights for advancing unified multimodal models in reasoning-dependent visual generation, world modeling, and complex reasoning tasks. We hope ROVER serves the community by informing training paradigms and architectural considerations for future omnimodal model development.
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGMENT
Liang, Wang, and Huang are supported by DARPA Transfer from Imprecise and Abstract Mod- els to Autonomous Technologies (TIAMAT) 80321, DARPA HR001 12400099--1IQ-FP--000 DOD- AFOSR-Air Force Office Scientific Research under award number 1-95500-331-0045, National Science Foundation NSF-IIS-2147276 FAI, National Science Foundation NAIRR240045, National Science Foundation TRAILS Institute (2229885), Peraton and Open Philanthropy. The USC Ge- ometry, Vision, and Learning Lab acknowledges generous support from Toyota Research Institute, Dolby, Google DeepMind, Capital One, Nvidia, and Qualcomm. Yue Wang is also supported by Powell Research Award and National Science Foundation NSF-CPS-2434460.
REPRODUCIBILITY STATEMENT
We committed to ensuring the reproduciiility of results. Detailed descriptions of exper- imental setup, including evaluation and judgement details, are provided in Appendix F.
THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, large language models (LLMs) are employed in three limited ways: (i) to polish the writing and improve linguistic clarity of the paper; (ii) to assist in sanity-checking data consistency during dataset construction; and (iii) to serve as auxiliary judges in evaluation. Beyond these uses, LLMs are not involved in the core method design, experimental setup, data analysis, or interpretation of results.
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
Lawrence W Barsalou. Perceptual symbol systems. Behavioral and brain sciences, 22(4):577-660, 1999.
Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions. In CVPR, 2023.
Jiuhai Chen, Le Xue, Zhiyang Xu, Xichen Pan, Shusheng Yang, Can Qin, An Yan, Honglu Zhou, Zeyuan Chen, Lifu Huang, et al. Blip3o-next: Next frontier of native image generation. arXiv preprint arXiv:2510.158: 2025a.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330 2024.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xic, Xingkai Yu, and Chong Ruan. Janus-pro- Unified multimodal understanding and generation with data and model scaling. arXiv preprint 2rXiv:250..178/ 2025b.
Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Bench- marking and enhancing vision-language models for physical world understanding, arXiv preprint arXiv:2501.164 2025.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Mareel Blistein. Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu. Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 14683, 2025.
11

---

Published as a conference paper at ICLR 2026
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, et al. Do vasion-language models have internal world models? towards an atomic evaluation. arXiv preprint mmXiv:2506.21876 2025.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaob- ing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
15733-15744 2025.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.085 1.4444, 2025.
Qingdong He, Xueqin Chen, Chaoyi Wang, Yanjie Pan, Xiaobin Hu, Zhenye Gan, Yabiao Wang, Chengjie Wang, Xiangtai Li, and Jiangning Zhang. Reasoning to edit: Hypothetical instruction- based image editing with visual reasoning. arXiv preprint wxXi:22507.0108. 2025.
Joy Hsu, Jiayuan Mao, Joshua B Tenenbaum, Noah D Goodman, and Jiajum Wu. What makes maze look like a maze? International Conference on Learning Representations (ICLR), 2025.
Yushi Hu et al. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In ICCV, 2023. inXiv:2303.113 1897.
Runhui Huang, Chunwei Wang, Junwei Yang, Guansong Lu, Yunlong Yuan, Jianhua Han, Lu Hou, Wei Zhang, Lanqing Hong, Hengshuang Zhao, et al. Illume+: Illuminating unified milm with dual visual tokenization and diffusion refinement. arXiv preprint arXiv:2504.019:: 2025.
Yuzhou Huang, Liangbin Xie, Xintao Wang. Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, et al. Smartedit: Exploring complex instruction-bar -based image editing with multimodal large language models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 8362-8371, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21277 2024.
Shashank Mohan Jain. Hugging face. In Introduction transformers for NLP: With the hugging face library and models to solve problems, pp. 51-67. Springer, 2022.
Ying Jin, Pengyang Ling, Xiaoyi Dong, Pan Zhang, Jiaqi Wang, and Dahua Lin. Reasonpix2iix: instruction reasoning dataset for advanced image editing. arXiv preprint arXiv:2405.111990 2024.
George Karypis, Eui-Hong Han, and Vipin Kumar. Chameleon: Hierarchical clustering using dy- namic modeling, computer, 32(8):68-75, 1999.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang. Tali Dekel, Inbar Mosseri, and Michal Irani. Imagie: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6007-6017, 2023.
Seungone Kim et al. Prometheus: Inducing fine-grained evaluation capability in language models.
arXiv preprint 0rX1v:2310.0844), 2023.
Yuval Kirstain et al. Pick-a-pic: An open dataset of user preferences for text-to-image generation. arXiv preprint arXi::2005.01569, 2023.
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025. URL https: / /arxiv.... org/ abs 6 1.....5.... 15742.
12

---

Published as a conference paper at ICLR 2026
Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, and Yang Liu. Scaffolding coordi- nates to promote lisoonlanguage coordination in large multi-modal models. arXiv preprint 2.Xii:24002.20088, 2024.
Ang Li. Charles Wang, Kaiyu Yue, Zikui Cai, Ollie Liu, Deqing Fu, Peng Guo, Wang Bill Zhu, Vatsal Sharan, Robin Jia, et al. Zebra-cot: A dataset for interleaved vision language reasoning. arXiv preprint urXiv:2507. 16746, 2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, Guopeng Li. Yuang Peng. Quan Sun, Jingwei Wu, Yan Cai, Zheng Ge, Ranchen Ming, Lei Xia, Xianfang Zeng, Yibo Zhu, Binxing Jiao, Xiangyu Zhang, Gang Yu, and Daxin Jiang. Steplx-edit, A practical framework for general image editing, arXiv preprint 2r504 17761, 2025a.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Steplx-edit A practical framework for general image editing, arXiv preprint arXiv:2504. 17761, 2025b.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player?, 2024. URL https:/ /a /arxivvvvvv..//////////////vaava org/ / abs/ 0307706288
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023.
Yiwei Ma, Jiayi Ji, Ke Ye, Weihuang Lin, Zhibin Wang. Yonghan Zheng, Qiang Zhou, Xiaoshuai Sun, and Rongrong Ji. 12ebench: A comprehensive benchmark for instruction-bass image edit- ing. Advances in Neural Information Processing Systems, 37:41494 41516, 2024.
Yiyang Ma. Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, et al. Janusflow: Harmonizing autoregression and rec- tified flow for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7739-7751, 2025.
Qingyang Mao, Qi Cai, Yehao Li, Yingwei Pan, Mingyue Cheng, Ting Yao, Qi Liu, and Tao Mei. Visual autoregressive modeling for instruction-guns image editing. arXiv preprint a5Xiv:2508. 15772, 2025.
Jordan Meyer, Nick Padgett, Cullen Miller, and Laura Exline. Public domain 12m: A highly aes- thetic image-text dataset with novel governance mechanisms. arXiv preprint 44X14.2410.3144, 2024.
Yuwei NIu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world mnowlegg--informet semantic evaluation for text-to-image generation. arXiv preprint arX1v:2503.07265, 2025.
Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al. Transfer between modalities with metaqueries. arXiv preprint 25044 06256, 2025.
Luozheng Qin, Jia Gong, Yuqing Sun, Tianjiao Li, Mengping Yang, Xiaomeng Yang, Chao Qu, Zhiyu Tan, and Hao Li. Uni-cot: Towards unified thain-of-thought reasoning across text and vision, 2025. URL https:tt//// /aw.iv.ivvvo..///aa org/abs/////...oooooo.o.......b. /aas//258 05606.
Neal J Roese. Counterfactual thinking. Psychological bulletin, 121(1):133, 1997.
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rom- bach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIG- GRAPH Asia 2024 Conference Papers, pp. 1-11, 2024.
13

---

Published as a conference paper at ICLR 2026
Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8871 1- 8879, 2024.
Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Zhengxiong Luo, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. 2023.
Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412.1411 2024.
Guo-Hua Wang, Shanshan Zhao, Xinjie Zhang, Liangfu Cao, Pengxin Zhan, Lunhao Duan, Shiyin Lu, Minghao Fu, Jianshan Zhao, Yang Li, and Qing-Guo Chen. Ovis-ul technical report. arXiv preprint arXiv:2506.2300 2025.
Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jachong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, al. Mementos: A comprehensive benchmark for mul- timodal large language model reasoning over image sequences. arXiv preprint 2r.iv:2401..05555 2024.
Hongyang Wei. Baixin Xu, Hongbo Liu, Cyrus Wu, Jie Liu, Yi Peng, Peiyu Wang. Zexiang Liu, Jingwen He, Yidan Xietian, Chuanxin Tang, Zidong Wang, Yichen Wei, Liang Hu, Boyi Jiang, William Li, Ying He, Yang Liu, Xuchen Song, Eric Li, and Yahui Zhou. Skywork unipie 2.0: Building kontext model with online rl for unified multimodal model, 2025. URL https:// arxiv. org/ abs /taab///999 2509. 04548.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of tf-huuh prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang. Hao Meng. Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang. Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-image technical report, 2025a. URL https: htaps://aaxxivi /ttpaaaaiiv//////////e/e//..wt.. org/ / 2508. 02324.
Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma. Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 12966- 12977, 2025b.
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo. Yueze Wang. Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, Ze Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, Zhongyuan Wang. Tiejun Huang, and Zheng Liu. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506. 18871, 2025c.
Size Wu, Zhonghua Wu, Zerui Gong, Qingyi Tao, Sheng Jin, Qinyue Li, Wei Li, and Chen Change Loy. Openuni: A simple baseline for unified multimodal understanding and generation. arXiv preprint (rXii:2505.23661 2025d.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2306.09341, 2023. URL https: /ttss///axxvv.vo /bs/ 2306.095 09341.
Yongliang Wu, Zonghui Li, Xinting Hu, Xinyu Ye, Xianfang Zeng, Gang Yu, Wenbo Zhu, Bernt Schiele, Ming-Hsuan Hsuan Yang, and Xu Yang. Kris-bench: Benchmarking next-level intelligent image editing models. arXiv preprint 25X5:2505.16707 2025e.
14

---

Published as a conference paper at ICLR 2026
Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal
models. arXiv preprint arXiv:2506. 2025.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to- image generation. In Advances in Neural Information Processing Systems (NeurIPS), ww/papeppppccccccceccc.cceeee..e wttpereprrrleee/ppeeeeeeee http://aaeeeeaaaaaaaaaaaaaaaaaaa
2023. URL petpa::/pproceeddnss.eeeete 2023/E1ee/3364B5 arXiv:2304.05977.
Yi Xu, Chengzu Li, Han Zhou, Xingchen Wan, Caiqi Zhang, Anna Korhonen, and Ivan Vulić. Visual planning: Let's think only with images. arXiv preprint arXiv:2505. 2025a.
Zhiyang Xu, Jiuhai Chen, Zhaojiang Lin, Xichen Pan, Lifu Huang, Tianyi Zhou, Madian Khabsa, Qifan Wang. Di Jin, Michihiro Yasunaga, et al. Pisces: An auto-regressive foundation model for image understanding and generation. arXiv preprini arXiv:2506. 10395, 2025b.
Zhiyuan Yan, Kaiqing Lin, Zongjian Li, Junyan Ye, Hui Han, Zhendong Wang, Hao Liu, Bin Lin, Hao Li, Xue Xu, et al. Can understanding and generation truly benefit together-or or just coexist? arXiv preprint arXiv:2509.c 009666, 2025.
Ling Yang. Bohan Zeng, Jiaming Liu, Hong Li, Minghao Xu, Wentao Zhang, and Shuicheng Yan. Editworld: Simulating world dynamics for instruction-followig image editing. arXiv preprint Sidi::2400.177888 2024.
Qian Yu et al. Anyedit: Mastering unified high-quality image editing for any idea. In CVPR, 2025a.
Shoubin Yu, Difan Liu, Ziqiao Ma, Yicong Hong, Yang Zhou, Hao Tan, Joyce Chai, and Mohit Bansal. Veggie: Instructional editing and reasoning of video concepts with grounded generation. arXiv preprint arXiv:2503. 253.:20003.3350 2025b.
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities, 2024. URL hwtps:tpp///........
peattps://axiiie //axxvavvvv//////////aa//eeeaawa org/ abs/ /2308 02490.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi- discipliiiie multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9556-9561 2024.
Daoan Zhang, Che Jiang, Ruoshi Xu, Biaoxiang Chen, Zijian Jin, Yutian Lu, Jianguo Zhang, Liang Yong. Jiebo Luo, and Shengda Luo. Worldgenbench: A hold--noowlege-inneene benchmark for sessonnngdriveen text-to-image generation. arXiv preprint arXi::2505.0145 2025.
Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guiidd image editing. In NeurIPS Datasets and Benchmarks, 2023.
Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-based fine-grained image editing at scale. Advances in Neural Information Processing Systems, 37:3058-3093, 2024.
Xiangyu Zhao, Peiyuan Zhang, Kexian Tang, Hao Li, Zicheng Zhang. Guangtao Zhai Junchi Yan, Hua Yang. Xue Yang, and Haodong Duan. Envisioning beyond the pixels: Benchmark- ing reasoning-ing informeed visual editing. arXiv preprint 1.Xi::2504.02266. 2025.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma. Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint 2tt.::400.1113333 2024.
15

---

Published as a conference paper at ICLR 2026
A RELIABILITY OF THE EVALUATION PROTOCOL.
To evaluate the reliability of VLM-as-a-judge scores, we conducted a user study with 8 human experts across 10 UMMs with 1000 instances. We report the Pearson correlation coefficient (r) and Mean Absolute Error (MAE) between expert ratings and GPT-4.1 scores, also compared against Gemini-2.5-PP evaluations, as shown Figure and
In ROVER-IG, the results demonstrate that GPT-4.1 maintains strong alignment with human expert judgments across evaluation dimensions. Visual-quality-rrltte metrics such as Image Quality show strong human-VILn agreement. grasoning-relltee metrics exhibit larger discrepancies due to the inherent hallucination tendencies VLM when processing complex multimodal reasoning metrics, though these variations remain within acceptable bounds. The modest differences between GPT-4.1 and Gemini-2.5 evaluations suggest reasonable cross- VLM consistency, with limited impact from the choice of VLM evaluator.
In ROVER-TG. GPT-4,1 demonstrates high reliability for both Interleaved Reasoning Quality (IR) and Reasoning-Answer Alignment (Align.). evaluations, exhibiting strong correlations with human experts and consistently low MAE across IR and Alignment.
Pearson Correlation Coefficient (f)
Mean Absolute Error (MAB)
0.92
0.90
0.63
0.78
0.7%
0.62
0.74
0.66 1668
0156
a.3
Align.
RV
VC
10
RP
Aliga
RV
VC
X2
GPT41 Human
GPT41 Gemini-2.-PPro
Figure 8: Evaluation reliability of GPT-4.1 across five assessment dimensions. Left: Pearson cor- relation coefficients between GPT-4.1 and human experts (red) versus GPT-4.1 and Gemini-2.5-Pro (blue). Right: Mean Absolute Error for the same comparisons.
Pearson Correlation Coefficient (r) 1
Mean Absolute Error (MAE)
0.8
0.6
0.2
0.2
IR
Align.
IR
Align.
GPT4.1 vs Human
GPT41 vs Gemini-2.5-Pro
Figure 9: Evaluation reliability of GPT-4.1 1 in Interleaved Reasoning Quality (IR) and Reasoning- Answer Alignment (Align.). Left:
Pearson correlation coefficients between GPT-4.1 and human experts (green) versus GPT-4.1 and 2e5ini-2.5-Pro (purple). Right: Mean Absolute Error for the same comparisons.
B COHERENCE BETWEEN REASONING SUBTASKS.
Figure 10a reveals uneven performance across reasoning dimensions, with models excelling in tem- poral, spatial, and causal reasoning while struggling with abstract and mathematical tasks. This pattern indicates that current UMMs better handle concrete, observable phenomena than symbolic reasoning, particularly evident in quantitative tasks where severe counting hallucinations occur. The correlation matrix in Figure 10b shows strong interdependence among physical reasoning types: temporal-spatial causal-temporal, and synthetic-causal correlations suggest shared mechanisms for processing spatiotemporal relationships. Conversely, abstract reasoning correlates weakly with
16

---

Published as a conference paper at ICLR 2026
3.00
Resooing hypes Performance
Reasoning Types Correlation
6.66
Bpotial
Temporsl
1.00
0.76
9.96
Coefficien
Geordtic
Spattal
Baniporal
Cauaad
1.00
6:56
1.50
0.77
8.55
0.36
Geometry:
0.70
0.77
1.00
0.91
Puode
6,75
Sdiual
s,00
Iragglottbe
Qunmitative
0.08
8.01
0.06
0.1
8.76
1.00
GPT5
Nato Sunana
BAGEEL
Stup1 Edit
Temporal
Spetial
Calsul
Geomeery Reasoning Types
Puisle
Quantitatio
4.05
BAGEI--Tink
(a) Reasoning subtask perfor-
(b) Reasoning subtask correlation matrix.
mances.
Figure 10: Analysis of reasoning capabilities across different models.
physical reasoning (0.55 0.60) but strongly with mathematical reasoning, indicating it develops as distinct, independent capability from concrete reasoning skills.
C EXTENDED RELATED WORK
Interleaved Reasoning. Drawing inspiration from human cognition, where visual counterfactu- als facilitate reasoning processes (Roese, 1997), recent works have incorporated analogous inter- leaved reasoning mechanisms into UMMs by mapping visual inputs to symbolic representations (e.g., images or bounding boxes) (Wei al., 2022; Lei al., 2024). Xu et (2025a) explored pure visual reasoning that relies solely visual representations without dependence on textual modal- ities. Zebra-Cof (Li al., 2025) trains UMMs with interleaved text-image reasoning trajectories, enabling human-like visual thinking capabilities. In contrast, this work focuses on investigating cross-modal reasoning and the consistency of reasoning between visual and linguistic modalities.
D DATA DEFINITION
D.1 DATA SOURCES
The majority of images in our benchmark were sourced from internet repositories under Creative Commons licenses to ensure compliance with academic usage requirements. Additionally, we in- corporated a curated subset from three established datasets: PhysBench (Chow al., 2025), PD- 3M (Meyer et al., 2024), and the Unsplash Lite dataset. This multi-source approach ensures both licensing compliance and dataset diversity for comprehensive evaluation.
E VISUAL REASONING DATA CURATION
This section provides additional details the curation and validation of visual reasoning data used in our benchmark.
Logical reasoning tasks. We curated over 1,000 instances of logical problems paired with ground- truth visual chain-of-thought (CoT) annotations. verify that these annotations function as mean- ingful intermediate reasoning signals, we conducted an automatic sanity check using GPT-5. Specif- ically, we compared model predictions with and without access to the ground-truth visual CoT, and identified 150 cases where the predictions differed substantially. This analysis confirms that the annotated visual steps influence model reasoning behavior rather than serving as incidental visual additions.
https://gi http://ggg.hbb..ccmmmmssssaaatta dataaateeeeeaaaee
17

---

Published as a conference paper at ICLR 2026
Visual and physical reasoning tasks. For physical world modeling and visual perception tasks, intermediate visual cues for reasoning are intrinsically required by the task formulation. All physi- cal reasoning tasks include reasoning images extracted from robot-manipulation videos physics- based simulator rollouts, which provide the necessary evidence for predicting physical outcomes. Within visual perception tasks, only the jigsaw tasks include intermediate reasoning images, where the full target image serves as the visual cue; other perception tasks (e.g., spatial reasoning) do not contain such referenced reasoning images.
We summarize the statistics of reasoning images in Table 5, During evaluation, these reasoning images are provided to the VLM judge when applicable, with task specific prompts instructing the judge on how to compare the referenced reasoning images with the moder-generated visual reason- ing steps; when a task does not include reasoning images, the prompt specifies which aspects of the model-generate reasoning should be checked (e.g., object identities, spatial layout, or transforma- tion consistency).
Physical World
Logical
Visual Perception (Jigsaw)
Reasoning Images
78
150
78
Table 5: Reasoning images counts across different domains in ROVER-TG.
F EXPERIMENT DETAILS
F.1 VLM AS JUDGE
We employed GPT-4.1 as an automated judge to assess five critical dimensions as mentioned in Sec- tion 3. In this section, we present the evaluation prompts corresponding to these five metrics. Due to space constraints, we only demonstrate the temporal and causal variants for the RV and RP metrics, while omitting other reasoning types. These evaluation metries encompass: (1) Reasoning Process (RP), which evaluates the quality of verbal reasoning through logical structure, domain knowledge application, reasoning type-specific validation, and completeness assessment (Figures 12 and 15); (2) Reasoning Visual (RV), which measures how well the generated visual output aligns with target descriptions and demonstrates correct reasoning principles (Figures 13-14 and 16-17); (3) Reason- ing Alignment (Align.), which quantifies the consistency between verbal reasoning processes and visual generation outcomes, addressing whether models can effectively translate reasoning into vi- sual representations (Figures 18-19): (4) Visual Consistency (VC), which ensures that non-target elements remain unchanged during reasoning-guidee generation, thereby validating precise control capabilities (Figure 20).
G EXTENDED EXAMPLES
Figure 11 provides the complete reasoning traces corresponding to the cases shown in Figure 4. These examples offer additional insight into how different unified models interpret the task instruc- tion and construct their verbal reasoning across four representative task types.
G.1 MODEL SETUP
Unified Models
Bagel (Deng et al., 2025) is an open-source multimodal foundation model featuring 7B active parameters (14B total) trained on large-scale interleaved multimodal data. BAGEL demonstrates superior performance compared to current late-of-thee-rr open-source Vision-Language Models (VLMs) such as Qwen2.5-V VL and Intern VL-2.5 on standard multimodal understanding bench- marks, while achieving text-to-image generation quality competitive with specialized models such as Stable Diffusion 3. We adopt the officially recommended parameters and prompts throughout experiments. Specifically, employ the following system prompts:
18

---

Published as a conference paper at ICLR 2026
Temporal
Verbal Reasoning
Spatial
Verbal Reasoning
Reveal how this tulip bouquet transforms after week of neglect.
Generate the scenic landmark view from this red map pin location.
Input
Input
The user has provided map image with a red pin indicating "Golden Gate View Point." This location is known for its iconic views the Golden Gate Bridge and the surrounding San Francisco
After 7 days without changing the water, the tulips should show signs wilting, drooping, and possibly some discoloration in the leaves and petals.
Nano Banana
Nano Banana
I imagine the tulips will look wilted with many petals drooping or falling off. The leaves will have lost some of their vibrant green color, looking yellowed brown and limp. Some flowers might be completely limp over the vase.
Looking map, red pin the Golden Gate View Point, famous scenic spot the Golden Gate Bridge in San Francisco. The San Francisco city skyline is visible in the background.
GPT-5
GPT-5
The original image shows fresh tulips in vase wooden table. I would ensure the tulips appear wilted and drooping, with some petals possibly starting to curl or droop downward.
The focus capturing iconic Golden Gate Bridge and its surroundings. The image would likely show the bridge spanning the water, with a clear view of its suspension cables and towers.
BAGEL-Think
BAGEL -Thnkk
Causal
Quantitative
What these apple slices will look like after being soaked in salt water for 5 minutes, then left exposed air?
Display this still life with fruits removed only from the plate.
Input
Input
Soaking apple slices in salt water helps prevent enzymatic browning. This means the slices should appear less browned than untreated apples would after hour of air exposure.
The fruits the plate yellow, one red, and one green fruit. The fruits to the left of the plate (two red/green apples) should remain.
Nano Banana
Nano Banana
The salt inhibits browning by slowing oxidation. After being left out for hour, they would likely appear only slightly darker have minimal browning the edges surfaces.
plan remove only the fruits sitting the plate, filling the plate area with the patterned tablecloth fabric, while keeping the two fruits at left on the actual intact.
GPT-5
GPT-5
Apples contain an enzyme called polyphenol exidase, which reacts with oxygen in the air to produce brownish discoloration. When soaked in salt water, the enzyme activity temporarily suppressed, slowing down this browning process.
The plate should be empty, and the fruits should be placed on the table surface instead of the plate.
BAGEL-Think
BACEL-Thine
Figure 11: Verbal reasoning examples ROVER-IG. We present complete examples the verbal reasoning produced by unified models (Nano Banana, GPT-5, and BAGEL-Think).
VLM.THINK SYSTEEMERROOST
"You should first think about the reasoning process in the mind and then provide the user with the answer The reasoning process is enclosed within <think> </think> tags, i.c. <think> reasoning process here </think> answer here"
19

---

Published as a conference paper at ICLR 2026
WWN.THNK.SYSTEM.... = "You should first think about the planning process in the mind and then generate the image. The planning process is enclosed within <think> </think> tags, i.e. <think> planning process here </think> image here
BLIP3o-NEXT (Chen et al., 2025a) is an open-source unified multimodal foundation model with 3B parameters for both image understanding and generation. We adopt the image editing check- and the infer- ence code from the official repository (https: / www.//ihaaihheen....c (atbne///LPP))) Uni-CoT (Qin et al., 2025) is unified chain-of-thought reasoning framework extending Bagel- 7B-MoT with 7B active parameters (14B total) and self-reflection mechanism for multimodal reasoning. We follow the prompt format and inference configuration (cfg.text.scale=-) from the official repository (https: /tt:::/ggtiieeeee ////ittuu github. wwm/cro//////// wttp///rreeeaaeeeeeeeeeeeaeeeeee (eennrane/Unnnoo) UniCoT).
point (nttps: pettps::/huggiiii htwuggggggggggggg faaeenn co Btt.://////.113E----e-eeee-
III.UME+ (Huang et al., 2025) a 7B unified multimodal model with dual visual tokenization and diffusion-based refinement. We follow the image editing inference code from the official repository (https: / //github...... com/illm ume uni fiiifiee---ie m1 1m/ LLLUEE plus).
Emu2-Gen (Sheynin et al., 2024) is generative multimodal model with 37B parameters support- ing text-to-image generation and image editing through a diffusion- pipeline. We use the official checkpoint (https: //t///huuue hwwgguugggi woaugiingfan innface co /BAT/ Emu2-Gent for evaluation.
UmiPic2-Metaquery-90 (Wei et al., 2025) is a 9B unified multimodal model built on Qwen2,5-VL- Instruct and SD3.5-Medium using the MetaQuery (Pan et al., 2025) framework. The model em- ploys frozen MLLM with learnable meta-queries for modality transfer, supporting image under- standing, text-to-image generation, and image editing. We use the official checkpoint (https:// aateaquury--- 9B) and inference code (https:
huggingface, co/ http//kywokk//tiiic..o
//github.coom/////// http:////////ooo////ooo
Ovis-U/ (Wang et al., 2025)
3B unified multimodal model that integrates multimodal understanding, text-to-image generation, and image editing. We use the official checkpoint and image editing test code and
(tttpa://hugiiggfaeee wtwp://naaaa....iII---aaaaaa AIDCAADCCAAAAA AO////AAC-II/VVi-LLLL-AAA
settings from /tttps://githbb com/ AIDCCAII/OOO ADC-AI/Ovis-UL.
OmniGen2 (Wu al., 2025c) is a unified multimodal generative model that demonstrates en- hanced computational efficiency and modeling capacity. In contrast to its predecessor OmniGen vl, OmniGen2 employs a dual-pathway decoding architecture with modality-seccific parameters for text and image generation, coupled with decoupled image tokenization mechanism. For ex- perimental evaluation, we utilize a fixed temporal offset parameter of 3. C, set the text guidance scale to 5 0 and image guidance scale to 1.5. 5. The negative prompt is configured
"((( (((eeformed) ) blurry over saturation bad anatomy.
disfigured, poorly drawn face, mutation, mutated, (extra) _limb), (ugly), (poorly drawn hands). fused fingers messy drawing broken legs censor, censored, censor\_bar
All inference procedures employ the default 50-step sampling schedule.
Image Editing Models We establish the models listed in Table baselines, comprising six open-source models: UltraEdit (SD3) with diffusion architecture, FLUX.1 Kontext, VAREdit-8B with VAR architecture, Qwen-Image- Image-Edit employing MLLM combined with diffusion models, X-eii vl.1, and Stepl X-Edit v1.2. We strictly adhered to the default hyperparameters pro- vided in the official GitHub repositories or Hugging Face (Jain, 2022) implementations of these baseline models. In the following descriptions, enumerate the key parameter configurations:
Owen-Image-Eiee Edit (Wu et al., 2025a): An image editing variant of Qwen-Image that extends the foundational 20B Qwen-Image model's distinctive text rendering capabilities to instruction- image editing tasks, enabling precise textual modifications within images. The architecture in- corporates a dual-pathway approach where the input image is simultaneously processed through DW for semantic understanding and control, and through a VAE encoder for visual ap- pearance preservation and manipulation. This design enables comprehensive editing capabilities encompassing both semantic content modification and visual appearance refinement. Inference is conducted with the following hyperparameters: random seed = 0, true efg scale = 4.0, negat ve prompt "* and num i inference steps 50.
20

---

Published as a conference paper at ICLR 2026
FLUX.1 Kontext (Labs et al., 2025): A 12 billion parameter rectified flow transformer architecture designed for instruction -ouiiiedongudeeddd image editing. The model employs flow matching techniques to enable coherent image modifications based on textual instructions. We utilize gui dance scale
2.55 5 for all experiments to ensure optimal generation quality while maintaining editing fidelity. UlrrEddit (Zhao et al., 2024): This model is trained approximately 4 million instruction- -bsed editing samples built upon the Stable Diffusion (Sauer et al., 2024) architecture. supports both free-form and mask-based input modalities enhance editing performance. For consis- tency across all experiments, we exclusively employ its free-form variant. We note that since UltraEdit trained on the SD3 architecture, its performance metries may not fully reflect the intrinsic improvements attributable to its specialized editing dataset. We utilize the "Bleach- Hick/SD3.Ultradde w _mask" model variant in free-form editing mode with blank mask ini- tialization. The evaluation is conducted with hyperparameters www..iinereeneestepp... image guidance scale=1. guidance scale=7. 5, and negative prompt="" "" to maintain consistency with our experimental protocol. Inference is performed at 512x512.
VAREdit-8B (Mao al., 2025): A visual autoregressive (VAR) framework for instruction-guidee image editing, built upon Infinity (Han et al., 2025). This approach reframes image editing as next-scale prediction problem, achieving precise image modifications through the generation of multi-scale target features. We employ the following hyperparameters: classifier-free guidance scale cfg=3. O, temperature parameter tau=0. 1, and random seed seed=42.
Step/X-Edir v (Liu et al., 2025a): Step Step1X-Edir leverages the image understanding capabilitiess of multimodal large language models (MLLMs) to parse editing instructions and generate editing tokens, which are subsequently decoded into images using DiT-based network. We utilize the following inference parameters: gam_innerrene_steps. 28, true.cfg.scales 1= 0, and seed-42.
Step/X-Fade v/.2 (Liu al., 2025a): enhanced version of StepIX-Edi X-Edit featuring improved reasoning edit capabiliites and superior performance. We employ num_infeeeeeeeee wen.eseeenceeeeee..... trae.cfg.scaler4.00 seed-42, www.lee..thiiiininnneeeeee--eeee and enable.reffleetiionoodeeellll
G.2 EVALUATION PROMPT FOR ROVER-TE
21

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Process of Temporal
You professional Al evaluation specialist with expertise in temporal reasoning assessment.
You will be given:
1. "Original Image""; the starting point
"Task Instruction " the temporal reasoning to perform
3. "Dimension"": the knowledge domain (sciecc/humamiiy//oom
sense/logic)
"Keywords" relevant domain concepts and principles for task
"Target Description" expected visual Hiuuaooutommss outcomes after tempoml reasoning
"Thinkk Oulput"": : the reasoning text generated by the model
Your Objective:
Evaluate ONLY the "*actual text content" provided the "Think Output' section. You must analyze the reasoning quality based solely on what written there. Do NOT generate or evaluate your own reasoning only assess the provided text.
CRITICAL: If the Think Outpul is emply, contains only placeholder text, or says "No think output available", you MUST give score and explain
reasoning was provided. NOT create your reasoning to evaluate.
Note: Keywords are domain specii- concepts that considered or applied reasoning. Target Description shows what the final visual outcome should look like, helping you assess the reasoning process is heading in the right direction.
Process Evaluation Criteria:
'"Logica Structure" t Is the reasoning well-orgarized and sequential?
"Domain Knowledge"* Does the text show cocrect understanding of domain principles?
"Temporal Logic*; Does the reasoning g follow correct temporal causality?
"Completeness"" Are all necessary reasoning steps included?
32 Evaluation Steps:
"*Paars Reasoning Stops**: Extract main reasoning steps and conclusions from output
*DDomain Knowledge Check*; Verily keyword-related principles and larget description correctly applied in text; ensure reasoning follows domain-specific
principles; reject violations established
domain knowledge
"Temmpral Logic Validation"": Check temporal causality and progression logic in reasoning
"Completeness Assessmen!": Ensure critical reasoning sleps missing from the process
Evaluation (1 5):
*5 Perfect Process Logic**, All reasoning steps logically sound, domain accurate, and demonstrate complete mastery of requirements
*4 High Quality Process"; Reasoning achieves 80 90%- of requirements with only minor gaps imperfections that don't
affect core logic
*33 Adequale Process": Reasoning meets basic requirements (60-70%) but has noticeable flaws or missing important
elements
*2 Poor Process" Reasoning has major logical fails address most requirements 50% achievement)
*1 Failed Process""; Written reasoning is fundamentally flawed, missing, or completely off-track (=30% achievement)
228 Example: Plant Growth
"Task"" "Show
this scedling look
months"
"Think Output" "I need to consider how plants grow over time. In months, through photosyathesis, the leaves will expand to capture more sunlight, the stem will elongate support growing plant, and the root system will develop underground to absorb more nutrients."
"Evaluation""
""Process Steps" Identifies photosynthesis as growth mechanism, Considers multiple plant parts
*Domain Knowledge"*; v Correctly applies plant biology principles, 3-month timeframe appropriate
3. *Temporal Logic"; / Sequential growth process described, Cause-effect relationships clear "Completeness"" Major growth aspeds covered, Underground above-ground development
"""casoning prctoniingppoesssooe score" (Comprchensive accurate reasoning process)
Imput
"Original Image"
*Task Instruction" (prompt)
"Dimension"": (dimension)
"Key "ords" (keywords)
"Target Description*: [
"Think Outpul" [think_oulput]
32 Output Format
'reasoning_ wrcptiiiinpppossseee pwwppppsccreeeee score":
"reasoning" 'I. Process Steps Domain Knowledge Check Temporal Logic Validation 4. Completeness Assessment"
Figure 12: Prompt used for evaluating the reasoning process of temporal (RP).
22

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Visual of Temporal
You are professional Al evaluation specialist with expertise in temporal reasoning assessment.
You will be given:
1. "Original Image"" the starting point
2. *GGneratedd Image"; the result after temporal reasoning
3. "Task Instruction". ""; the temporal reasoning task to perform
4. "DDimension*: the knowledge domain sedencshumanity/ommmn )
5. "oeyw" relevant domain concepts and principles for this task
"TTrget Description": expected visual outcomes after temporal ressoning
7. "Target Image" (if available): reference image showing the expected result
Note: Keywords are domain-specific concepts that should be considered applied in the reasoning, Target Description shows what the final visual outcome should look like, helping you assess the visual result aligns with expectations. Target Image is provided, use it the primary reference for evaluation; otherwise, rely the Target Description.
Your Objective:
Evaluate whether the "generated image" matches the target description (and target image available) and demonstrates correct temporal reasoning. Focus comparing visual result with the expected outcomes.
*t Visual Temporal Logic Principles:
"SSqquential Progression" " Visual changes follow natural temporal order order
"Causality Over Time**: Each visual stage logically leads next
"Process Continuity" No impossible visual jumps or missing critical stages
"Time-Sea Stale- Consistency"" Visual changes match the specified time duration
* Domain-Speciin Specific Considerations:
** "Science" Appll scientific principles and natural laws; verily reasoning follows established scientific facts and theories; reject unscientific claims or impossible phenomena
" "Hmmanity*: Consider cultural, historical, and social contexts; ensure reasoning respects cultural norms and historical accuracy; avoid cultural insensitivity or anachronisms
"Coommon Sense**: Use everyday knowledge and practical understanding: verify reasoning aligns with woaldworld experience and logical expectations; reject unrealistic or impractical scenarios
"Logic"*; Follow formal reasoning and mathematical principles ensure logical consistency and mathematical accuracy, reject logical fallacies mathematical errors
x* Evaluation Steps:
1. "Target Match"" Does the generated image match the larget description (and target image available)?
2. "VVisial Changes Analysis"" What has visually changed from original generated image?
3. "DDmmin Knowledge Check" Do visual changes align with keyword-related related principles? Ensure visual reasoning follows http://uuuuuuuloomeeeeeett
domain-specific
principles: reject violations of established domain knowledge 4. "Temporral Logic Validation"" Is the visual progression temporally sound?
*t Evaluation Scale (1 to 5):
""5 Perfect Target Match" Generated image "precisely matches" larget description (and larget image if available) with "flawless temporal logic**; all required temporal changes present and accurate with *"zero gaps errors**
"44 High Quality Match" Generated image achieves 80 90%+ of target requirements with only minor details missing or slightly incorrect; core temporal changes are correct
.*3 Adequate Match" Generated image meets basic requirements (60-70%) but has notable gaps, wrong aspects, or incomplete temporal dhanges
"" Poor Ma Generated image fails most largel requirements (30-50% achievement) with major gaps or incorrect
temporal reasoning
""1 Failed Match" Generated image completely fails match larget shows fundamental temporal logic enrors (<30% achievement)
Example I (Score: 5): Perfect Plant Growth
" "Task"" 'Show what this seedling will look like after 3 months"
"in"ensoon"":
"science"
Keywords" "plant development, pholosynthesis, growth'
"Tageet Description* "leaves expanded and more numerous; stem visibly longer, root system extended underground
"Evaluation".
1. "VVisaa Changes"*, / Leaves significantly expanded, v Stem dearly elongated, Root system visible underground
2. "DDmain Knowvledge"", /f Crowth follows photosynthesis principles perfectly. / 3 3-month timeframe accurate
3. "Temporal Logic": v/ All development stages shown correctly, of Natural growth progression
4. "Completeness"" All major growth aspects visible, v Above and below ground development
Figure 13: Prompt template for evaluating visual-temporal reasoning capabilities (RV), (Continued
in Figure 14)
23

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Visual of Temporal
""ressoning_ visual score"": visual_score":: 5 (Perfect temporal resoning with complete visual representation)
Example (Score: Good Growth
"Task": "Show what this seedling will look like after months"
"Dimension"": "science"
"Keywords": "plant development, photosynthesi, growth"
"larget Description" "leaves expanded and more numerous; stem visibly longer; root system extended underground"
"Evahuation":
1. 'VVisual Changes"": Leaves expanded, / Stem elongated,
system visible 2. "DDmmin Knowledge"*: / Crowth follows photosynthesis principles, 3 3-month timeframe appropriate
"Temporll Logie""; Sequential development stages shown, X Missing intermediate growth phases
4. "Coopletenees"" " v Major grow th visible, X Underground development not represented
"reasoningg visual scare""; 4 (Strong visual progression but incomplete representation)
*** Example (Score: Poor Plant Growth
"Task"" "Show what this seedling will look like after months"
"Dimension"": "science"
"Keywords":: "plant development, photosynthesis, growth"
"argett Description's 'leaves expanded and numerous; visibly longer; system extended underground"
" "Evaluation":
1. *Visual Changes*t: Leaves barely changed, X Stem same length, X No root development
"DDomann Knowledge"*, X Growth doesn't follow photosynthesis principles, X 3-month timeframe ignored
3. "TTmmprrl Logie""; No clear development stages, X Unrealistic growth pattern "Completeness" X Minimal growth visible, Most requirements not met
"rressonig visual score" 2 (Poor temporal reasoning with minimal visual changes)
*2x Example (Score: 1): Failed Plant Growth
"Task""; "Show what this seedling will look like after months"
"Dimension"": "science"
"Keywords" "plant development, photosynthesis, growth"
"Target Description" " 'leaves expanded and more numerous; stem visibly longer; root system extended underground"
"Evaluation":
1. "VVisul Changes""; Plant appears dead/willed, growth visible, X Wrong direction
2. "Domain Knowledge" ! X Completely violates plant biology. Shows impossible outcomes
3. "Temmoral Logie"; No logical progression, X Contradicls natural growth Fundamental mi minudeerstgninng
"Completeness". X No target requirements
""ressoning visuual_score"" score"": (Complete failure temporal reasoning)
Input
"Immge Original Image"* (the starting point)
"Image Generated Image" (the result after temporal reasoning)
"Image 3: Target Image"" (if available, the reference showing expected result)
Task Instruction": (prompt)
"Dimension":: (dimension)
"x [keywords]
Target Description". Slarget_descripee
Output Format
II
"reasoning visual_scords score": X,
"reasoning" "1. Targel Match 2. Visual Changes Analysis 3. Domain Knowledge Check 4, Temporal Logic Validation"
|I
Figure 14: Prompt template for evaluating visual-temporal reasoning capabilities (RV). (Continued from Figure 13)
24

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Process of Causal
You professional Al evaluation specialist with expertise causel reasoning assessment.
You will be given:
1. "OOigginl Image"; the starting point
2. TTask Instruction" the causal reasoning task to perform
3. "Dimmensin"" knowledge domain crtenre/hmannity/oom )
4. "Keywonds" relevant domain concepts and principles for this task
""aagget Description" th expected visual outcomes after cousal reasoning
6. "TThink Output": the reasoning text generated by the model
Your Objective:
Evaluate ONLY the "actual text content" provided in the "Think Output" section. You must analyze the reasoning quality based solely what written there. Do NOT generate evalluate your own reasening only the provided text.
Cf the Think Output empty, contains only placeholder text, says "No think output available", you MUST give score of and explain that no actual reasoning was provided. Do NOT create your own reasoning to evaluate.
Note: Keywords domain-spec concepts that should be considered applied in the reasoning. Target Description shows what the final visual outeome should look like, helping you assess if the reasoning process is heading in the right direction.
13 Causal Logic Principles:
Cfuse-Fffect Relationships":
connection between cause observed effect
Mechanism Consistency" " Intermediate steps follow logical causal chains
"ntervention L changes produce expected outcomes
"Causal Completeness" necessary causal factors represented
# Domain Specific Considerations:
"Science" Apply scientific principles and natural laws; verify that reasoning follows established scientific facts and theories; phenomena
zeject unscientific claims
"Humanity"* Consider cultural, historical, and social contexts; ensure reasoning respects cultural norms and historical acauracy avoid cultural insensitivity or anachronisms
"Common Sens"": Use everyday knowledge and practical understanding: verify reasoning aligns with real world experience and logical expectations; reject unrealistic or impractical scenarios
"Logic**: Follow formal reasoning and mathematical principles, logical consistency and mathematical accuracy; reject logical fallacies or mathematical errors
aa Evaluation Steps:
""Identify Causal Chain"": What cause-effect sequence demonstrated?
2. **Domain Knowledge Check*: Does causation follow relladd-rllteed principles and target description? Ensure reasoning
principles; reject violations established knowledge 3. *Mechanism Validation"*, Are causal steps logically connected and complete?
follows domain specific
4. "*Effect Assessment". Do observed effects expected causal outcomes?
# Evaluation Scale to
"5 Perfect Causal Logic"h. All cause-effect relationships follow domain principles flawlesdly with complete mastery of requirements
"44 High Quality Causal Logic""; Causal reasoning achieves 80- 80-90%+ of requirements with only minor causal inconsistencies that
don't affect core logic
"" Adequate Causal Logie" Causal reasoning meets basic requirements (60-7%%) but has noticeable flaws or missing important
elements
Poor Causal Logie"". Causal reasoning has major causal errors or fails address most requirements (30-50% achievement) Causal Logic"" Causal reasoning fundamentally flawed, missing or violates basic causal principles (+30%. achievement)
Example: Potato Oxidation Prevention
* "Tas"* "Apply lemon juice prevent these potatoes from browning"
DDieensonn": "science"
" "Keywords"t: 'citric acid, enzymatic browning, oxidation prevention"
Target Description" "cut potatoes remain white/pale after lemon juice application"
" "Fvaluation"*
1. *Cauuaa Chain " Lemon juice applied to potato surfaces, v Potatoes remain white/pale
2. "Domain Knowledge": Citric acid prevents browning v Application method appropriate
3. "Mechanism Validation* y Chemical prevention process shown, X Some areas missed during application
4. "Effect Assessment* /f Most potato pieces remain unbrowned, X One piece shows slight browning proponsinnnnpoosssseo pwwpppoooooooeeeooeeeeee score" 4 (Sound causal reasoning with minor application gaps)
"reassoning
3? Imput
* "Origiinl Image"
Task Instruction (prompt]
** *DDmeension": " (dimension)
"* "ordssrss"" (keywords)
** "Tagge Description" [target description)
Think Output"*: (hink_output)
*> Output Format
II
"reasoning_i prcpeeppoosssss_ prw.osssscree..____ppppppp_ X,
"reasoning" '1. Causal Chain 2. Domain Knowledge Check 3. Mechanism Validation 4. Effect Assessmen!"
B
Figure 15: Prompt template for evaluating process of causal reasoning capabilities (RP).
25

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Visual Causal
You a professional ovaluation specialist with expertise causal reasoning assessment.
You will be given:
"Origginal Image"" the starting point
"Generated Image""; the result after causal reesoning
Task Instruction 1 the causal reasoning lask perform
4. "Dimension": the knowledge domain socence/humaniillloomee l
"Keywords" relevant domain concepts principles for this lask
6. Targgt Description" " expected visual outcomes after causal reasoning
Target Image" available): reference image showing expected result
Note: Keywords are dopain-specific concepts that should be considered or applied in the reasoning, Target Description shows what the final visual outcome should look like, helping you the visual result aligns with expeclations. If Target Image is provided, use it as the primary reference for evaluation; otherwise, rely on the Target Description.
Your Ohjective:
Evaluate whether *'visual changes" in the generated image correctly demonstrate causal reasoning following domain principles. Focus comparing the visual result with the expected outcomes.
# Visual Cousal Logic Principles:
"Cause-Elfect Relationships": Visual changes show clear cause-effect connections
"Mechanism Consistency **; Visual intermediate steps follow logical causal chains
"Iterventtion Logic"; Visual applied changes produce expected outcomes
*Causal Completeness"" Visual representation includes necessary causal factors
#8 Domain Speciff Considerations:
"Science"": Apply scientific principles and natural laws; verify that reasoning follows established scientific facts and theories; reject unscientific claims impassible phenomena
"Humanity** Consider cultural, historical, and social contexts; ensure reasoning respects cultural norms and historical accuracy; avoid
insensitivity or anachronisms
"Common Sense""; Use everyday knowledge and practical understanding: verify reasoning aligns with real woal-world experience and logical expectations: reject unreolistic impractical scenarios
"Logic": Follow formal reasoning and mathematical principles; logical consistency and mathematical accuracy; reject logical fallacies or mathematical errors
## Evaluation Steps:
"Target Match": Does generated image match target description (and target image available)?
2. *Visual Changes Analysis* What causal effects are visually apparent?
*Domain Knowledge Check visual changes align with key principles? Ensure reasoning follows
domain speciifi scientific/cultural// wtw.u//wwrra//ooommmeeeeeaeee
principles; reject violations of established domain knowledge cousal steps logically connected complete?
Mechhaissm Validation"*:
#9 Evaluation Scale (1 to 5):
"5 Perfect Target Match" Generated image perfectly matches target description target image if available) with correct causal logic
High Quality M image achieves 80-90%+ larget requirements with only details missing
slightly incorrect; core causal changes are correct
**3 Adequate Match" Generated image basic requirements (60-70%) but has notable gaps, wrong aspects,
incomplete causal changes
Match" Generated image reasoning
target requirements (30-50% achievement) major gaps incoerect
Failed Match" Generated image completely fails to match target or shows fundamental causal logic errozs (430%
achievement)
152 Example (Score: 5): Perfect Potato Prevention
"Task"" 'Aply lemon juice to prevent these cut potatoes from browning"
"Dimension": "science"
"Keywords"" "citric acid, enzymatic browning, oxidation prevention"
"Target Description": "cut potatoes remain white/pale lemon juice application"
Figure 16: Prompt template for evaluating visual causal reasoning capabilities (RV). (Continued in Figure 17)
26

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Visual of Causal
"Evaluation":
1. "TTarget Match"": v All potatoes remain white/pale, vf emon juice clearly applied
2. "Visual Changes"": Lemon juice
potato surfaces, Potatoes maintain original 3. *DDomain Knowledge**, /f Citric acid prevention shown correctly, v Application method appropriate
4. "Mechanism Validation"", / Chemical prevention process visible, V Completr coverage achieved
"rresoning, visual score":: (Perfect causal reasoning with complete prevention)
## Example 2 (Score: 3); Adequate Polalo Prevention
"Takk** AAply lemon juice prevent these potatoes from browning"
"Dimension"" "scienae"
" "Keywords"" acid, enzymatic browning, oxidation prevention"
"Target Description": "cut potatoes remain whits/pale after lemon juice application"
"Evaluation":
1. "Taaget Match": X Some potatoes show browning, Incomplete prevention
2. VVVisual Changes"", Lemon juice partially applied, Some missed
3. "Domain Knowledge",. Basic citric acid concept shown, X Application incomplete
4. "MMchhanim Validation":; X Chemical prevention partially failed, X Coverage gaps
""remoning score (Adequate reasoning partial prevention)
Example (Score: 1): "Task"" 'Apply lemon juice prevent these cut polatoes from browning"
Prevention
"Dimension"" "science"
"Keywards"": "citric acid, enzymatic browning, exidation prevention"
"TTrget Description": "cut potatoes remain whito/pale after lemon juice application"
"Evaluation"*:
'Target "atch":: All potatoes heavily browned, prevention visible
2. "Visual Changes"", X No lemon juice visible. Potatoes completely exidized
3. ".DDmmin Knowledge"*
citric acid application shown, Wrong approach 4. "Mechanism Validation""; X No chemical prevention, X Complete failure
""reasoning_ visuel_ (Complete failure of causal reasoning)
Input
"Image 1: Original Image" (the starting point)
"Image Generated Image" (the result after causel reasoning)
""Image 3: Target Image** (If availab,,,,, the reference showing expected result)
**Task Instruction (prompt)
"DDimension*. (dimension)
"Keywerds"": (keywards)
*TTagee Description": (target description)
#2 Output Format
||
"reasoning_ visual score": X,
"reasoning. "1. Target Match 2. Visual Changes Analysis 3. Domain Knowledge Check 4. Mechanisa Validation
II
Figure 17: Prompt template for evaluating visual causal reasoning capabilities (RV). (Continued from Figure 16)
27

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Aligmment
You are a professional Al Avaaluaaion specialist with expertise in causal reasoning assessment.
You will be given:
You professional Al evaluatioo specialist focusing on process-visual reasoning alignment assessment.
You will be given:
"Original Image""; the starting point
"Generated Image"": the reasoning result
*Task Instruction" what reasoning should be performed
*TThinnk Output" reasoning process text generated by model
Your Objective:
Evaluate whether the "rressonigg process text** and the visuel reasoning result" are aligned and consistent with each other. Focus on whether the model thought and what the model visually produced match.
#2 Alignment Evaluation Criteria:
*Process Visual Consistency":: Do the written reasoning steps match the visual changes?
"Conclusion Coherena"* Do text conclusions align with visual outcomes?
"Step-by-Slep Alignmen!*: Does each reasoning slep in correspond visual evidence?
"Logical Consistency"": "* Are there contradictions between thought process and visual result?
42 Domaln-Sperific Considerations:
"Science"", Apply scientific principles and natural laws; verify that reasoning follows established scientific facts and the ories; reject unscientific claims or impossihle phenomena
"Humanity"" Consider cultural, historical, and social contexts; ensure reasoning respects cultural norms and historical
accuracy, avoid cultural insensitivity anachronisms
"Common Sense"" Use everyday knowledge and practical understanding: verify reasoning aligns with real-word world experience
logical expectations; reject unrealistic impractical scenarios
"Logie""; Follow formal reasoning and mathematical principles; ensure logical consistency and mathematical accuracy; reject logical fallacies mathematical errors
Key Questions:
1. "Does the visual result reflect the written reasoning?** Are the visual changes consistent with what was described in the
think output?
2. "Are the conclusions sligned?* Do both process and visual reasoning reach the same conclusions?
3. **Is the reasoning coherent?" Are there contradictions between what was thought and what was visually produced?
"IIs the task prompt correctly understood?" Do both process text and visual result demonstrate correct understanding of what the task asking for?
42 Evaluation Scale (1 to 5):
**5 Perfect Alignment**: Process text and visual result are "'completely consistent" and mutually supporting with "*xero contradictions** " all process claims match visual evidence exactly; AND both correctly understand and implement the task
prompt
**4 High Quality Alignment""; Process and visual achieve 80-90%+ alignment with only minor inconsistencies that don't affect core reasoning: AND both generally follow the task prompt correctly
**3 Adequate Alignment", Some alignment present (60-70%) but clear discrepances between process and visual reasoning:
notable inconsistenciis exist; OR good internal alignment but significant misunderstanding of task prompt
.22 Poor Aligament Minimal alignment (30-50%) with major contradictions between written process and visual result; significant mismatches; OR both process and visual fundamentally misunderstand the prompt result contradictory or completely unrelated (430% alignment); complete
*1 Alignment" Process text
failure to understand task prompt
Figure 18: Prompt template for evaluating reasoning alignment capabilities (Align.). (Continued in Figure 19)
28

---

Published as a conference paper at ICLR 2026
Prompt Reasoning Alignment
ff CRITICAL ALIGNMENT CONSTRAINE:
AAligmmen score cannot exceed visual reasoning score by more than point."
-B visual reasoning alignment
most 2
If visual reasoming 2, alignment can at most
If visual reasoning alignment
If visual reasoning 4-5, alignment can be 4-5
This ensures logical consisteney: cannot have high alignment with poor visual reasoning.
*# Reasoning Steps:
"Extract Process Claims"; What does the think output claim will happen or should be done?
"Identify Visual Evidence*: What changes actually visible in the generated image?
"Compare Allgnment**: Do the process claims match the visual evidence?
"Assess Consistency"*, Are there any contradictions between thought and visual result?
"Evaluate Prompt Understanding" *. Do both the process text and visual result correctly understand and implement the prompt requirements?
6. "Domain Knowledge Check" Do both process and visual reasoning follow domain-specific
scientific/cultuaal eommoneessclliiiaa principles?
alignment respects established domain knowledge reject
violations of domain principles
"Apply Alignment Constraint", Ensure alignment score does visual reasoning quality by more than point
<F Input
"Image Original Image" (the starting point)
"Image 2: Generated Image* (the reasoning result)
*TTak Instruction th: [prompt]
""Tiink Output" [think output)
## Output Format
"ressoning, alignment score':
"reasoning". "1. Process Claims 2. Visual Evidence 3. Alignment Comparison 4, Consistency Assesmmnnt Prompt Understanding Domain Knowledge Check Alignment Constraint"
Figure 19: Prompt template for evaluating reasoning alignment capabilities (Align.). (Continued from Figure 18)
29

---

Published as a conference paper at ICLR 2026
Prompt for Reasoning Visual Consistency
professional visual evaluation specialist focusing image con sistency
given:
""Oiiginal Image""; the starting point
2.*"eerraeed Image"": the result after reasoning/editing
3. "Task Instruction" the reasoning editing lask performed
Your Objective:
Evaluate whether "hon target elements" the generated image remain ""visually consistent" with the original image. Focus exclusively elements that should NOT have changed according the task instruction.
Consistency Evaluation Guidelines:
Preserve:
"Background Elements** 7 Sccnery, environment, setting details not mentioned in task
* "Unrelated Objects**; Items involved
ressoning/edlting process
"Structural Elements"*; Basic composition, layout, perspective (unless task requires change)
"Identity Preservation". People, animals, or objects should maintain their core identity
Styll Consistency" t Overall visual style, lighting conditions, color palette
Elements That May Change (Task-Dependent):
"Target Objects" Items explicitly mentioned in the task instruction
"Direct Consequences" Changes that logically result from the intended transformation
*Process Effects" Visual effects directly caused by the reasoning process
a2 Evaluation Scale (1 5):
Perfect Consistency**: non-target elements remain "*visually identical** to original with "*xero unintended changes"" perfect preservation of all non-instrucied elements
Minor Inconsistency"; "Minimal unintended changes" that are barely noticeable and don't affect coherence: only very small discrepancies
*33 Noticeable inconsistency":. "Clear unintended changes" in background or unrelated elements that affect coherence;
notable incensislencies exist
*22 Significant Inconsistency" "Multiple unintended changes" that significantly compromise visual cuherence: major
inconsistencies
*1 Severe Inconsistency" "Major unintended alterations" that make image appear largely different; tundamental
consistency breakdown
Reasoning Steps:
"Identify Target Flements"*: What elements should change according the task?
**Isolate Preserve Elements"*, What elements should remain unchanged?
"*Comaaa Preservation "5: Are the preserve elements visually consistent with original?
4. Assess Impact": How do any inconsistencies affect overall visual coherence?
33 Imput
"Image Original Image"* (the starting point)
"Image Generated Image" (the result after reasoning/ddttin)
"Task Instruction" (prompt]
Output Format
visuual consistency. score": X. "reasoning": Target Elements
Preservation Comparison Impact Impact
Figure 20: Prompt template for evaluating visual consistency (VC.). (Continued from Figure 20)
30

---

Published as a conference paper at ICLR 2026
Prompt for Image Quality
You are a profensional image quality asessoor specializing in Al Al-generated content evaluation.
You will be given:
".""enerated Image": Al-generated image evaluate
Objective:
Evaluate the "perceptual quality" of the Al-generated image, focusing technical excellener, visual coherence, and absence generation artifacts.
Quality Assessment Dimensions:
*** Structural Coherence
" Snaatomy/Coometry" Correct proportions, realistic structures, proper object shapes
"SSpatial Relationships Logical positioning, appropriate scale relationships
"Compositional Logic"": Coherent layout, proper perspective
Fidelity
"Texture Quality** Realistic surface textures, appropriate material appearance
"DDeaal Clarity" Sharp important details, appropriate level detail throughout
"Coloo Accuracy":: Natural color distribution, proper lightingshhadw
283 Generation Artifacts
"Duplication Issues" Repealed elements, phantom objects, merged features
"Blenndin Problems"* Unnatural transitions, ghosting effects, edge artifacts
DDistortion Erroes" Warped features, impossible geometries, scale inconsistencies
*** Overall N Naturalness
"PPotorealismm" Does image
believable?
"Coherent Style"": Consistent visual style throughout the image
Professional Quality": Would this pass as high-quality content?
Evaluation Scale (1 5):
"5 Excellent Quality":: "Professional-grad image** with "'no noticeable artifacts flaws";; perfect technical excellence
photorealistic quality
"44 Quality " "High-quality image"
impression; minimal quality
isaues
""3 Acceptable Quality*:; "Decent image"* some noticeable flaws usable; clear quality problems
"2 Poor Quality"*: "Multiple significant flaws that detract from image usability; major quality problems
Very Poor Quality":; "Major structural problems"" severe artifacts, unusable quality: fundamental quality breakdown
Quality Checklist:
each dimension, mark (satisfactory) (problematic):
Structural coherence: //X
Viisual fidelity: VIX
Artifact- free: JIX
naturalness:
Reasoning Steps:
"Sttrucural "n Assess geometric and anatomical correctness
*"Fidelity Evaluation Check texture, detail, and color quality
**Artifact Detoction":: Identify any generation artifacts distortions
"Naturllnsss Assesment**, Evaluate overall believability and professional quality
#t Input "Generated Image"
Output Format
"image quality score":
"reasoning": '1. Structural Analysis Fidelity Evaluation 3. Artifact Detection 4. Naturalness Assessment"
Figure 21: Prompt template for evaluating image quality (IQ.). (Continued from Figure 21)
31

---

Published as a conference paper at ICLR 2026
World Model for Embodied Task
SYSTEM PROMPT = You are robotics trajectory planner with visuali risualization zatien capabilities. when given robotics scene, you must:
1. Generate a trajectory visualization image:
Overlay 10 waypoint markers the input scene
Style: Blue circles with white cutlines connecting trajectery lines
Labels: 'trajl', 'traj2',
'traj10'
Reference the example image (Image 2) for exact visualization style
2. Output pixel coordinates based on your visuali lization:
Format: I [xl,, yll, [x2, y21,
|x10, y1o1l
Coordinate system: (0, 0) at tep-left corner
Constraints:
Start from current end-effector position
End at task completion position
Generate smooth, collision-fre free waypeints
All coordinates must be within image boundaries
Format your output as: <think>
[Your analysis and planning process]
</think>
Final Answer: [Trajectory coordinates: [[x1, yll, [x2, y21, .... x10, y
32

---

Published as a conference paper at ICLR 2026
World Model for Physical Task
SYSTEM_PROMEE You are a physics simulation with image generation capabilities. Given a physical scenario, you must:
1. Analyze the initial scene (Image
What objects are present? (cars, balls, liquids, blocks, pulleys, etc.)
What are their initial positions and states?
what forces motions will be applied?
2. Use Image 2 (if provided):
It may show additional context of the scene
3. Generate physies simulation result image:
Simulate what happens when physics is applied and generate the
resulting scene:
Apply gravity, ummentum, friction, collision dynamics
Simulate the complete physical precess (cars moving and cclliding
balls falling, liquids flowing, pulleys rotating)
Generate an image showing the final state or outcome after physics simulation
The simulated image should naturally show where objects end what gets affected, and what the result is
Format your output as:
<think>
1. Initial state: [describe the setup]
2. Generate the physics simulation image
3. Analyze the generated image: [what does the simulaticn show?]
4. Determine answer: [based on the generated image, what is the result?]
</think>
Final Answer:
[exact answer format requested questica)""
33

---

Published as a conference paper at ICLR 2026
Logic Math
WYSTEM_PROMPE Youu are a helpful assistant. You need to think about the given prompt/question and any hints provided, then generate USEFUL VISUAL AIDS based the hints during your thinking process, and finally answer the question based your analysis and the generated images.
IMPORTANT REQUIREMENTS:
1. You MUST generate images that USEFUL VISJAL AIDS for solving the problem (e.g., with auxiliary lines, labels, annotations, constructions that help solve the problem)
2. Do NOT generate images that merely replicate the given figure without adding helpful information
3. You MUST provide final answer after your thinking precess
Enclose your thinking process within <think> </think> tags, generate relevant images during thinking, then provide your final answer.
Format your output as: <think>
Step 1: Analyze what auxiliary constructions would help sclve this problem.
Step Generate
visual aid with those constructions. [generate USEFUL images with helpful additions like auxiliary lines labels, or constructions]
Step 3: OBSERVE the generated image carefully and use the visual informaticn
perform your reasening.
Step 4: Based on what you see the generated image, work through the soluticn.
</think>
Final Answer: [your answer based the generated images and analysis]
REMEMBER:
The generated images must be USEFUL VISUAL AIDS that add value beyend the original figure.'"
34

---

Published as a conference paper at ICLR 2026
Visual Perception for Jigsaw
SYSTEM PROMPT You are a helpful assistant solving visual jigsaw puzzles. You need to analyze the puzzle image with a gray pox covering part of it, then generate completed image by filling in the missing area, and finally select the correct option.
SYSTEM_PROMPT
IMPORTANT REQUIREMENTS:
1. You MUST first generate completed image by filling in the missing
covered by the gray box 2. Compare your generated full image with the original puzzle to validate consistency
3. Use the gencrated complete image to determine which optien (A, B C, D) correctly fills the missing area
4. Do NOT answer before generating the completed image
Enclose your thinking process within <think> </think> tags, generate the completed image during thinking, then provide your final answer.
Format your output as: <think>
Step 1: Analyze what is visible in the puzzle and what patterns/ objects might be the missing area.
Step 2: Generate a completed image by filling in the missing top part of the puzzle.
[Generate the full completed image]
Step 3: Carefully observe your generated complete image and compare with the original puzzle to ensure consistency,
Step 4: Compare your generated complete image with each option (A,
B, C, D) te find which one matches the missi ng area in your generated image.
</think>
Final Answer: IA/B/C/D)""
35

---

Published as a conference paper at ICLR 2026
Visual Perception for Multi-view Reasoning
SYSTEM_PROMPT
SYSTEM PROMPT You are a helpful assistant analyzing multi- view images to determine camera novement direction. Given two images taken from different canera positions around the same scene, you need reason about the spatial relationships and determine the canera rotation direction.
IMPORTANT REQUIREMENTS:
1. You MUST first generate wider-angle image taken from farther away that includes ALL objects visible in both Image and Image
2. This generated image should be like stepping back and using a wider lens - showing more of the scene in one frame
3. Use this wider-angle view to understand the spatial relationship between the two camera positions
4. Determine if the camera rotated countercloakwise (left) or clockwise (right) from Image 1 to Image 2
5. Do NOT answer before generating the wi wider-angle image
Think it like this: If you step back from the scene and take photo with a wider-angle lens, you can see all the objects from both viewpeints in one image.
Enclose your thinking process within <think> </think> tags, generate the wider-angle image during thinking, then provide your final answer.
Format your output as: <think>
Step 1: Identify all objects visible in Image 1 and Image 2.
Step 2: Generate a wider-angle image from farther away that includes all these objects.
[Generate the wider-angle image showing the complete scene] Step 3: Use this wider view to understand where the two cameras were positicned.
Step 4: Determine the rotation direction: counterclockwise (left) clockwise (right)?
</think>
Final Answer: (left/right)""
36


Reviews:
[{"review_id": "oba71l1NoD", "reviewer": "Reviewer_Apxs", "summary": "The paper introduces ROVER, the first human-annotated benchmark explicitly designed to evaluate reciprocal cross-modal reasoning in Unified Multimodal Models (UMMs). It addresses the fundamental limitation of existing benchmarks, which treat understanding and generation abilities in isolation, failing to assess how one modality can guide, verify, or refine outputs in the other. 17 SOTA UMMs have been evaluated on the visual generation and text generation settings.", "questions": "Please see the weakness above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well structured and presented overall, with a helpful project page.\n- It addresses an important gap in UMMs by benchmarking and evaluating reciprocal cross-modal reasoning.", "weaknesses": "- Table 1 should include comparisons across more aspects. Additional explanations are needed in both the text and the table caption: benchmark dataset scale, whether it is for VG/TG/both, and clarifications on the multi-dimensional and hybrid evaluations and the types.\n- This work's emphasis on intermediate reasoning as a core signal for multimodal reasoning distinguishes it from existing benchmarks. However, the data curation process for these progressive reasoning steps is under-specified, especially for the TG setup. The paper should clarify exactly what intermediate data is curated for various sub-tasks, dataset statistics, and how it is used for evaluation.\n- The current evaluation depends only on GPT-based judgment. Introducing objective, automatically computed metrics would improve the reliability of the fine-grained reasoning evaluation. \n- Would it also be beneficial to include the text reasoning chain for the TG task? Clarification is needed on how the progressive visual reasoning steps are validated as active reasoning components rather than decorative elements, as claimed in line 246.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4oWrq9Ykvc", "reviewer": "Reviewer_FBtf", "summary": "This paper introduces ROVER, a new benchmark designed to evaluate reciprocal cross-modal reasoning in unified multimodal models (UMMs), i.e., the ability to use one modality (text or image) to guide reasoning and generation in the other. Existing evaluations tend to isolate modalities, emphasizing either textual or visual reasoning in isolation, which fails to capture the intended integration of modern UMMs.\n\nROVER fills this gap through over 1,200 human-annotated tasks grounded in 2,048 images, spanning two complementary settings: (1) verbally-augmented reasoning for visual generation, where structured verbal reasoning guides faithful image synthesis, and (2) visually-augmented reasoning for verbal generation, where models generate intermediate visualizations to support their reasoning.\n\nThe authors evaluate 17 state-of-the-art UMMs and find that cross-modal reasoning ability correlates with visual generation performance, especially for interleaved text–image tasks. However, most models remain weak in visually-augmented reasoning, particularly in logical reasoning scenarios. \n\nOverall, the work is a good first step towards analyzing cross model reasoning abilities of current UMMs and avenues of improvement, but still requires more work to solidify its usability and interpretability.", "questions": "I have added my questions as part of weakness itself, would urge the authors to respond to them. In its current state the work does not merit publication, however if the authors make the necessary clarifications and substantiate their claims, the benchmark would be more useful and i can consider bumping my score.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 4, "strengths": "1. Importance of cross model reasoning in UMM and problem formulation in two complementary settings of verbally-augmented reasoning for visual generation (ROVER-IG) and visually-augmented reasoning for verbal generation (ROVER-TG) is interesting, useful and novel.\n2. Careful dataset design into top level domains and subtasks for both ROVER-IG and ROVER-TG\n3. Detailed metrics that aim to provide a holisitic understanding of the model performance in either settings.\n4. Interesting analysis like coherence between reasoning substasks.", "weaknesses": "The paper gives a good shot to cover a novel perspective but falls short in these following areas:\n\n1. Stretch / Over claims: \na) \"Pg 5 section 4.1 (last para) the authors claim that gaps in reasoning process and alignment is the fundamental driver of diminished visual generation performance\" but as seen for table 2, if you look at natural science or logic for instance for both closed and open source model, similar RP and align scores show great variability in RV scores. \nb) \"Pg 7 section 4.2 Models demonstrate superior interleaved reasoning performance on physical world and visual perception tasks compared to logical reasoning challenges\" is not supported in table 3, model perform similarly for the best for visual perception only and they have similar low performance in logic and physical world domains.\n\n2. Clarifications\na). It is difficult to infer anything from the % reported in the paper, none of them mention if its absolute, or relative and relative with respect to what ?\nb) visual generation performance on pg 5 last paragraph is vague. from reading context, i can map it to RV but would urge the authors to make explicit connections between numbers and metrics, especially when they define them\nc) Section 4.3 Cross-modal Reasoning matters for UMMs: Could not follow through this analysis, CLIP-1 and edit world are introduced out of the blue without prior context. Fig on pg 9 top right has the corresponding details but is not reference in text and the figure itself is unclear, with some bars having +ve/-ve value and being of different lengths + no caption. Could not make sense of this at all\n\n3. Judge reliability evaluation\na) Human correlation of RV one of the important metrics for IG is low\nb) Only IG metrics undergo reliability evaluation what about TG metrics which are also llm judges ?\nc) The models used for judge calibration are either closed source models or the strongest open-source model. This could potentially add bias in score calibration, having a weaker open source model being part of calibration can ensure that the entire spectrum of scores is callibrated.\n\n4. Missing details\na) Fig 4, it would be nice to see the reasoning generated by the models in addition to input text and generated image, to better analyze reasoning alignment.\nb) Table 2 does not clearly indicate which models are interleaved vs single turn, image editing only vs UMM but uses these terminologies in the analysis section when table 2 is referenced. Appendix does provide some insight to them but still models like Show-o2, Blip3o-8b, Janus-Pro-7b, etc have not been classified, making it difficult to relate to the claims in the paper\n\n5. Overall the paper writing needs to improve, better references to figures and tables, improved captions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lnh7LATe31", "reviewer": "Reviewer_icDF", "summary": "This paper proposes ROVER, a benchmark with over 1,200 tasks and 2,048 images for reciprocal cross-modal reasoning. ROVER has two parts: ROVER-IG (language guiding image generation) and ROVER-TG (vision aiding text generation). They tested 17 UMMs with a VLM judge plus expert checks, finding cross-modal reasoning ties to visual generation, but models struggle with vision-aided logic tasks.", "questions": "1. How does the up-to-date Gemini-2.5-pro perform on this benchmark?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is generally well-written and easy to follow, with clearly illustrated figures.\n2. ROVER covers a wide range of both language-reasoning tasks and visual-reasoning tasks, and uses a comprehensive evaluation method (VLM + expert validation) to ensure reliability.\n3. The authors evaluate 17 unified multimodal models and provide insightful findings.", "weaknesses": "1. The benchmark heavily depends on a \"VLM-as-a-judge\" for scoring complex reasoning qualities. The paper's own user study (Figure 8) shows that while correlation is good, there are noticeable discrepancies, especially for reasoning-related metrics. This introduces a potential bias, where the benchmark might favor models whose outputs align with the judging VLM's own reasoning patterns.\n2. As listed in Table 3, language-only models often match or exceed the performance of unified models on reasoning tasks, questioning whether the current task design truly requires cross-modal reasoning for optimal results (\"thinking with images\").​", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HHvua8memy", "reviewer": "Reviewer_Bjyh", "summary": "This paper introduces ROVER, a benchmark designed to evaluate the reciprocal cross-modal reasoning capabilities of Unified Multimodal Models (UMMs). The authors argue that existing benchmarks fail to assess how models use one modality to guide or verify outputs in another, instead testing text and image abilities in isolation. ROVER addresses this gap with over 1,200 tasks that require integrated reasoning across modalities, focusing on verbally-augmented reasoning for visual generation and visually-augmented reasoning for verbal generation. By testing 17 state-of-the-art UMMs, the study finds that cross-modal reasoning skills strongly correlate with visual generation performance. However, it also reveals that current models are severely limited in visually-augmented reasoning, showing particular weakness in logical tasks compared to perception and physical modeling.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The paper introduces the first benchmark that requires generating both visual and textual content for joint visual and textual reasoning, effectively unifying the two modalities. The authors conduct extensive experiments demonstrating that incorporating multimodal generation improves performance compared to text-only generation during evaluation. The finding that models with stronger image–text interleaving capabilities outperform image-editing models is also noteworthy. Overall, the paper provides clear evidence that text generation supports image generation, and image generation, in turn, enhances textual reasoning.", "weaknesses": "1. The paper's evaluation methodology relies heavily on the quality of generated images, particularly for Reasoning Visual (RV), which requires generating coherent images to facilitate correct reasoning. However, the use of a VLM as a judge is questionable. Figure 8 reveals a significantly low correlation (0.63) and a high MAE of nearly 1.0 between GPT's evaluations and human judgments. Assuming the four human evaluators provide a more reliable gold standard, this discrepancy undermines the validity of using VLMs to assess RV quality. This concern is amplified by prior work demonstrating that even state-of-the-art VLMs struggle with fundamental spatial and temporal reasoning.\n2. In addition to the questionable reliability of the VLM-as-a-judge paradigm, the paper fails to address the financial costs associated with using the GPT for evaluation, a notable omission given the large volume of generated images involved.\n3. While the authors present Figure 7b to show the correlation between different task types, this finding is largely unsurprising due to the semantic definitions of the tasks. Crucially, it remains unclear how this analysis provides actionable insights or how it might guide the development of future models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)