Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking
Authors: Shengqiong Wu
Bobo Li
Xinkai Wang
Xiangtai Li
Lei Cui
Furu Wei
Shuicheng YAN
Hao Fei
Tat-Seng Chua
Venue: ICLR 2026 Poster (2026)
Keywords: unified understanding and generation
multimodal reasoning
multimodal generation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce the interleaved Analyzing–Drafting problem-solving loop for synergizing learning between understanding and generation.

Abstract: Unified Vision–Language Models (UVLMs) aim to advance multimodal learning by supporting both understanding and generation within a single framework. However, existing approaches largely focus on architectural unification while overlooking the need for explicit interaction between the two capabilities during task solving. As a result, current models treat understanding and generation as parallel skills rather than synergistic processes. To achieve real synergy, we introduce the interleaved Analyzing–Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations. By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy. To train this mechanism, we design a two-stage strategy: supervised learning on interleaved thought data to initialize alternation, followed by reinforcement learning to promote adaptive and autonomous control. Extensive experiments demonstrate that AD-Loop consistently improves performance across standard benchmarks for both understanding and generation, with strong transferability to various UVLMs architectures. Visual analyses further validate the effectiveness of implicit visual thoughts. These results highlight AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation.

Published as a conference paper at ICLR 2026
SYNERGIZING UNDERSTANDING AND GENERATION WITH INTERLEAVED ANALYZING-DRAFTING THINK-
ING
Shengqiong Wu Bobo Li', Xinkai Wang" Xiangtai Li,, Lei Cui', Furu Wei', Shuicheng Yan', Hao Fei's, Tat-seng Chua National University of Singapore, UUiiversity of Oxford, 3 Nanyang Technological University, *iicrosoft Research
ABSTRACT
Unified Vision-Language Models (UVLMs) aim to advance multimodal learn- ing by supporting both understanding and generation within single framework. However, existing approaches largely focus on architectural unification while overlooking the need for explicit interaction between the two capabilities during task solving. As a result, current models treat understanding and generation as parallel skills rather than synergistic processes. To achieve real synergy, we in- troduce the interleaved analyzing-Drafiune new think paradigm that dynamically alternates between analytic and drafting op- erations. By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy. To train this mechanism, we design two-stage strategy: supervised learning on interleaved thought data to initialize alternation, followed by rein- forcement learning to promote adaptive and autonomous control. Extensive periments demonstrate that AD-L-oop consistently improves performance across standard benchmarks for both understanding and generation, with strong transfer- ability to various UVLMs architectures. Visual analyses further validate the effec- tiveness implicit visual thoughts. These results highlight AD-Loop as princi- pled and broadly applicable strategy for synergizing comprehension and creation. The project page is ADD-oopio.
problem-solvie loop (AD-Loop), a
INTRODUCTION
Unified Vision-Languns Models (Wang et al. 2024; Wu et al. 2025b; Deng et al.) 2025 Wu
al.) 2025a Xie et al. (025) can handle both understanding and generation tasks, attracting significant research attention as they hold the potential to move beyond task-specific solutions toward general multimodal intelligence. Recent advances in multimodal large language models (MLLMs) (Luu et al. 2024a Bai et al. 2023 Dai et al. 2023) and diffusion- based generative trans- formers (DrTs) (Podell al. [023] Yang al.] 2024 All All 2025) have substantially improved per- formance in both multimodal comprehension and content creation. Building upon these advances, early efforts (Shen et al., 2023) Wu et al., (024c) have attempted to integrate existing understand- ing and generation models within single framework, enabling both capabilities simultaneously. However, such straightforward integration merely co-locates the two paradigms without enabling genuine interaction mutual reinforcement. In essence, understanding and generation are inher- ently complementary (Paivio et al. 2006) Ellamil et al.] 2012). Robust understanding provides the semantic foundation for faithful generation, while successful generation results serve as tangible evidence of visual comprehension. Therefore, an effective unified model should not only combine the capabilities but also establish mutually reinforcing loop.
'Email: swu u.nus.edu
1 Work done during a remote internship at NUS.
1 "ooresponning author: Hao Fei; Email: haofei7419@ @mmiil.commmmmmmmmmmmmm

---

Published as a conference paper at ICLR 2026
Query: Idewiit the funetiooal
Final Ardwer: The functional relotionship that the kertle heated on the stove, and subrequent- ly used to hot water into the
Query: Generase traffic light where of the lights green.
relationship between the ketile, stove and
Initial Analysis
Verificction via Visual Hypotheses
Initial Andlysis
Verifiattion Andlysis
Eupicizap
empty. two potwet al relations:
The kettle metallic. -. the cup ceramic.
Draft A is coherent with commonssnnse functional usage Dreft lass plassitia, casamic cupa are rarely used direcily for stovetop
traffic light typically consists
The top is red, the middle is yellow, and the botson green. But, the query roquines that none of lights be green, so the green light should be switched
Relation A: kottle is placed on the stowe for heutingwates
three lights.
Relation cup directly associated
with the stove for horheeinngguuu
Hypothesis Visualization
Fine-Gr Fine-Grained Eleboration
Errtial Generction
Final GenercTion
Gere
Draft A
Draft B
Figure 1: Illustration of the interleaved analytic drafting problem-solving loop, where understand- ing and generation interact synergistically to yield accurate solutions.
To realize such synergy, early works have sought to build coherent unified architectures, for instance by casting both understanding and generation autoregressive (AR) next-token prediction (Zhan
iet al. 2024| Wu et al. 2025a] Wang et al. 2024 Team 2024a). While this design provides a unified interface from an engineering standpoint, it often suffers from issues such as parameter competition
and task interference. More recent efforts explored (i) decoupled encoders with multi-head outputs
to reduce representational conflicts (Chen al. 2025c Ma al. 2023 Kou et al. [2024] Deng al.
2025), and (ii) hybrid AR- AR-diffusion models (Xie et al. [2025] Zhou et al. 2024| Xiao et al.) (025b) that take advantage of efficiency with high-fideitty rendering. Overall, these explorations of synergy have predominantly focused on improving architectural design unify the two abilities, while over- looking a crucial point: during task solving, the understanding and generation modules often lack close and explicit interaction, thereby failing to realize genuine synergy between comprehension and generation. To illustrate this, when a user's s instruction is ambiguous, an understanding module could first propose several plausible candidate solutions for question, then invoke generation to produce sketches or key visualizations as a means of "verifying" these candidates, ultimately yield- ing the correct answer. Conversely, once the generation module produces initial results, it could query the understanding module for high-level guidance, such as attributes or reasonable spatial lay- outs, to progressively refine the output (see Fig. D. This motivates new perspective: instead of treating understanding and generation as co-existing skills, we argue they should be interleaved in a problem-solving solving loop.
To combat this, we propose a novel thinking paradigm, termed the interleaved analyzing -uaftiing-drafting problem-solving loop (AD-Loop). The core idea is to enable the model to dynamically alternate between understanding and generation, thereby fostering an organic synergy that enhances over- all goblem-solvingg capability. Specifically, building upon existing UVLMs, we design to inter- leave textual thoughts, such as semantic abstraction and reasoning, with visual thoughts, including sketching and spatial layout. This dynamic switching between analytic and synthetic modes enables the model to iteratively refine its reasoning and outputs. Furthermore, we introduce a two-stage training strategy to effectively optimize the model, as shown in Fig. [ In the first stage, we con- struct supervised datasets of interleaved textual-visual thoughts, which initialize the model with the ability to alternate between analyzing and drafting in a guided manner. In the second stage, we employ reinforcement learning with hybrid feedback to encourage the model intelligently and au- tonomously decide when to invoke understanding versus generation, thereby fostering adaptive and self-directed synergy. Importantly, our proposed training framework and problem-solving paradigm are urchitecture-ggnssee making them broadly applicable to wide range of existing UVLMs. By integrating interleaved analyzing and drafting, our approach substantially enhances their ability to achieve deep synergy between comprehension and creation.
We conduct extensive experiments to evaluate the effectiveness of the proposed AD-Loop. First, across widely used understanding and generation benchmarks, our method consistently delivers broad and significant performance improvements, including an average +2.3% improvement on un- derstanding tasks and an overall score of 86% on GenEval. Next, ablations across thinking strategies demonstrate that interleaving analytic and drafting thinking yields clear advantages. We then con- duct transfer studies across different UVLM architectures, showing that our approach can be seam- lessly applied to diverse models and significantly enhance both understanding and generation bench- marks compared to the original baselines. Furthermore, visualizations of implicit visual thoughts
2

---

Published as a conference paper at ICLR 2026
confirm the rationality of our design. Finally, through detailed case analyses, we present compelling evidence that our model alternates between understanding and generation during booblem-solving leading to superior outcomes. Our contributions can be summarized as follows:
We propose a fundamentally new strategy for synergizing understanding and generation by introducing the interleaved
- thinking mechanism, which enables tight, explicit interactions between the two capabilities.
We develop novel two-stage learning strategy that equips UVLMs with the ability intel- ligently adopt interleaved analytic-drafting problem-solving loops during the task-solving process. Moreover, the framework is architecture-agnool Extensive experiments on understanding and generation benchmarks demonstrate the ef- fectiveness of our approach, providing intuitive visual analyses that confirm both the ratio- nality and efficacy of the proposed synergy.
across different UVLMS.
2 RELATED WORK
The mutual benefits between understanding and generation have long been a central theme Paivio
al. 2006 Ellamil et al. 2012 Fei et al. (025). Enabled by rapid progress in multimodal large
language models (MLLMs) (Liu et al.) 2024a Bai et al. 2023; Li et al.) [2224] Fei et al. 2024)
and diffusion- generators (Podell et al. [023] Yang et al. 2024). state-of-the-art systems now achieve strong performance on each task in isolation. This has sparked growing interest in unified vision-languaae models (UVLMs) (Xie et al. 2025 Kou et al. 20224) Wu et al. 2025b), which sup- port multimodal understanding and generation within single framework. Early explorations (Shen)
et al. 2023 Wu et al. 2024c) largely composed powerful specialist models by directly connecting an understanding model with generation model. However, such plug-and-play integration merely
locates the two capabilities and does not realize mutual assistance. More recent efforts pursue more coherent formulations that jointly model both tasks, e.g., casting both as autoregressive next-token prediction /Wu et al., 2025a Team 2024a Chen et al. 2025c b) Geng et al.) al.] (025b) or adopting uni- fied AR-diffusion architectures (Xie al. 2025| Zhou al.) 2024]]een
al. 2005). Yet even these parallel, independently callable skills; at inference time, the two modules rarely engage in close, explicit interaction, and genuine synergy remains elusive. In contrast, we introduce a novel thinking paradigm that enables true synergic learning between understanding and generation through an interleaved analyzing-draffee
tighter designs still treat understanding and generation
loop.
A complementary line of work (Shao et al. 2024 Xu et al. 2024 Shen et al.) 2025 Wang et al.,
2025a) focuses on endowing MLLMs with strong reasoning capabilities. Early approaches follow the Ohain-of-Thougee paradigm (Wei et al., 2022 by decomposing problems into substeps and su- pervising stepwise rationales. Subsequent methods scale test-time computation (Snell et al. via seff-consistency majority voting and incorporate search- inference (aoo et al. 2024).
2024)
while training-time credit assignment is improved through reinforcement learning (RL) or prefer- ence optimization tailored to reasoning. More recently, RL has been explored elicit emergent reasoning abilities that improve multimodal understanding (Huang et al. 2025 Chen et al. 2025aa Yuan et al. 2025) and generation (Xiao al., 2025b Diang et al.] 2025a). In light of this, some ex- plorations (iing et al. 2025b: Yan et al. 2025) attempt to leverage RL to co-optimize understanding and generation, thereby strengthening their respective capabilities. Beyond textual-only reasoning thoughts, several studies (Cheng et al. 2025a Shao et al., (024) interleave textual reasoning with visual evidence, either via tool-augmented pipelines (Zheng et 2005| /hhng et al., 2025a Man et a 2025 Su et al.] 2025) by directly generating visual traces (Chern et al.) 2024 Li al. 2025 Yang et al. 2025b). Building on these advances, our work models the synergy between understand- ing and generation
interleaved analyzing -draf-nng problem-solving loop that jointly produces textual and visual thoughts, thereby enhancing synergic learning within UVLMs.
3 METHODOLOGY
Building upon a UVLM, we model the synergetic understanding and generation thinking process as interleaved analyzing-drafting problem-solving loop. Given an input, the model alternates between analyzing (producing textual thoughts) and drafting (producing visual thoughts) before de- livering the final output. To achieve this, design a two-stage training pipeline: Stage 1 performs supervised training to imitate interleaved thinking, and Stage 2 leverages reinforcement learning to enable the model adaptively decide when to invoke analysis drafting.
3

---

Published as a conference paper at ICLR 2026
Stage-1; Initially learning for interleave thinking
Identify functional
<think>
relationship between the
[T-T]
Vison Encoder
kettle, stove and cup
[V-T]
Cluster
[V-T]
MSE
[V-T]
Encoder
W
Vison de
[T-T]+
[V-T]
[V-T]
MSE
<think
CE Gold analytic thoughts
Gold drafted thoughts
[Answer]
Gold answer
Stage-2: Rl.-driven adaptive interleaved thinking
UVLM (Policy Model)
Think AD-Loop
Intra reward
Group Computation
4
A
Reference Model
01
Reward Model
G/2
Heward
Think w/o AD-Loop
reward
6/22
4/22
update...
KL
Figure 2: Pipeline of our training framework. Stage-1: train the UVLM to emit interleaved thinking traces. Stage-2: apply GRPO for hybrid reasoning. The policy samples multiple traces with/with- out the interleaved AD-Loop for each input. reward model scores outcomes, and then group- normalized advantages applied to update the policy, teaching the model when AD-Loop helps.
3.1 MODEL. ARCHITECTURE
Let the imput be = (q.I). where is text and I = {I LM (M is an optional set of images. The input image set I {Tm) is encoded by vision encoder into visual embeddings, which fused with the textual stream and then processed by an LLM backhone for reasoning. The model outputs thinking trace wrapped in <think> and </think> tags and a final outcome:
<think> [T-T] [V-T] [T-T] [V-T] </thnn>> [Answer],
(1)
where (T-T] denotes text thought, [V-T] denotes visual thought, which enclosed by two special tokens marking the beginning and end of the visual thought. [Answer] is the final text or image. For image synthesis, prior unified models typically adopt either (i) discrete codebooks that predict quantized "image tokens" (Chen al.) 2025c Xie et al. 2025). (ii) continuous latent regressors that predict low-dimensional vectors consumed by a renderer (Deng et al. 2025). Emitting a full visual scene during the thought phase requires long token streams (often hundreds of tokens prolonged diffusion steps). which increases latency and entangles reasoning with pixel-level details that are irrelevant the decision. Motivated by cognitive accounts that internal imagery is schematic rather than pixel-accurate (Shepard & Metzler] 1971). we replace full rendering during thinking with compact set of latent visual thoughts. {u, }y=ı, which summarizes only the factors useful for reasoning under a strict budget K < the tokens required to render an image, This design preserves the sufficiency of visual cues for the downstream decision while avoiding the cost of pixel-level synthesis in the thought process.
3.2 STAGE-1: SUPERVISED TRAINING OF INTERLEAVED THINKING
We begin with supervised fine-tuning on interleaved reasoning data. This approach mitigates the instability commonly observed in cold-start RL for reasoning and provides strong initialization for next-stage reinforcement learning.
Dataset Construction. Our interleaved corpus comprises two halves: i) Understanding. We lever- age the existing multimodal CoT resources (Cheng et al. 2025b Zhang et al. 2025b Shao et al. 2024]. reorganizing their rationales and cropping the referenced regions according to the provided annotations construct the interleaved schema. We further synthesize AD-Loop thinking traces by instantiating schematic grid tasks following Li et al. (2025). total, the understanding portion contains 2 20 interleaved examples. ii) Generation. We use the GoT dataset (Fang al. 2025) and

---

Published as a conference paper at ICLR 2026
additionally construct Inter- T21 set from X-to-image corpora (Xiao et al. 2025a), augmenting each instance with an interleaved thinking trace. This yields 22K interleaved generation traces.
Training. To teach the model AD-Loop style of thinking. we fine-tune the UVLM directly on the above collected corpus, as shown in Fig. _ A practical issue is that the data sources provide explicit visual thoughts (i.e., pixel images), whereas our reasoning format requires latent visual thoughts. We therefore convert each explicit visual-thoughe image I into a compact set of tokens via a frozen generator-side encoder and a deterministic clustering step. Technically, we reuse the encoder for image generation from the unified model. Given I, the encoder yields a grid of latent tokens {zi)/=1. Rather than supervising the model to emit the full grid, we reduce { to small, semantically coherent set { with K < N. Following Jin al. (2024). we adopt density peaks clustering mechanism to construct the token clusters. For each cluster Cj. we produce c,1 Eiec, 2;-
representative latent visual thought token Vj the union of members Vyy
The clustering mechanism yields stable targets that merge tokens based on semantic proximity in the latent space, rather than through naive spatial pooling. We order v {uz) ) by the center coordinates of their clusters (top-left to bottom-right) to obtain a deterministic sequence. Let T and V* denote the gold text-thought and visual thoughts sequences in the <think> block, and let o" denote the final output. The final training objective is to optimize:
Lsi LCB (T, TM + a Liis (V. V*)
Loue (o, 0²);
task ourput
1ext thoughts
latent visual thoughts
where Lvis is mean-squared error, and Lout is the original task loss. a is the coefficient weight.
3.3 STAGE-2: RL-DRIVEN ADAPTIVE INTERLEAVED THINKING
After Stage-1 fine-tuning, the model has acquired an initial interleaved thinking capability. How- ever, some queries might be solved confidently with an isolated thinking mechanism, i.c., invok- ing only the understanding or generation capability. Accordingly, Stage-2 further strengthens the model's 's thinking competence and makes the policy adaptive for each input. The model should be able to decide whether to think with AD-Loop (V++) or without it (V-).
Drawing on group-relative preference optimization (Guo al.) 2025 Jiang al.) [025c]. we intro-
duce an adaptive interleaved thinking regimen, as illustrated in Fig. 2 For every query. the policy explores both thinking modes and receives inter- and intra-group normalized feedback. The resulting group-relative advantages drive on-policy updates toward a parsimonious strategy. We next detail the sampling scheme in Adaptive Interleaved Thinking, the reward design in Reward Assignment, and the update rule in Policy Optimization.
daptiiv Interleaved Thinking, For each query q. we sample two groups of trajectories from the old policy Totd: one with the AD-Loop enabled (fo: ),/(), and one with the AD-Loop disabled (fo7 G/2). Let be the total number of samples per query:
{ot) G/22
Tolds {0 [
Told; O 1,/ U (o G/2
(3)
Reward Assignment. Following Guo et al. 2025), each trajectory is assigned a sealar reward composed of formul component and a content component. For generation tasks, the content term aggregates alignment and quality scores (c.g.. preference feedback (Wu et al. (023), unified scor- ing (Wang et al. 2025b), or text-image alignment (Geng et al.) (025a)), while for understanding tasks, we apply rale- checks to enforce correctness:
Tbase Tbase(o)
F'urmal + Teuntent-
(4)
To explicitly encourage useful AD-Loop, we add a margin-based bonus V+ only when outper- forms the strongest V candidate and the AD-Loop contributes meaningfully:
r(a,t) ) = Thase (o; (AAD-Loopla) max 0, Tbuse (o max Pbaae 6): r(o,) ) = Touse (o (5)
where ((AD-Loopla) is an indicator that the answer is correct and AD-Loop thinking is employed. The margin parameter 8 requires the V+ candidate to exceed the strongest y- baseline by least
5

---

Published as a conference paper at ICLR 2026
Table 1: Comparison with baselines on multimodal understanding benchmarks. "Und." and "Gen."
denote understanding and generation, respectively.
Model
#Params
POPE
MME-PT
MMB
SEEDI
GQAt
MMMU1
MM-Vet
Und. Only
LLaVA=v15 (Lu el al. 2024n
7B
85.9
7510.7
64.3
38.6
62.0
354
31.1
Qwen-VL-Chin Bar et 2023
7B
1487.5
60.6
58.2
57.5
IDEHCS Laurencon
2025
InstructBLIP Daa
13B
78.9
1212.8
49.5
25.6
Und. and Gen
Emuü (Wang aral. 2024
8B
85.2
1243.0
58.3
68.2
60.3
3
37.2
Show-o [x1c
2025
1.38
80.0
1097.2
58.0
26.7
Liquid (wu al 2024a
1448.0
61.1
MMaDA Yang et
2025a
8B
86.1
1410.7
68.5
64.2
61.3
30.2
Janus-Pro (Cnch et 20230
7B
87.4
1567.1
79.2
72.1
62.0
41.0
50.0
BAGEL (Deng st
2025
1687.0
85.0
55.3
67.2
AD-Loop (Uürs))
7B
90.
1696.0
87.6
74.4
63.8
37.3
69.7
Table 2: Comprehensive generation comparison GenEval (Ghosh et al.
2023
benchmark. "Und."
and "Gen." " denote understanding and generation, respectively.
Model
Single Obj.t
Obj.t
Countingt
Colorst
Position
Attri.
Overall
Only
Emu3-Gen Wang al. 2024
0.98
0.71
0.34
0.81
0.17
0.21
0.54
SDXL (Podcll al 2025
0.98
0.74
0.39
0.85
0.15
0.23
0,55
PLUX.I-d03 Yang et al
2024
0.99
0.81
0.79
0.74
0.20
0.47
0.67
SD3-Medium (Al 2025
0.99
0.94
0.72
0.89
0.33
0.60
0.74
Und, and Gen.
Show-o (Xie
0.52
0.82
0.53"
TokenFlow-XI XL
2025
0.95
0.60
0.41
0.81
0.16
0.24
0.55
Janus-Pro (Chen al. 2023e
0.99
0.89
0.59
0.90
0.79
0.66
0.80
BAGEL Deng el al.
2025
0.99
0.94
0.81
0.88
0.64
0.63
0.82
MindOmmi Xiao ct al.
2025b
0.99
0.94
0.71
0.90
0.71
0.71
0.83
AD-Loop (Ours)
0.94
0.80
0.86
6, filtering out spurious wins and favoring the simpler text-only mode unless a meaningful gain achieved. À scales this bonus, balancing its impact against the base reward. Furthermore, introduce inter-group reward to indicate which thinking mode performs best for each query:
1. if arg max fr(o*), r(o-))
Tinter (o,"
m'e(+
Timura =r(,") m e (+,-) (6)
0, otherwise
Optimization.
Following group relatteleaivve preference optimization, combine intra-group ad- vantage with an optional inter-group term:
Tinraa (o.) mcaannrimat (0;)) stul (Fintaa((()))
Timter (o:) mman (oraan(mienn (Fimter (0;)) (f (oz))
A,
+7
GRPO for intra-group advantage Aun
GRPO for inter-group advantage Ame
This group-relative advantage A; of i-th response encourages the model to prioritize the response with higher relative quality. 7 is the weighted parameter. Following Guo et al.) (2025), we apply KL-divergence regularization with hyperparameter 39 and clipping trick optimize the model.
EXPERIMENTS
4.1 SETTINGS
We construct the training dataset to emerge the synergy between understanding and gen- cration, as described in Sec. (3.2 encompassing interleaved reasoning for both understanding and generation. The detailed statistics presented in Appendix I
Dataset.
Implementations. Our backbone BAGEL-7B so400m/14 (Tschannen al. 2025 adopted the image encoder for understanding, and the
(Deng et al. 2025) in which SigLIP2-

---

Published as a conference paper at ICLR 2026
Table 3: Comparison with different thinking strategies. Isolated thinking: understanding or genera-
tion performed in isolation; T: analyzing-only thinking; T + (explicit): supervised extual-visun
(explicit) interleaving: T + I (implicit): learned textual-visual (implicit) interleaving: 7 /T + I adaptive interleaved thinking that automatically selects with or without interleaved thinking.
Understanding
Generation (WISE)
Think Strategy
Math Vistat
LogieVista
SAT
Cultural
Spacef
Biologyt
Isolated thinking
61.5
40.2
0.63
0.44
0.68
0.44
T T + T (explicit)
68.3
44.1
0.74
0.67
0.69
0.56
72.9
46.6
0.81
0.73
0.74
0.64
T + (implicif)
73.6
47.2
0.84
0.75
0.77
0.65
TIT+1
75.8
49.5
0.89
0.79
0.78
0.68
Dagral
Prongt AAAAke cus faed..
ArT
cls ao
bell le thop
Deresind intarseeed
suudl doncssing shieed and
ultmasad drflo Sea (10)) Bicisoo slageiis
Latvegg wnd cnmiinss sigp
Lay wod canped click
andiier bed sheve
holpe cossting Vitle
Figure 3: Qualitative comparison: original prompt (left), self-think mode, interleaved thoughts, and text-only thoughts filtered from the interleaved thoughts (right). means latent visual thoughts.
FLUX pre-trate pre-trained VAE (Yang et al. 2024) is utilized as the image latent encoder/decoder for gen- eration. We train the model in two stages. At stage-1, we fine-tune the curated interleaved reasoning corpus using a global batch size of 256 and a cosine learning-rate schedule with initial learning rate 1xe-5 The maximum number of latent visual thought is 2tage-2, we implement RL with the VERL framework (Sheng et al. 2025). We fix the random seed 42 ensure reproducibility. The policy is optimized with Adam W. a constant learning rate of 2xe-6 a global batch size of 64, and 8 rollouts per prompt. The objective combines the clipped policy-gradient loss (c=0.5) with KL regularization to frozen reference model weighted by 0.01.
K=16 tokens. At
4.2 MAIN RESULTS
Understanding. We compare the proposed method against state-of-ther understanding -onll and unified models on multimodal understanding benchmarks, including POPE (Li et al. 2023b). MME (Zhang et al., (02T), MMB (Liu et al. 2024b), SEED (Li al. 2023a), GQA (Hudson & [aaninng20199 MMMU (Yue et al. 2024). MM-Vet (Yu et al. 2023). As highlighted in Table our method consistently yields the best overall results. This improvement stems from interlcaving visual-text reasoning, which effectively mitigates conflicts between understanding and generation while unlocking their synergy. Notably, even compared to MMaDA (Yang et al., 2025a), which utilizes RL-based thinking training, our approach still yields significant gains, confirming that gen- cration can indeed enhance understanding.
Generation. We assess visual generation performance on the GenEval benchmark (Ghosh et al,, 2023). As shown in Table 2 our method achieves the highest overall score of 86%, outperforming both generation-only and unified baselines. In particular, it delivers substantial improvements in fine-grained attributes in positional accuracy and attribute correctness. Moreover, compared with MindOmni (Xiao et al., (2025b) that employs textual thinking only, our approach achieves better performance, clearly demonstrating the effectiveness of incorporating visual thoughts.
4.3 ABLATION ON THINKING TYPES
We systematically evaluate reasoning strategies across both understanding and generation bench- marks, including Math Vista (Lu et al. 2023), Logic Vista (Xiao et al.) 2024). SAT (Ray et al. 2024). and WISE (Niu al. 2025). Results are summarized Table First, compared with the no- no-think setting, equipping the model with reasoning capability (T-think) yields substantial improvements in both understanding and generation, confirming the effectiveness of reasoning guidance. Second, augmenting textual reasoning with visual thoughts further boosts performance, highlighting that vi- sual cues supply fine-grained details complementary to text. When comparing explicit (I' + --thikk) and implicit (T + I-think) visual thoughts, we observe only marginal differences. This can be partly owed to the learning paradigm, and explicit supervision introduce pixel-level noise, whereas
7

---

Published as a conference paper at ICLR 2026
MMatar
Math
MM-Vet
X10
GenEval
x10- WIIE (culeure)
Abs. geii
5.8
4.9
5.7
5.6
16
5
1.5
2
0
0
Janus-Pro Pro BAGEL Janus-Pro BAGEL
Janus-Pro BAGEL.
Janus-Pro BAGEL
Janus- Pro BAGEL.
Figure 4: Absolute performance gain after applying AD-Loop, comparing Janus-Pro with discrete
tokenization and BAGEL with continuous embedding for visual thoughts learning.
Table Comparison of latent visual representations lear the generation (Gen.) understand- ing (Und.) encoder.
MMstar
Mist
Logic Vista
GenEval
WISE (cultural)
WISE (Biology)
From Gen, Encoder
54.9
75.8
47.5
0.86
0.79
0.68
From Und. Encoder
51.6
70.9
44.3
0.84
0.71
0.61
?
?
Figure 5: Examples of latent visual thoughts. Each case shows the original image (left) and the corresponding visual thoughts (right), capturing abstract visual structures.
0.5
Beapline
Vie
Skit
08-
VicT
6.76
Capabiliine
6:75
00
Rasia
0.6
B4B
06
(0:47
06
6.32
04
Indicaiiv
Dadicaive
Nometial
Spatial
Meshanial
Sequance
Puzzle
Ruiation
CS-OCR
Table
Complex
Infogrppiic
Char
10
C5 Moch
Physics
Figure 6: Performance across skills and capabilities on the Logie Vista dataset, comparing models with and without visual thoughts, alongside the proportion of visual-thought usage.
implicit reasoning captures compact salient cues. Nonetheless, combining the two in our hybrid setting achieves the best results across all tasks, suggesting strong complementarity.
Further qualitative evidence is provided in Fig. With the raw prompt alone, the model tends to generate superficial, semantically shallow outputs. Adding self-think produces scriptions, yet still overly abstract and often misaligned with user intent. By contrast, interleaved thoughts guide faithful, detail-oriented outputs (e.g., correct wheels/screens). Finally, filtering terleaved traces to text-only frequently reintroduces errors (c.g., lighting/positionnngg underscoring the necessity of visual thoughts for high-fidelity controllability.
detailed de-
4.4 ANALYSES AND DISCUSSION
In this section, we are about to answer the following in-depth questions:
Can this method extend into different structures of unified MLLMs? Current unified MI.MMs adopt different architectures for visual content generation. One line of work, including our backbone, generates continuous embeddings. while another, exemplified by Janus-Pro (Chen al.) 2025c), produces discrete tokens. We apply our proposed method to both paradigms, described in Sec. $3.2 As shown in Fig. Unified-RI RI consistently improves both understanding and generation across the two settings. These results demonstrate that approach is architecture-agnord and can serve as a general mechanism for strengthening emergent task synergy in unified MLLMss.
RQ-1:
RQ-2: Should visual thoughts be derived from understanding or generation? We compare visual thoughts derived from the understanding versus the generation encoder. As shown in Table using the generation encoder yields consistently better results for both understanding and generation tasks. This be attributed aspects. On the one hand, observe that models converge
8

---

Published as a conference paper at ICLR 2026
Prompt
Bilp-3o
MindOmni
BAGEL
An bottlc in water
trafflic light signaling drivvers propore stop.
chemellonn blending brown leaf
sitting windoweill, with a bird fooder hanging outside the window. The feeder should be slightly to right the window.
during winter.
Figure 7: Comparison of unified MILMs T2I generation. Existing models often fail prompts requiring deeper reasoning, whereas our approach yields more faithful outcomes.
faster under this setting, likely because generation-orienteed thoughts are already extensively pre- trained. Moreover, generation-bast visual thoughts inherently capture both semantic and pixel- level information, making them more informative and beneficial for multimodal reasoning.
RQ-3: What do the implicit visual thoughts look like? We visualize the clustering results in 55g.5] The observations align well with our expectations: latent visual thoughts encode semantically coherent information while preserving coarse pixel-level structures. This allows the model to recover the overall contours the original image and to unify conceptually similar regions. For example, in case (4), distinct regions depicting watermelons and lemons consistently represented by the same latent token, reflecting their shared conceptual category.
RQ-4: When are the visual thoughts needed? By comparing performance across different task scenarios and analyzing the proportion of visual-thought usage, we obtain the results shown Fig Overall, integrating Al-Loop thoughts improves performance across a wide range of questions, with pronounced gains in spatial and mechanistic reasoning. Fine-grained trends show preferential activation for rotation, complex OCR, and 3D perception, while usage drops for tables, sequences, and symbolic reasoning, where text-only chains suffice. These patterns indicate our adaptive policy that selectively invokes visual thoughts when they offer the greatest benefit.
RQ-5: Case study. Finally, we provide qualitative analyses of our model's capabilities in text-to- image generation. As demonstrated in Fig. 7 baseline models often falter on iintensiv prompts, capturing superficial cues rather than underlying logic- for example, outputting red/- green for "a traffic light signaling drivers to prepare to stop" (correct: yellow), rendering green chameleon on brown leaf (ignoring adaptive coloration), or producing bright sunflowers for "a sunflower field during winter" (ignoring seasonality). In contrast, outputs with prompt semantics and the required reasoning. On the understanding side (Fig. 10). our model exhibits stronger spatial reasoning, correctly localizing objects and motions where baselines fail to do so. These cases highlight how interleaved reasoning facilitates faithful, reasoning aware generation and more robust understanding, Additional examples be found in Appendix ş
method consistently aligns

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this work, introduce unified interleaved thinking framework, i.c., AD-Loop, for synergizing understanding and generation in UVLMs. Our proposed two-stage learning paradigm first initial- izes an interleaved AD-Loop thinking through supervised fine-tuning, and then employs hybrid re- inforcement learning to enable the model to adaptively invoke interleaved thinking when beneficial. This allows the model to learn when and how to leverage different capabilities, thereby unlocking genuine synergy across tasks. Extensive experiments on diverse multimodal understanding and gen- eration benchmarks demonstrate the effectiveness of approach, with particularly strong gains reasoning driven tasks. These results highlight the potential of interleaved AD-Loop as a general mechanism for advancing unified multimodal intelligence.
ACKNOWLEDGEMENTS
This work was supported by a Senior Al 2050 Fellowship from the Schmidt Sciences Foundation, awarded to Michael Wooldridge.
REFERENCES
Stability AI. Stable diffusion 3 medium:
Multimodal diffusson transformer for pho-
torcalistic text-to-image generation, 2025. stable-di -
URL
um
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Greg Brockman, Vicki Cheung. Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint
540, 2016.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or ri? an early investigation into training rl-like reasoning large vision-language models. arXiv preprint ar arXiv:2504.11. 10.500441166.. 2025a.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models-architecttur,, training and dataset. arXiv preprint arXiv:2505.0955 2025b.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.172 17811. 2025c.
Zihui Cheng, Qiguang Chen, Xiao Xu, Jiaqi Wang. Weiyun Wang, Hao Fei, Yidong Wang, Alex Jin- peng Wang, Zhi Chen, Wanxiang Che, and Libo Qin. Visual thoughts: A unified perspective of understanding multimodal -hain-off-hhught CoRR, 0is/2505.11510, 2025a.
Zihui Cheng, Qiguang Chen, Jin Zhang, Hao Fei, Xiaocheng Feng, Wanxiang Che, Min Li, and Libo Qin. Comt: A novel benchmark for chain of multi-modal thought on large vision-language models. Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2025b.
Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large mul- timodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06733, 2024.
Ethan Chern, Zhulin Hu, Steffi Chern, Siqi Kou, Jiadi Su, Yan Ma, Zhijie Deng, and Pengfei Liu. Thinking with generated images. CoRR, abs/2505.2252. 2025.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards seneral-purposs vision-language models with instruction tuning. Advances in neural information processing systems, 36:49250-49266 2023.
10

---

Published as a conference paper at ICLR 2026
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683, 2025.
Melissa Ellamil, Charles Dobson, Mark Beeman, and Kalina Christoff. Evaluative and generative modes of thought during the creative process. Neurolmage, 59(2):1783-177) 2012.
Rongyao Fang. Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Xihui Liu, and Hongsheng Li. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing. CoRR, abs/2503.10639, 2025.
Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, and Shuicheng Yan. Vitron: A unified pixel-level vision Ilm for understanding, generating, segmenting, editing. 2024.
Hao Fei, Yuan Zhou, Juncheng Li, Xiangtai Li, Qingshan Xu, Bobo Li, Shengqiong Wu, Yaoting Wang, Junbao Zhou, Jiahao Meng, et al. On path to multimodal generalist: General-level and general-bench. In Proceedings of the International Conference on Machine Learning, 2025.
Zigang Geng, Yibing Wang. Yeyao Ma, Chen Li, Yongming Rao, Shuyang Gu, Zhao Zhong, Qinglin Lu, Han Hu, Xiaosong Zhang, et al. X-omni: Reinforcement learning makes discrete autoregres- sive image generative models great again. arXiv preprint arXiv:2507.22055. 2025a.
Zigang Geng. Yibing Wang, Yeyao Ma, Chen Li, Yongming Rao, Shuyang Gu, Zhao Zhong, Qinglin Lu, Han Hu, Xiaosong Zhang, et al. X-omni: Reinforcement learning makes discrete autoregres- sive image generative models great again. arXiv preprint UxXiv:2507.22058. 2025b.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 22132-52153 2023.
Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12944, 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. CoRR, abs/2503.06749, 2025.
Drew A Hudson and Christopher Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6700-6709, 2019.
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703, 2025a.
Jingjing Jiang, Chongjie Si, Jun Luo, Hanwang Zhang, and Chao Ma. Co-reinforcement learning for unified multimodal understanding and generation. arXiv preprint arXiv:2505.17534 2025b.
Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, and Furu Wei. Think only when you need with large hebrid-reasoning models. arXiv preprint arXiv:2505. 14631, 2025c.
Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13700- 13710, 2024.
Siqi Kou, Jiachun Jin, Zhihong Liu, Chang Liu, Ye Ma, Jian Jia. Quan Chen, Peng Jiang, and Zhijie Deng. Orthus: Autoregressive interleaved image-text generation with modality-speciff heads. arXiv preprint 4rX 2024.
11

---

Published as a conference paper at ICLR 2026
Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al. Obelics: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural Information Processing Systems, 36:71 (8771683-71700 2023.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. Transactions on Machine Learning Research, 2024.
Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Bench- marking multimodal IIms with generative comprchension. arXiv preprint arXiv:2307.16125, 2023a.
Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of offtuuuut... arXiv preprint arXiv:2501.07542 2025.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305. 10355, 2023b.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 26296-26306, 2024a.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: your multi-modal model an all-around player? In European conference on computer vision, pp. 216-233, 2024b.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi. Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:23 022555, 2023.
Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A unified tokenizer for visual generation and understanding. CoRR, abs/2502.20321, 2025.
Yunze Man, De-An Huang, Guilin Liu, Shiwei Sheng, Shilong Liu, Liang- Yan Gui, Jan Kautz, Yu- Xiong Wang, and Zhiding Yu. Argus: Vision-centric reasoning with grounded chain-of-thoughh In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 14268-14280, 2025.
Yuwei NIu, Munan Ning, Mengren Zheng. Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.0m 072665, 2025.
Allan Paivio, James M Clark, et al. Dual coding theory and education. Pathways to literacy achieve- ment for high poverty children, 1:149-210 240, 2006.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint 20X002230. 2023.
Qwen, An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu. Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Har Zeng, et al. Sat: Spatial aptitude training for multimodal language models. arXiv e-prints, pp. arXiv-2412, 2024.
12

---

Published as a conference paper at ICLR 2026
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hong- sheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for seain-of-thoughhe reasoning. Advances in Neural Information Processing Systems, 37:8612-8642, 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. VLM-R1: A stable and generalizable rl rtyyt large vision-language model. CoRR, abs/2504.07615, 2025.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yucting Zhuang. Hugging- gpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:381 154 38180, 2023.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp. 1279-1297, 2025.
Roger N Shepard and Jacqueline Metzler. Mental rotation three dinensionai objects. Science, 171(3972)701-700: 1971.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can more effective than scaling model parameters. arXiv preprint arXiv:2408.033144.2 2024.
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, al. Openthinkimg: Learning to think with images visual tool reinforcement learning. arXiv preprint arXiv:2505.08617. 2025.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024a.
Qwen Team. Qvq: To see the world with wisdom, December 2024b. httettpp////g github. gitthub.ic/blog/gqq 72b http://vvvvgb--eeeeeeeeeeeeeee
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features, 2025.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXi::2409, 18869, 2024.
Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: comprehensive survey. arXiv preprint arXiv:2503. 12605, 2025a.
Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multi- modal understanding and generation. arXiv preprint arXiv:2503.05236, 2025b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thoouhh prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-244 24837, 2022.
Chengyue Wu, Xiaokang Chen, Zhiyu WU, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattrn Recognition Conference, pp. 12966-12977, 2025a.
13

---

Published as a conference paper at ICLR 2026
Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable and unified multi-modal generators. arXiv preprint ar arXiv:2412.04332, 2024a.
Shengqiong Wu, Hao Fei, Xiangtai Li, Jiayi Ji, Hanwang Zhang, Tat-Seng Chua, and Shuicheng Yan. Towards semantic equivalence of tokenization in multimodal llm. arXiv preprint 17X 2024b.
Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multi- modal 1lm. In Forty-first International Conference on Machine Learning, 2024c.
Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Zhonghua Wu, Qingyi Tao, Wentao Liu, Wei Li, and Chen Change Loy. Harmonizing visual representations for unified multimodal understanding and generation. arXiv preprint arXiv:2503.21973 2025b.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2.306. 09340,941, 2023.
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang. Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 13294-13304, 2025a.
Yicheng Xiao, Lin Song, Yukang Chen, Yingmin Luo, Yuxin Chen, Yukang Gan, Wei Huang, Xiu Li, Xiaojuan Qi, and Ying Shan. Mindomni: Unleashing reasoning generation in vision language models with rgpo arXiv preprint arXiv:2505. 13031, 2025b.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal I1m logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.000 04973, 2024.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In The Thirteenth International Conference on Learning Representations, 2025.
Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv preprint urXiv:2411.104400 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 1993--15935 2023.
Zhiyuan Yan, Kaiqing Lin, Zongjian Li, Junyan Ye, Hui Han, Zhendong Wang. Hao Liu, Bin Lin, Hao Li, Xue Xu, et al. Can understanding and generation truly benefit together-or just coexist? arXiv preprint ar 200.::5000006666 2025.
Chenglin Yang, Celong Liu, Xueqing Deng, Dongwon Kim, Xing Mei, Xiaohui Shen, and Liang- Chieh Chen. bit flux. arXiv preprint arXiv:2412.1865 2024.
Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505. 15809, 2025a.
Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, and Chuang Gan. Machine mental imagery: Empower multimodal reasoning with latent visual tokens. arXiv preprint arXiv:2506.1777 17218, 2025b.
Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng. Li Shen, et al. Mulberry: Empowering milm with ol-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18311 2024.
Weihao Yu, Zhengyuan Yang, Linjie Li. Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint ar 2308 230.2930.0249 02490, 2023.
14

---

Published as a conference paper at ICLR 2026
Ruifeng Yuan, Chenghao Xiao, Sicong Leng, Jianyu Wang, Long Li, Weiwen Xu, Hou Pong Chan, Deli Zhao, Tingyang Xu, Zhongyu Wei, et al. Vl-cogito: Progressive curriculum reinforcement learning for advanced multimodal reasoning. arXiv preprint arXiv:2507.22600 2025.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multi- modal understanding and reasoming benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-9567, 2024.
Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Ziyu Liu, Shengyuan Ding, Shenxi Wu, Yubo Ma, Haodong Duan, Wenwei Zhang, et al. antemimm-ccomposee 5-reward: A simple yet effective multi-modal reward model. arXiv preprint arXiv:2501. 2025.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, et al. Anygpt: Unified multimodal Ilm with discrete sequence modeling. arXiv preprint wrXi::2402.12226, 2024.
Xintong Zhang. Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, and Qing Li. Chain-of-focus: Adaptive visual search and zoom- ing for multimodal reasoning via RL. CoRR, abs/2505.15436. 2025a.
Xintong Zhang. Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl. arXiv preprint arXiv:2505. 15436, 2025b.
Yunhang Shen Yulei Qin Mengdan Zhang, Xu Lin Jinrui Yang Xiawu Zheng, Ke Li Xing Sun Yun- sheng Wu, Rongrong Ji Chaoyou Fu, and Pcixian Chen. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint S.Kiv:2306.139944 2021.
Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang. Chao Shen, and Xing Yu. Deepeyes: Incentivizing "thinking with images" via reinforcement learning. CoRR, abs/2505.14362, 2025.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408. 1039, 2024.
15

---

Published as a conference paper at ICLR 2026
APPENDIX INDEX
This supplementary material includes the following sections:
Clarification on the Use of Large Language Models (cf. (A)
Ethics Statement (cf. B
Reproducibility Statement (ef.
C
Detailed methodology (cf.
(
Comparison with Existing Synergetic Learning Methods (cf.
E
Detailed Dataset Construction (cf. F
Detailed Setting (cf. [G)
Extended Experimental Results (cf.
Thinking Strategies Comparison with Existing VLMs via Visual Information (cf. !
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this paper, we employ the large language models (LLMs) to help the dataset construction and improve the clarity and readability of English writing. Specifically, we employ the QVQ-72B- Preview to construct the Inter-T2I dataset. Additionally, LLMs were utilized to refine sentence structure, correct grammatical errors, and enhance the overall presentation our draft. The technical content, research ideas, experimental design, analysis, and conclusions were entirely conceived, implemented, and validated by the authors without reliance on LLMs.
B ETHICS STATEMENT
All datasets used in this work are publicly available and open-source. During the process of con- structing our experimental data, employed open-source text-to-image generation models. To mitigate risks of harmful content, discrimination, or bias, all generated samples were manually screened and filtered ensure suitability for research purposes. Our approach is built on an open- source foundation model that provides strong assurance for generating non-harmful and bias-free content. Nonetheless, as with any generative system, potentially harmful outputs will never occur. We therefore emphasize that users and practition- ers should exercise caution when deploying such models in downstream applications, especially in sensitive domains.
impossible guarantee that unintended
We do not involve any human subjects, private or proprictary data, or personally identifiable in- formation (PII) in this research. The work complies with community standards data usage and research integrity. No legal, ethical, or security violations are posed by the methodologies or exper- iments presented in this paper. Furthermore, for academic and scientific exploration. We explicitly discourage misuse of the proposed methods for generating harmful, misleading, discriminatory content. Future applications should integrate appropriate safeguards, fairness considerations, and content moderation mechanisms.
highlight that contributions are intended solely
C REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, we have made a concerted effort to provide all neces- sary details and materials. We provide comprehensive details of the proposed AD-Loop framework, including its definition, input-output formulation, and implementation (Section $3.I). The model backbone and training methodology are described detail in Section $3 Appendix $D We fur- ther report all hyperparameter settings and training configurations in Section b.ction4.1]an and Appendix G using fixed random seeds to ensure the replicability of the experiments. All datasets used in this study publicly available open-source resources, and the data construction process, along with the amount of data used at each training stage, is thoroughly documented in Appendix $F Finally, will release the full codebase and data processing scripts to the community upon acceptance.
16

---

Published as a conference paper at ICLR 2026
D DETAILED METHODOLOGY
D.1 LATENT VISUAL THOUGHTS CONSTRUCTION.
Given an image visual thought 1, the image encoder yields a grid of latent tokens where the image encoder be either VQ tokenizer (Sun al. 2024 Chen et al.
2 L.V
2025c
VAE encoder Yang et al. 2024 Deng et al. 2025). Inspired by Wu et al. (024b), we then calculate
the local density Pr of the token 2; E Z by referring its neighbors:
Pi = e==epp(- K
1
E
1|zm: 2i
Zmm (K.N(..N
where (Z Z) denotes the K-nearest neighbors of in Z. We then measure the minimal distance 6, between the feature 2; and other features with higher density:
miitmppepp ç(zm; Zi), if m, n Pm > pi
=
(9)
maxm (( zi),
otherwise
In essence, ò; represents the distance between the given token 2, and other high-density tokens. We summarize the score s; of the feature by combining the local density P; and minimal distance 6; as Ps x 6;- We identify those tokens with relatively high scores, 8j, as cluster centers and then allocate other tokens their nearest cluster center based the Euclidean distances. Finally, utilize the average token within each cluster to represent the corresponding cluster. The latent visual thought token of the merged patch token is the union of the vision regions within the corresponding cluster.
D.2 REWARD MODEL
In Stage each model response evaluated with two complementary signals: (i) a format reward that enforces structural validity (e.g., required fields, schema conformity), and (ii) content reward that measures semantic fidelity and task-specific quality. Below, we describe the content rewards used for different task types.
Understanding-Ine Task Reward Model For tasks with deterministic targets. such as multiple- choice or numeric questions, we employ rule-based matching after normalization (e.g.. case folding, whitespace removal, and unit normalization). This yields precise, low-variance signal of correct- ness. For open-ended understanding questions, we rely on an external, learned reward model as the judge; specifically, we use M.mmlL--CCoposs22..... responses by holistic relevance and coherence with the instruction.
(Zang et al.) 2025). which scores
Generation-Tast Task Reward Model For assessing the quality of generated images, we employ two complementary criteria: 1) Semantic alignment score: Measures agreement between the generated image and the ground- truth prompt via cosine similarity between CLIP image and text embeddings. 2) Human preference alignment score: Captures perceived aesthetic quality and prompt adherence using learned preference models, namely HPS v2 (Wu al.) 2023) and ImageReward (Xu et al.) 2023 These signals are aggregated to provide fine-grained, content-aware feedback that guides the model toward faithful, high-quality generations while maintaining the required output format.
D.3 THE OPTIMIZATION OBJECTIVES IN STAGE-2
After obtaining the advantages, we apply the standard objectives as described in Guo et al. (2025) optimize model. In addition, to prevent the optimized policy my from diverging excessively from the reference model Tref. KL-divergence regularization term DxL is introduced. The overall optimization objective is:
max E (g 12 (0 R9
G
(o()s)
A, clip To (o():)
(10)
min
Ta (o;) 1 +))A)) 3 DKI.. (Toll
T
Tea (ox)
i=1
where e, 3 are the hyper-parameters.
17

---

Published as a conference paper at ICLR 2026
UVLM
UVLM
UVLM
Text Thought
Text Thought
Isolated Learning
Duol Learning
Co-Learning
Figure 8: Comparison of existing mechanisms for synergizing understanding and generation, in- cluding isolated learning where the two abilities are trained independently, dual learning which leverages cross-modal reconstruction for mutual supervision, and co-learning which jointly opti- mizes both tasks with paired samples.
E COMPARISON WITH EXISTING SYNERGETIC LEARNING METHODS
We here position our approach in relation existing methods for synergistic learning between un- derstanding and generation. As illustrated in Fig. g prior work can be broadly grouped into two major directions.
Unified Models for Synergistic Learning. A common line of research aims unify understand- ing and generation within a single framework. Representative examples (Xie et al. 2025 Wu et al., 2025a Deng et 2025 Xiao et al., 20256) include models that adopt a purely autoregressive for- mulation, or hybrid designs that combine autoregressive and diffusion- based paradigms. In these approaches, synergy encouraged through shared parameters, which enable implicit interaction between the two tasks. However, during training and inference, the two abilities remain largely independent. As a result, such models primarily learn to master understanding and generation in parallel, rather than achieving genuine synergy in task-solving.
Learning Optimization for Synergistic Learning. Another approach is to design learning schema that fosters synergy between understanding and generation. For instance, dual learning methods Yann et al. 2025) translate visual inputs into textual descriptions and then regenerate vi- suals from those descriptions, with reconstruction quality serving as the optimization signal. Co- learning strategies (Jiang et al., (025b) further extend this idea by coupling each sample with both generation prompts and multimodal understanding queries, thereby jointly improving performance on the two tasks. While effective for mutual supervision, these approaches still operate at the stage of co-training skills, rather than enabling active collaboration between them during task execution.
Our Contribution. contrast, our work introduces fundamentally different perspective. We argue that real synergy should emerge not only during the learning phase but also in the problem- solving process itself. Specifically, we propose the Analyzing-Drafting groblem-solving Loop (AD- Loop), novel problem-solving mechanism in which understanding and generation at appropriate moments to jointly address a task. Furthermore, we develop a two-stage training strategy that equips unified multimodal models with this capability. In particular, our second-stage hybrid learning scheme enables the model to adaptively and intelligently decide when to invoke understanding or generation, thereby achieving organic integration of the abilities and realizing genuine synergy.
interleaved
F DETAILED DATASET CONSTRUCTION
Here, outline the process of constructing training data. The overall statistics presented in Table and an instance example visualization of each constructed dataset is shown in Fig. 9
AD-Loop Dataset for Understanding Task. To construct the interleaved analyzing- drafting loop for understanding tasks, we draw upon the following representative sources:
CMoT (Cheng et al. 20256) is a chain of multimodal multimodal-thought dataset that requires mul- timodal input and multi-output reasoning output. It consists of four categories: (1) Visual
18

---

Published as a conference paper at ICLR 2026
User query
Analyzing-Draftinn prablem-sohving loop
Answer
What the tangrata like! dancing C
primary
depictid.... likeness to body..
As visible resemblance to armss
As represented,
As visible
In man duncing
onion
focus given
these legs.
similar head
B
What the person playing?
Identify person image.
Identify objects associated with the person.
Query the object relationship with person find the answer.
Haseball
Your task is.. devise un action plan rench the goal ... your action plan.
Go Down
Go
Right
Go Down
Go Up Ge Right
The image (estures a rustic seane with a wooden cutting board V-T] as the central element. board, there is a loaf of French bread. sliced into thick picces [V-T] , with fow slices already cut. To the left the bread, there small white right of tread, there bottle of Italian "Geurnet Toscano," with dark green cup and a purple label. The bottle positioned upright, with the label facing forwurd. In backgreund, there bost filled with green olives [V-T], adding the scene. The overall composition balanced,
Virgin Olive Oil, sliced bread, and a plate with olive wooden surface
with single olive oil droplet
single green olive Virgin Olive VV1 labeled
[V-T].
Mediterracan
bread
bottle as the focal points, suroundeed by the olives and the cutting boand, creating harmonious visual
Figure 9: Visualization of the constructed dataset. Given a user query, we establish an analyz- ing drafting problem-solvingg loop in which textual and visual thoughts alternate dynamically, ulti- mately yielding the final answer.
Creation, (2) Visual Deletion, (3) Visual Update, and (4) Visual Selection to comprehen-
sively explore complex visual operations and concise expression in real scenarios. The final average number of AD-Loop is 3.
Visual-CoT (Shao et al. 2024) provides intermediate reasoning annotations with bounding
boxes that highlight critical regions necessary for answering visual reasoning questions. We derive explicit visual thoughts by zooming into and cropping the annotated regions, which serve as supervision signals for the visual-thought channel, resulting in an average of 2 per
AD-Loop trajectory.
CoF (Zhang et al., 2025b) identifies focus regions during question answering. We derive explicit visual thoughts by zooming into and cropping the annotated regions, which serve supervision signals for the visual-thought channel, yielding an average of 3 per sample. Visual Spatial Planning (VSP) Yang et al. (2025b) formulates schematic grid-based navi- gation tasks, in which agent must move from designated start to destination while avoiding "holes". Following Li al. (002), we render these states using the OpenAl Gym framework (Brockman et al. 2016), with the initial map and action sequence inputs. Visual thoughts correspond to the visualization of each movement step, with an average of 3 latent visual thoughts.
AD-Loop Dataset for Generation Task. For generation tasks, construct interleaved analyz- ing-diafting resources from the following datasets and models:
GoT-T2I (Fang et al. 2025) provides both reasoning traces and bounding-box annotations salient objects. We crop the corresponding image regions based on the bounding box information and append these cropped patches directly after the associated textual descrip- tions, thereby forming explicit visual representations. We treat the annotated bounding boxes of salient objects latent visual thoughts, resulting in average of 4.
19

---

Published as a conference paper at ICLR 2026
Table The statistics of the constructed dataset for two-stage learning. 'Avg.' indicates the average number of latent visual thoughts within the datasets.
Task
Datasct
Avg.
Stage-
Stage-2
Total
CMoT Cheng et al.
2025b
3
IK
IK
2K
Visual-Cor Shao et al. 2024
2
12K
IK
13K
Understanding
CoF Zhang
2025b
3
4K
1K
5K
VSP Yang al. 2025b
3K
1K
4K
Go (Fang ct al. 2025)
12K
3K
15K
Generation
Inter- T21
4
10K
2K
12K
Inter-T21 We leverage X-to-Image (Xiao et al.) 2025a) corpora to obtain high-quality prompt-image pairs, and employ QVQ-72B- (Team 2024b) to construct reflective critiques conditioned on visual inputs. To enrich intermediate visual thoughts, we adopt Fluxl-dev (Yang et al.] (024)) generate visual hypotheses guided by subgoals extracted from the prompts. We generate visual hypotheses based on sub-goals extracted from the prompts and retain only the first occurrence of each object avoid duplication. The aver- age number of latent visual thoughts is 4.
G DETAILED SETTINGS
G.1 TRAINING CONFIGURATION
We adopt two distinct UVLMs as the backbone of our framework. While both models incorpo- rate dedicated heads for video understanding and generation, they differ fundamentally in their ap- proaches to video synthesis: Janus-Pro employs LLM backbone that generates discrete tokens, whereas BAGEL adopts design that generates continuous tokens. Below, we detail the architectural designs of these two models.
BAGEL (7B) (Deng et al. 2025) is initialized from Qwen2.5 LLM (Qwen et al., 2025 For vi- sual understanding, BAGEL adopts SigLIP2-so400mg
(Tschannen et al. 2025) with fixed 384- resolution to convert the raw pixels into tokens. For visual generation, BAGEL EL utilizes a pre-trained VAE model from FLUX Yangg al. 2024) to convert images from pixel space latent space and vice versa. The latent representation is then processed by a 2 x patch embedding layer to reduce the spatial size and match the hidden dimension of the LLM backbone.
Janus-Pro (7B) (Chen et al. 2025c utilizes the SigLIP /Zhi et al. (023) to extract high-dimensional semantic features from images. These features flattened from 2-D grid into 1-D sequence, and an understanding adaptor is used to map these image features into the input space of the LLM for understanding. For visual generation tasks, Janus-Pro adopts the VQ tokenizer from Sun et al. 2024), which discretizes images into token IDs for autoregressive generation.
During the training, the maximum number of each latent visual thought is set 16 tokens. At stage 2, we set À = 1.0 and a margin = 0.2 for RL training. We set the weighting hyperparameters and y 1. We set the KL coefficient to 0.001 and e 0.5.. Besides the interleaved data,
we also leverage 4K text-only reasoning data (Huang et al. 2025
Li et 2024) in Stage 2.
G.2 EVALUATION METRICS
Here, we detail the evaluation metrics employed in the experiments. For understanding tasks, we report accuracy, following standard practice Xie et al. )2025); [eng al. (2025): Ma et al.) (2025), to measure the percentage of correctly predicted answers each dataset. For generation tasks, conduct evaluations on two benchmarks:
(1) GenEval. We adopt the official GenEval toolki to assess text-to-image generation quality. GenEval comprises six task categories, including single object, two objects, count- ing, colors, position, and attribute binding. The GENEVAL score is formulated binary correctness measure, indicating whether all elements specified the prompt are faithfully
hhtpss//gituh..co//gghooo 3/genvval
20

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison of explicit and latent visual thoughts on the spatial visual planning task at levels 3 and 6.
Type
Model
Level 3
Level 6
Anole
Chern al.
2025)
0.02
0.00
Explicit
MVoT
ct al.
20255
0.21
0.03
Mirage
Yang et al.
2025b
0.75
0.39
Implicit
Ours
0.81
0.47
rendered in the generated image. This setup directly evaluates the model's compositional alignment between textual instructions and visual outputs.
(2) WISE. We select subset of WISE domains (e.g.. biology, culture, and space) for eval- uation. Following prior work Deng et al. (2023); [Chnn et al. (2025c), we report WiScore, the primary metric of the benchmark, which emphasizes the accuracy of depicted objects and entities grounded in world knowledge. WiScore is computed as weighted combina- tion of three components: Consistency, Realism, and Aesthetic Quality. Higher WiScore values indicate stronger capability in generating images that correctly represent real-world concepts while maintaining visual plausibility.
H EXTENDED EXPERIMENTAL RESULTS
Comparison of Explicit and Implicit Visual Thoughts. We compare existing explicit and im- plicit approaches on the spatial visual planning task. As shown in Table explicit methods yield the weakest performance. intermediate visual thoughts, the quality of these drafts is heavily constrained by the model's in- herent generation capability. In such cases, the model struggles to construct faithful visualizations from high-level semantic representations, resulting in low-quality visual thoughts. Building upon these imperfect drafts further propagates errors, resulting in cumulative degradation of reasoning. By contrast, latent (implicit) visual thoughts achieve consistently stronger results, they bypass the limitations of explicit drafting and enable more faithful internal reasoning representations.
likely reason is that when the model is required to explicitly draft
Visualization Understanding Tasks. We further provide qualitative visualizations of the model's 's performance on various understanding tasks. For instance, in mathematical reasoning prob- lems (Fig. IT). the model is first able to analyze the question, then drafts intermediate visual sketches analogous to human scratch work, and finally derives the correct solution. Similarly, in more abstract scenarios, such as tangram analysis (Fig. (2), the model successfully generates accurate intermedi- ate visualizations that facilitate the correct interpretation, significantly outperforming approaches like BAGEL, which also employ self- thf-kihin strategics.
Visualization on Generation Tasks. We also provide additional qualitative results generation tasks. Compared with alternative reasoning-bes based methods, our model demonstrates superior per- formance on prompts that require commonsense reasoning. For example, shown in Fig. 13 base- line models tend to generate kangaroos sitting directly on the ground, whereas our model correctly depicts the kangaroo using its tail for support, which more consistent with reality. In creative syn- thesis scenarios, such as the counterfactual prompt "a plant watering a gardener" our model is still able to produce outputs that faithfully capture the user's intent. Furthermore, Fig. [4 contrasts gen- erations produced by the base model with and without our thinking mechanism, where our approach consistently yields higher-qualitg images that better align with user instructions.
Visualization on Editing Tasks. We further compare our approach with BAGEL (Deng et al., 2025) on diverse image editing tasks, both with and without the "think" process, as shown Fig 15 The results indicate that our method consistently produces more faithful and semantically aligned edits. For instance, when asked to infer the missing element in a visual pattern, BAGEL either fails to capture the correct reasoning generates unrelated content, whereas our model accurately iden- tifies the missing orange. In procedural editing tasks such as heating corn kernels until they pop, approach generates realistic popcorn, while BAGEL outputs less plausible textures. Similarly,
21

---

Published as a conference paper at ICLR 2026
for biological transformations (e.g., albinism in corals) and structural corrections (e.g., fixing reasonable parts of a bicycle), our method yields coherent results and visual fidelity. These comparisons highlight the strength of our interleaved thinking process in producing edits that are both accurate and visually credible.
that respect both semantic intent
Challenging Cases. Fig. 16 and 17 present several challenging examples where our model still exhibits limitations. On the understanding side, the model may occasionally over-draft on simple queries. For very basic attribute questions, the introduction of additional reasoning steps can lead to unnecessary elaboration and, in some cases, incorrect predictions. Similarly, although the model shows improvements on visually grounded mathematical tasks, such as Math Vista (Lu et al. 2023), it still struggles on problems that require strict symbolic reasoning, where applying AD-Loop may introduce deviations from the optimal logical path.
On the generation side, the model continues to face difficulties in rendering fine-grained visual details, including scene text, subtle body parts (e.g., fingers), and numerically precise elements. A plausible explanation is that the compact latent visual representations may under-encode such fine details. We anticipate that incorporating more detailed feedback into the Stage-2 RL reward, encouraging the model to attend fine-grained latent visual thoughts, could further mitigate these issues in future work.
I THINKING STRATEGIES COMPARISON WITH EXISTING VLMS VIA VISUAL INFORMATION
Compared existing "alternation-basee "integration- visual information argumented multimodal reasoning works (Zhang et al., 2022aa Zheng et al.] (025), the proposed AD-Loop is fundamentally different across several objective and design perspectives.
(1) Objective: achieving genuine synergy between understanding and generation. The goal of AD-Loop is not merely to alternate modalities but to enable a mutual strengthening between the model's understanding and generation capabilities. The entire framework, both in design and training. is centered around this objective, which distinguishes from prior works that only combine modalities without enabling reciprocal influence.
(2) Modeling perspective: internal, bidirectional, and iterative interaction rather than passive alternation. In AD-Loop, both textual and visual information are derived from the model's 's internal reasoning without relying on external tools for visual integration. The process is not fixed alter- nation but dynamic switching between analytic (textual) and synthetic (visual-drafting) modes, allowing iterative refinement of reasoning. Moreover, unlike prior methods that use explicit images pixel-level drafts, AD-Loop introduces latent visual thoughts, which serve concept-alignee abstractions. These representations compress visual regions into semantic clusters and disentangle relevant from irrelevant information, enabling visual reasoning without pixel-level noise compu- tational overhead.
(3) Learning perspective: adaptive rather than mandatory or heuristic alternation. As dis- cussed in Appendix E, existing synergistic or alternating approaches typically rely on fixed sched- ules or heuristics. In contrast, AD-Loop explicitly adaptive: our hybrid learning scheme with Stage-2 group-relative RL allows the model to intelligently decide when to invoke understanding or drafting. This results in an organic integration of the two capabilities and is key to realizing genuine synergy, rather than superficial alternation.
22

---

Published as a conference paper at ICLR 2026
[Question]: were any the objects the inttil frame that you con stil see in the second frome moved from their original position??
[Correct answer]: chair wea moved laft and towerds ds from the camera
[BAGEL w/ Think] determine it any objects were
from onggnal posttions between the tvo trames, need
carebdly compare the poattions the cejects in both images.lninn Inint. "'ddntlly key Obyects:"inn -*Chhir- The chair is
prominent object beth fr "*Gutorg:" The guitars are also noticsable in both fromes.n "Deek"" The deck and
its contents are another key siement 1nin2. "Compare Postions:"*nn ""Charc" In the hiss frams the chair posiiioned towards the loft sice of the imagh, closer to the camera. In the second frome, the chair is moved the (ight side of the image, further away from the camere. in "Guttars"" sl The quitars appear be in the same general area both frames, but their towar)
positions might have changed slightlyln DDDkk" The dest and contents seamn have shifted si siightly, Thh chair posttion. wes moved the right mway from camers Anini herefore the correct snower is: charr was moved right away the cameral vomeratrrtniinal srewer: 'chair was moved right and mwny from the camera
(lita
Inved. "*Conclusion
original
[anus-Pro) Prol Step Observ the nntiill fiams and the secand framna Vo- In the intial frame, there a chair on the left aidee of the mom In- the second fiame, the chair scill
left
slightly
the objects The appears general postion Snin cejects, intnStee Compane optione Ophion suggests
slightly
and towards the camera in Option suggests the chalr was moved right and away from posttion the second frame does not match the desorption moving lett or right and towards or away from the comera intnfinnll answer: chair was not moved from its original position.
inmera.ininStee 4: Deteriine the correct answer:n chai's
[Harmon] to determine it any objects were moved, we need to compare the bions of the items in both fromes. In the inital frame (lett side the image), there is chair that appenns to ba tacing towards the camera and has its legs spread out for stabiiity the second frame (right side of the imaget, the same chair seems to have beon repositioned with Its logs now bloser together and angled slighty away from the lamir the second frome, the chairs position relative to the but
has changec: Paatiann.ttoTherafooo
centered the intially (atvie moved right away from cameral)
indicates that chair indeed moved from its original
[Ours] decide whether an object moved betveen frames, we compare roletive positionaa key items, the iridal Frame the rolling chair sits beside puitar case on the flor [V-I], with severil gusters neaeyy and the table close to its right/in in the Frmma 2. only part the chair visible and appears behind the gutee case [V: TI, indicating a chift toward the camera and lettward reletive to the original layout; it's also forther from the table than before/nn Final onswerr was moved left and towards from the cameral]
Figure 10: Qualitative comparison across baseline unified MLLMs on multimodal understanding. Correct answers are marked in green, errors in red, and [V-T] denotes latent visual thoughts.
[Question]: AB is the diameter of OO. Point C point on AB. Point D on OO, AD - AC. Connect DC and extend tt intersect O0 at point E. Connect AD, AE, OE. ZBAD 30", degree measure of ZCOE is
[Option]: A:30"
B:35"
C: 40
4:45"
[Correct Answer]:A A
[BAGEL Think] Final answer:
[BAGEL w/ Think] find the measure of COE, we can use properties of circles and angles. Let's break down the problem step by step.inin1. **Identify the given information:
circle O such that AD- - ACAm AD - AC implies that triangle ADC is isosceles with AD - ACA. <BAD - An/v2. **Understand the properties of circlas:*\in -The angle subtended by a dia dnsmeterin a circie is right angle (90'). Therefore, LADB 30 An -innce AD AC, triangle ADC is isasceles, LADC ZACD.Inin3. *CClcculate <ADC:* Since <BAD 30" and ZADB m 90", ZABD 60 (since the angles in triangle sumn to 180"),An - In triangle ADC, ZADC LACD. Let's denote each of these angles as xAA Therefore, 2x + 30* 180" (since the sum of angles triangle ADC is 180").nn Sooving for x: 2x 150", s0 75'An -Tha,, LADC inscribed angle that subtends are AC, are AC is 2 75 - A50"An "The central angle <COE that subtends the same arc AC is equal to the measure of the arc, which is 150" In However, ZCOE the angle at the center of the circle, and since ZCOE is the central angle for are AC, is equal to the measure of the AC, which is AnV',, **Calculate the remaining angle:""n around point O An The angle ZAGE is the remaining angle that completes the circle/nn LAAE - 360 150* - Anin.. "Determine the correct option:**An The question asks for the measure ZCOE, which we found be 150 However, the options provided are A. 30', B. 35", C. 40", 45"Ann None of the options match 150', which suggests there might be misunderstanding in the problem setup or options provided. However, based on the given options and the logical steps, the closest and most reasonable snawer would be D. 45', as it is the only option that could be part of the correct calculation process. Annn answer: D.
AB is the diameter circle 0.In Paint C on AB.In Point
75t Anin4, *DDetermine <C Sinnc <ADC= 755, and <ADC
of
Since AB is the diameter, the total angle
[Janus-Pro] Step 1: Identify the key elements in the problem.1n An- O0 the -nrcle.nn AB the diemeter of Cn Point C a point on AB.An- Point D is an OO, AD AC. Connect DC and extend to intersect OO at point E.An- Connect AD, AE, OEAIn <BAD = AnlnStep Analyze given information.ln: Since AD triangle
isosceles. <BAD 30", <CAD 6D" An- Since AD = AC, <ADE= 60'.n- LADE + <COE = 180" AninStep 3: Calculate ACOE.M--a LADE 60" An- LADE ZCOE = 180 60' <0OE 180" An- 2COE 180* 60 2COE = 120", AninStep 4: Determine the correct answer.n- The degree measure of ZCOE is
120" Annnninnl answer: 120*
[Ours] <think-To answer this question, the image is depicted below: [V-T]in Let's think step by step: 1) Let O(0,0), 1,11,0), B(1,0). Then AB is the diameter. 2) 2BD 30 = are - 60", so D 1- (1/2, /3/2), 3) Triangle ABD is 30"-80"-90" right triangle, hence AD- (3. Given AC AD and lies AB, gat C= C= 0), 4) Line DC intersects the circle again at E= -1/2), Vector OC lies along the x-axis (0"). OE has angle -30". Therefore, <COE = 30", Athinko-answerr-inal answer: Aslanswer>
Figure 11: Understanding Comparison across UVLMs the math question answering task. Correct answers are marked in green, errors in red, and [V-T] denotes latent visual thoughts.
23

---

Published as a conference paper at ICLR 2026
[Question]: What does the tangram in the image show closer likeness to?
[Option]: A: dog's head
B: someeone flexin
C: mailbox
D: worman
Answerct C
[BAGEL w/o think] Fnnwer: B
[BAGEL w/ think] To determine what the langram in the resembles, need to carefully analyze the shapes and their arrangement. Vnim1. **Identify the Shapes"*: The tangram conslsts of seven pieces: five triangles, one square, and one parallelogram. These pieces are arranged to larger shape.Inin2: *AAnalyze Arrangemen** positioned on the left side.An The square located at the top centerAn The parallelogram is bottom center.in The remaining triangles are arranged around these centrel Intnp *Compare with Options"*An - O*Oppio A (a dog's head)** arrangement does not resemble dog's (someone flexin)**: The shapes not form
largest triangle is
shapes do not form typical features a dog's face.in "OOption flexing their muscles. The arrangement lacks typical features of human figure.In **ptionn [mailbox)**: The arrangement does not resemble a mailbox. The shapes do not form the typical rectangular and triangular features of e mallbox.\n "*Option (woman)"* The errangement the shapes does resemble women. The largest triangle be
the head, the square the upper body, and pamallloggam the lower body. The other triangles can be interpreted as arms and Ings. *CConclusion** Based on the analysis, the tangram in the image most closely answer: D.
resembles a woman,
[Janus-Pro] Pro] answer question, set of gsometric shapes that are typically used in tangram Anz.les.An.. The shepes arranged that resembles dog's head/In3. The yellow shapes form the outline of the dog's head, with the pink shape representing the bodyAAn. The cyan shape is dogsanesdee on analysis, tangram in the image
analyze tangram shapes
arrangement innge.inin1. image shows
positioned resemble
or dog's head. Aniiiinal enswer: A
likeness
[MindOmni] To determine what the tangram in the image shows claser likeness to, let's analyze the shapes and their parallelogram..rr
errangement:/nkee
image consists several geometric shapes: triangles, square, erranged in a that suggests recognizeble form.In3. The large yellow triangle at the top left resembles earAn4. The two yellow squares and the yellow triangle below them form a shape that looks like face with a mouth.Ins. The pink shapes at the resemble hody.ininConssddriigg
shapes
shapes most closely resembles face with mouth, which is cheracteristic of dog's head. VninFinll answer: A. dog's head
arrangement
[Ours] <thinkeThhe initial step involves examining regions of the image that are marked by specific colorinAs illustrated, these regions resemble the shape
pole [V-T],nNeet, sttention shifts segments highlighted by color. shown, these take form similar box [V-T].nnFinally, as depicted, bear likeness to flag [V-T] An Taken together, the overall tangram arrangement most closely aligns with the shape of a mailbox (V-T] </thnkk> Inin <answer> Final answer: Ox/answer>
focus the portions rendered the remaining Anlors.AnThese sections,
Figure 12: Understanding Comparison across UVLMS on multimodal content reasoning task. Cor- rect answers are marked in green, emrors in red, and [V-T] denotes latent visual thoughts.
24

---

Published as a conference paper at ICLR 2026
Prompt
Janus-Pro
Bilp-3o
Harmon
MindOmni
BAGEL
Ours
Newty aggs
kangarop sitting
freezes stop bear emerges
away. The image should highlight
hiker's emotional response
plants watering gardener with
A person checks their watch frequently under en umbrella while standing in pouring roin.
A banane placed in a fruit for throe days.
The dish of a person finishing cating
Alog reduced embers in dying fire.
Figure 13: Generation Comparison across UVLMs.
25

---

Published as a conference paper at ICLR 2026
MindOmni
MindOmmi
BAGEL think
BAGEL
Prompt
Ours
think
Generate image explain the cause of fellowing scene: collapsed sandcastle surrounded by footprints.
Generate an image following scene: shattered cookie surroundad by biscuits kitchen floor.
A birdbath centered between two gerden benches. squirral faces birdbath bench, while cat crouches right bench, gazing at the squirel..
bird parched on top dog.
grill fully burned briquettes.
A osmenthus tree during spring
Figure 14: Generation Comparison on MindOmni Xiao et al. 2025b) and BAGEL (Deng et al.,
2025) with/without Think process.Ous proposed method achieves more accurate generation results.
26

---

Published as a conference paper at ICLR 2026
BAGEL
BAGEL w/ think
Prompt
wlo think
Ours
Plnase drow the shape the '?' box at the top right corner
according the pattern.
Heat the corn kemeels picture until they
Changes organism after albinism image
O
Correct unreasonable parts of the bicycle in the image
Appearance of plant lacking weter the image
Figure 15: Editing Comparison on BAGEL (Deng et al. 2025
with/without think process.
27

---

Published as a conference paper at ICLR 2026
[Question] What is the most prominent feature in the image? [Option] A: The skyline, B: The golf course, C: The trees, D: The person
[BAGEL w/o Think] Final answer: C.
The image is taken from perspective that places the golf course in the foreground [V-T], with the and skyline serving as secondary elements [V-T]. The golf
[Ours]
the central focus, with the green and the golfer being the main subjects [V-T] The trees frame the scene
and add depth, but they are not the primary focus [V-T] The skyline, while visible, is in the
background and does not dominate the image [V-T] Therefore, the most prominent feature is the golf course. Final answer:
[Question] Find the greatest common divisor $3339$, $2961$, and $1491$.
[BAGEL w/ Think] Let's compute -see-by-stee using the Euclidean algorithmm. An1n1. Find
gcd(3339, 2961)\33339 2961 378 now compute ged(2961, 372)11 378 378
=26461n2961 2646 315 ged(378, 315) 315 6311ged(315, -3nng-3
63\nSo, 63 23 63 - 1449\n1491
gcd(3339, 2961) 63\nhn2. Now compute gcd(63, 1491)\n1491 +
1449 42\nNow gcd(63, 42):An63 42 210 ged(63, 1491) 21nnnniinnall Answer:\n21
[Ours] To find the greatest common divisor, we first determine the greatest common divisor of the first two numbers: $3339 2961)$. \n$3339 ledot 2961 378SIn$2961 = ledot 378 !odot 318 60$1n$318 5 lodot 60 + 18$1n$60 3 Acdot 18 + 6$1n$18 = \odot 6 O$inTherefore, $1gcd(3339, 2961) 6$. \n [V-T] Next, we find the greatest common divisor the result from the previous step (6) [V-T] and the number (1491): $1god(6, 1491J$. In$1491 = 248 lcdot 6 + 3Stn$6 Acdot 3 + 0$in [V-T] Therefore, $igcd(6, 1491) = 3$.
+ 318$1n$378
InThus, the greatest common divisor of all three numbers is $lmathbf(3]S. AnFinal answer: 3.
Figure 16:
Failure cases illustrating over-drafting on simple tasks, well as challenges on problems that require strictly logical mathematical reasoning.
Ours
Gemini
GPT 5.1
Lunar doorway with red "3Sl paper pasted upside down.
A close up hands holding book; title area blank.
balloons row; even pasitions are blue, odd positions
Figure 17: Failure cases showing that the proposed method still struggles with rendering scene text, well fine- graingr body details and numerically precise content.
28


Reviews:
[{"review_id": "c6hgUKaW3A", "reviewer": "Reviewer_QySp", "summary": "Existing approaches primarily focus on architectural unification while overlooking the importance of explicit interaction between the two capabilities during task solving.\nThis paper introduces the interleaved Analyzing–Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations.", "questions": "- When constructing the dataset, did you control the number of visual drafts generated during the thinking process? What were your design choices and constraints here?\n- In the final training loss (Equation 2), it appears that all loss components are simply summed together without weighting. Could you clarify whether any weighting scheme was considered or tested?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a two-stage training strategy: (1) supervised learning on interleaved thought data to initialize the alternation, followed by (2) reinforcement learning to promote adaptive and autonomous control.\n- The paper conducts extensive experiments and ablation studies to validate the effectiveness of the proposed method.\n- AD-Loop consistently improves performance across standard benchmarks for both understanding and generation, and further showed the adaptability of the proposed method on other unified VLM.", "weaknesses": "- The definition of the inter-group reward in Equation (6) is unclear. What does $m$ represent? Does it indicate whether the trajectory is AD-Loop-enabled or not? Additionally, in Equation (7), the intra- and inter-reward terms on the right-hand side seem to be missing the superscript $m$?\n- In the ablation study (Table 3), it is unclear whether the paper trained three additional variants corresponding to different thinking strategies and evaluated them separately, or whether they used the final model but selectively disabled certain thinking capabilities.\n- In Figure 4, what does unified-R1 refer to? Is it AD-Loop?\n- Although Figure 5 presents examples of latent visual thoughts, it would be beneficial if the paper also included qualitative examples illustrating the complete thinking trajectories along with visualizations of the generated latent visual thoughts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uhcGZfybvC", "reviewer": "Reviewer_h8QD", "summary": "The paper introduces the Interleaved Analyzing–Drafting Loop (AD-Loop), a framework for unified vision-language models that alternates between textual analysis and visual drafting during reasoning. Through supervised fine-tuning on interleaved reasoning traces and reinforcement learning using Group-Relative Preference Optimization (GRPO), the model learns to decide when to analyze or draft. Applied to UVLMs (BAGEL-7B), the method improves both visual understanding and image generation, setting new state-of-the-art results on several benchmarks.", "questions": "1. The qualitative examples focus on successes. Could you provide and analyze a few failure cases? Specifically, are there types of problems where the AD-Loop consistently fails or even degrades performance compared to simpler reasoning strategies?\n2. Could you provide more detail on the heuristics used to \"reorganize\" and \"synthesize\" the AD-Loop traces from existing datasets (Sec 3.2)? How did you ensure the quality and logical coherence of these generated traces?\n3. What is the typical computational overhead (e.g., latency, FLOPs) of invoking the AD-Loop for a complex query compared to a standard generation? How does the number of interleaved steps affect this cost?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The interleaved analyzing–drafting mechanism establishes a tighter synergy between vision understanding and generation than prior “unified” models. This new paradigm addresses a clear gap by turning generation and analysis into mutually reinforcing steps rather than independent skills.\n\n2. The two-stage training strategy (SFT of interleaved reasoning followed by RL) is well-motivated. This pipeline enables the model to learn the complex analyze-then-draft procedure in a guided way, then optimize adaptively via reinforcement learning. The use of preference-based rewards and an autonomous switching policy shows a way to balance the dual objectives.\n\n3. AD-Loop yields consistent improvements across diverse benchmarks for both modalities. The approach outperforms baselines on multiple datasets and even when applied to different underlying model architectures, indicating broad applicability.", "weaknesses": "1. The framework introduces significant complexity in both training and inference. It requires a specialized two-phase training (including an RL stage), and at runtime the model must perform multiple analyze–draft iterations per query. This likely incurs substantial computational cost and latency compared to standard one-pass models. The paper does not discuss inference speed or resource requirements, which raises practical concerns for real-world deployment.\n\n2. The method relies on a curated interleaved reasoning corpus for supervised training. It is not fully clear how generalizable these textual–visual “thought” sequences are. For example, if the SFT dataset is biased or lacks certain complex reasoning patterns, the RL stage will be unable to discover them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zmkptxJrVJ", "reviewer": "Reviewer_PeBS", "summary": "The authors propose a new method to improve the capabilities of unified vision-language models. They propose a new approach that combines analytic processes and drafting thoughts (textual and visual thoughts).  They design a two stage strategy for training consisting of supervised learning and RL.  They show performance improvements for benchmark tests for understanding and generation.", "questions": "see weaknesses", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The authors address an important issue with what seems to be an innovative approach.\n\nThe comprehensive evaluation for understanding and generation is also a strength.\n\nAblation of the thinking types section was also a strength.\n\nGood discussion addressing interesting questions spanning extensions into other MLLMs,  whether visual thoughts should be derived from understanding vs. the generation encoder, and the visualization of implicit visual thoughts.\n\nVery thorough methods section with detailed descriptions.", "weaknesses": "I was unclear about how novel the work is. There are many publications using visual representations (imagination) to augment language reasoning. Please expand on how this is different.\n\nPlease show some examples in Figure 7 where the proposed method did not generate better results. This would also be interesting. \n\nThere were parts where I had a difficult time following the methods and even the evaluation.  The paper is dense, but other reviewers more expert in the field might have an easier time following the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y5bMqCYtu2", "reviewer": "Reviewer_tVfz", "summary": "This paper proposes a new thinking paradigm called the Alternating Analysis–Drafting Problem-Solving Loop (AD-Loop), designed to achieve genuine synergy between the understanding and generation capabilities in Unified Vision-Language Models (UVLMs). The AD-Loop enables the model to iteratively refine its comprehension and output by dynamically alternating between textual thoughts (for analysis and reasoning) and visual thoughts (for sketching and spatial layout).", "questions": "Please elaborate on the specific composition and weight distribution of the internal reward (Intra-reward) and external reward (Inter-reward) in S2. In particular, how is the internal reward designed to encourage “intelligent and autonomous” alternating decision-making?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The AD‑Loop paradigm is proposed, emphasizing explicit, dynamic, and reciprocal interaction between understanding and generation. This concept goes beyond mere architectural unification, offering a new approach for achieving deeper levels of general multimodal intelligence.", "weaknesses": "Existing MLLM research has begun to explore the alternation or integration of textual and visual information. This paper needs to more clearly define the fundamental differences between AD-Loop and such works — specifically, how AD-Loop achieves genuine synergy rather than merely alternation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)