Understanding vs. Generation: Navigating Optimization Dilemma in Multimodal Models
Authors: Sen Ye
Mengde Xu
Shuyang Gu
Di He
Liwei Wang
Winston Hu
Venue: ICLR 2026 Poster (2026)
Keywords: Unified Multimodal Large Models
Text-to-image generation
Reasoning Models
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Current research in multimodal models faces a key challenge where enhancing generative capabilities often comes at the expense of understanding, and vice versa. We analyzed this trade-off and identify the primary cause might be the potential conflict between generation and understanding, which creates a competitive dynamic within the model. To address this, we propose the Reason-Reflect-Refine (R3) framework. This innovative algorithm re-frames the single-step generation task into a multi-step process of "generate-understand-regenerate". By explicitly leveraging the model's understanding capability during generation, we successfully mitigate the optimization dilemma, achieved stronger generation results and improved understanding ability which are related to the generation process. This offers valuable insights for designing next-generation unified multimodal models.

Published as a conference paper at ICLR 2026
UNDERSTANDING VS. GENERATION: NAVIGATING OP- TIMIZATION DILEMMA IN MULTIMODAL MODELS
Sen Ye!
Mengde Xu²
Shuyang Gu
Di He
Liwei Wang! 13344
Han Hu
State Key Laboratory of General Artificial Intelligence, Peking University Tencent
Center for Data Science, Peking University Center for Machine Learning Research, Peking University
ABSTRACT
Current research in multimodal models faces key challenge where enhancing generative capabilities often comes at the expense of understanding, and vice versa. We analyzed this trade-off and identify the primary cause might be the potential conflict between generation and understanding, which creates a competitive dy- namic within the model. To address this, propose the Censon-Reffect-tee (R3) framework. This innovative algorithm re-frames the single-step generation By explicitly leveraging the model's 's understanding capability during generation, we success- fully mitigate the optimization dilemma, achieved stronger generation results and improved understanding ability which are related to the generation process. This offers valuable insights for designing next-gent next-generation unified multimodal models. Code is available at https://github. http://gthuu.cooo//seeeeee www./ooooo///eeeeee see-//..-
task into multi-step process of "generate- wewerrrate-undestaadd--eee understand-regenerate
INTRODUCTION
For decades, the pursuit of Artificial General Intelligence (AGI) has been central goal in Al research. A key step toward this vision is the development of unified multimodal models capable of both understanding and generating visual information, much like humans do (Team et al.. 2023 OpenAl 2025). However, recent advance in large-scale multimodal learning points to a core dilemma that understanding and generation are hard to improve simultaneously (Henighan et al.) 2020: Wang et al., 2024; Wu et al. 2025bb Zhang et al. 2025a). For instance, models fine-tuned for high-fidelity image synthesis, such as those based on diffusion architectures, often struggle with tasks requiring precise visual understanding, including object counting spatial reasoning (Cho et al., 2023 Ghosh et al.)
2023 Wei et al., 2025). Conversely, models optimized for tasks like visual question answering (VQA) or dense captioning tend to exhibit weaker creative and generative performance compared to their
generative counterparts (Dong et al. 2023 Xie et al. 2024 Wu et al.) 20255).
A number of efforts have been made resolve this tension. Some researchers argue that different tasks require specialized tokenizers (Wu et al., 2025b Chen et al.) 2025), and thus propose unified tokenization schemes (Team 2024 Ma al.) (0255) to harmonize representation across modalities. Others attempt to disentangle understanding and generation by designing novel architectures that allocate separate capacity each function (Liang et al. 2024 Deng al. 2025 Zhang et al. (025a)). While these approaches achieve partial success, we argue that the crux of the conflict comes from the different training objectives. The generative objective typically maximizes the likelihood of samples under the data distribution, goal that can be optimized without the understanding ability. As result, the model's 's capacity may be monopolized, in competition with the robust understanding required. This raises fundamental question: should generation actively incorporate the model's 's understanding of the underlying semantics?
In this paper, we introduce the Geason-Reflect-Bee generation as a multi-step process rather than a single-shot mapping. Instead of treating generation and understanding competing objectives, R3 explicitly integrates understanding into the generative loop. By first reasoning the user request and producing an initial draft, then reflecting whether this generated result meets the request, and finally refining the output according the reflection,
(R3) framework, which re-conceptualizes
Cooresponning Author.

---

Published as a conference paper at ICLR 2026
the model transforms understanding from passive evaluation task into active component of generation.
Analogous to painter's creative process, our framework unfolds in distinct stages. In the Reason stage, the model first analyzes the user's intent to conceptualize the final image. It enriches the initial prompt by imagining and incorporating various fine- fineggaiined details, producing explicit textual blueprint before synthesizing an initial draft. Recognizing that high-fidelity generation from complex prompts rarely achievable in single attempt, the framework then enters iterative Reflect-Refine loop. Here, the model evaluates its output against the original prompt-a process that demands strong multimodal understanding. If the output aligns well, the procedure terminates; otherwise, the model formulates corrective textual instructions and refines the image accordingly. This self-correction cycle continues until satisfactory alignment reached, with the model itself deciding when to stop. The entire process is trained end-to-end using an outcome- based reward signal derived from the final image quality. To further enhance efficiency, we introduce an immediate rollout strategy that accelerates convergence without compromising performance.
Our experiments demonstrate that incorporating reflecting and refining into the generative pro- cess enables the model to effectively leverage its understanding capability, leading to substan- tial improvements across multiple text-to-image benchmarks. Moreover, embedding reflection within generation not only enhances output qual- ity but also exercises the model's understand- ing ability, thereby preventing the degradation typically caused by modeling capacity compe- tition. On tasks aligned with generative con- tent, we even observe notable gains in under- standing- for instance, counting accuracy im- proves from 79.3 to 84.6. Overall, the proposed R3 framework reconciles the long-standing con- flict between generation and understanding: it achieves stronger generation while simultane- ously preserving, rather than diminishing the understanding ability. This establishes promising path forward for the development of future unified multimodal models.
Generador ACC
90
Undenntnrding
80
$0.4
79.8
82.4
80.7
80.0
Accuracy
60
57.5
50
47.5
50.0
32.5
30
20
Gen+Undd
Figure I: Fine-tuning BAGEL exclusively on gen- eration understanding degrades the complemen- tary capability. Naive co-training shows minor gains, whereas our proposed method demonstrates significant improvement in both. Results are re- ported on counting subset of GenEval++.
Our contributions can be summarized as follows:
We provide systematic analysis of the conflict between generation and understanding in multi- modal large models, and identify its root cause: traditional approaches treat the two as independent tasks, which compete for model capacity and lead to trade-off where one improves at the expense of the other.
Building on this insight, we propose the Reason-RReficttReffine (R3) framework, which de- composes generation into a structured generiie-unneertan---eeenni process. By explicitly incorporating understanding into the generative pipeline, R3 mitigates the conflict caused by separate optimization, yielding stronger generation while preserving understanding ability.
Extensive experiments verify the effectiveness R3: by fully leveraging the model's own under- standing ability during generation, R3 not only achieves superior generative performance but also avoids the degradation of understanding. These findings shed light on the design of future unified multimodal models and point to new data strategies for balancing generation and understanding.
METHODOLOGY
2.1 UNIFYING GENERATION AND UNDERSTANDING
A key challenge in developing powerful, unified multi-modal models lies in effectively integrating both generation and understanding capabilities. As shown in fig.[] fine-tuning a model exclusively on of these tasks degrades its performance the complementary task. This suggests competitive
2

---

Published as a conference paper at ICLR 2026
The description of The target image How -
Unified Understanding and Generation MLLM
<thinko The model should generate The refined prompt </thinlo
<tinno The user wants a photo of Current image </thinko
Add two more cats.
repeat
reasoring stage
Current matches the target description. No further edit needed.
reflect and refine stage
Figure The inference pipeline our Reason- Reasoning to produce an initial plan and image. It then enters an iterative Reflect-Refine loop, assessing output and making corrections until the image aligns with the user's prompt or a stopping condition is met.
framework. The model starts by
relationship between the two tasks, where shared model parameters are optimized for one objective the expense of the other. Furthermore, naive training on a mixture data provides only negligible performance gains. This observation supports our core hypothesis: the optimization trajectories for generation and understanding are not inherently aligned. The two tasks are not coupled tightly enough to benefit from mutual training, suggesting they are driven by fundamentally different optimization dynamics. This observation leads us to crucial question: goals of generation and understanding, thereby reducing the conflicts that emerge when training single model on both?
possible to align the optimization
A possible solution is to intrinsically embedding visual understanding into the generative process. This paradigm shift ensures that the improvement of a model's generation capability is inherently dependent on its understanding ability, thereby preventing the common issue where training for one task degrades the performance of the other. To realize this vision, we introduce the Reason-Reflect- Refine (R3) framework( fig. ()). novel approach that recasts image generation of reasoning, reflection, and refinement. By embedding image understanding as a core component of generative ofain-of-thought enable the model to critically assess and progressively improve its own output, fostering synergistic relationship between understanding and generation.
iterative process
2.2 FRAMEWORK OVERVIEW
We build our framework on top of unified multimodal model, BAGEL (Deng et al.) 2025) (parametrized by 0), which is capable of image understanding, generation, and editing. The generation process is Yormulated as a sequence alternating text and image generation steps:
'' t", 1" To (-lc)
where e is the initial urov-idovided prompt, and at each step, the model generates text t' auto- regressively image I' by progressive denoising.
To make this process computationally tractable and modular, we decompose it into three distinct, alternating tasks. Assuming a Markovian property, we model the generation sequence of these specialized tasks:
1. Reason:
The process begins with the model expanding on the input prompt e to generate a more detailed, reasoned plan t!. The plan is expected in the format "<think>-lan</hhhnnk Then, the model synthesizes an initial image I' according to this plan. This modeled by the joint probability no (T ( (c)c) no (T (
While oo (c) is standard language
3

---

Published as a conference paper at ICLR 2026
Full-Trajectory RL strategy
Reasoning
Reflect-Refine
Reflect-Refine
prompt
Reasoning
Reflect Refine
Reflect Refine
advantage
Reasoning
Reflect-Refie Refine
Reflect- Reft-e
Tree-RL strategy
Reasoning.
achermage
Reflect-Refing
Reflect- Refice
advantage
Reasoning
Refine
Refine
prompt
Reasoning
Reflect Refine
Reflect Refine
same policy group
Figure 3: The training procedure, which alternates between optimizing the Reason policy and the Reflect-Refine policies. The replay buffer, populated by the Reason stage, provides on-policy data for training the subsequent stages.
modeling policy, and the diffusion policy my (7 [t_, can be calculated using stochastic differential equations (SDEs), similar to (Ma et al. 2024 Liu al. 2025)
2. Reflect: Upon obtaining an initial generated image, the model is then required to assess its alignment with the user's original intent e. This critical self-assessment process is termed reflection, which can be formally expressed as (o(t+=11/3,)) deemed satisfactory, the model is designed to produce definitive termination signal: "No further edit needed." However, if the output is still deficient, the model performs critical introspection, identifying the discrepancies between the current image and the desired objective. This introspection culminates in the generation of refined editing instruction ef+1, which serves as guidance for subsequent iterative improvements. To ensure structured and consistent output, employ a system prompt that strictly enforces the format: reiieniooeetectionn</hhiii instruction."
In cases where the generated output is
Refine: Subsequently, the model executes the generated editing instruction g++1 modify the previously created image I', and produce refined output [ This refinement step is formally modeled as a conditional generation process:Ty and-refinement loop is performed iteratively, forming chain-of of-thought that continues until the model's 's internal assessment confirms that the generated image satisfies all aspects of the user's s request.
I²). The entire reflection-
2.3 TREE- RL STRATEGY
The full, multi-turn trajectory can be conceptualized as a sequential process: Reason -> Reflect -> Refine -> Reflect Refine... This sequence can be theoretically framed as the chain-of-thought process of image generation, which could be directly optimized using reinforcement learning (RL). However, training end-to-end reinforcement learning (RL) models presents several challenges. First, the large number of training iterations can lead to error accumulation, causing instability. Second, the lack of explicit supervision for intermediate steps results low training efficiency. To overcome these issues, we propose tree-based RL strategy that provides clear supervision for the outcome of each intermediate step.
As illustrated in fig. 3 we split the trajectory into Reason stage and Reflect-Refine stages. Each stage populates its result (result image I' and current reward) to the next stage as initial condition. To enhance training efficiency and speed up convergence, we employ
importance sampling strategy when selecting for previous stage's results: Sampling more samples with diverse rewards. This strategic focus
correction allows the model learn and improve effectively, without
4

---

Published as a conference paper at ICLR 2026
compromising its overall task completion capability. All policies optimized with the GRPO loss function, as described in Sec As shown in fig. by employing this tree-based rollout strategy. model can be trained effectively on long, complex generation chains, allowing for progressive improvement through multiple rounds of refinement during inference.
2.4 STAGE- REWARD
We employ different reward models evaluate the various stages of our reinforcement learning (RL) training. Specifically, as described in sec- tion 2.2 the reasoning stage involves two policy processes: To ( c) and Ty (t)cc..
Ful strategy
staslegy
0.6
Traiming Roward
First, we measure the quality of the initial gener- ated image I2, using pre-defined reward model V (a pre-trained Eision-Langaae Model). This model outputs a scalar score V = v(4,.c)
0.2
Stops
[o, 1], which represents the alignment between the image and the prompt. We then define the rewards for each policy as follows:
Figure 4: Training reward curves of the Tree-RL. versus Full Trajectory RI strategies. The reward curve for Full Trajectory RL substantially lower than that of Tree-RL. This performance gap is at- tributed the high variance and noise introduced by the long trajectories inherent in the full trajec- tory approach, which complicates the advantage assignment problem.
For the diffusion generation stage No (1' (1 lt 1 the reward defined as jjddiflusion = Vj.
For the text generation stage (t (c), the re- ward rext = V, + formart. where format measures the accuracy of the generated format.
For the Reflect-Refine stage, the model generates textual reflection based on the image ī and its reward V from the previous step. This process leads to one of two distinct outcomes:
1. Correction Required:
the image does not align well with the prompt c (i.e., V. < 1), the model outputs precise editing instruction ez- The Refine step then generates improved image Ij. which is evaluated to get a new image reward Vy-
Correction Unnecessary: the image already satisfies the prompt (i.e., V 1), the model must output the specific phrase "No further edit needed." This serves termination signal for the process.
To encourage meaningful reflection and accurate termination, we use a correctness metric to evaluate the model's output. For an image with a reward of V the metric is defined as:
Vj I(ee
if i < if V =
cj=
"No further edit needed")
where I(-) is the indicator function. This metric is designed to reward two key behaviors: measurable improvement for flawed images (where V, V), and correct termination for images that already meet the prompt's 's criteria.
The rewards for the reflection and refinement steps then based this metric:
rellecttoon Cj + formuts releieeeennt
C
While our reinforcement learning objective does not directly optimize understanding tasks, the model develops robust visual comprehension by being trained evaluate image-prompp alignment via the reflection reward.

---

Published as a conference paper at ICLR 2026
3 EXPERIMENTS
3.1 CO-EVOLUTION OF UNDERSTANDING AND GENERATION
To validate the co-evolution of understanding and generation capabilities within our framework, conduct controlled experiments on the GenEval++ benchmark (Ye et al. 2022).
Our training prompts are generated by randomly combining elements from the official templates, ensuring no overlap with the test set. We employ Qwen-2.5-VL-777 as our reward model. By default, our method includes an initial reasoning stage followed by four reflection- refinement stages. Following the original benchmark setting, we use GPT-4.1 to evaluate generation quality.
Experimental Sctup:
Evaluation of Understanding: To assess the model's understanding capabilities, we introduce two novel evaluation protocols: Compositional Visual Question Answering (VQA) and Image-Text Alignment (TTA). The VQA task probes the model's ability to perceive compositional elements in its generated images. The ITA task evaluates its capacity to assess the overall quality and prompt- alignment of the generated images.
For both tasks, we followed a standardized protocol. We first generated corpus of images using a powerful, pretrained text-to-image generation model. Subsequently, used the Gemini-2.5-Flass model to create the gold-standee ground truth for both VQA questions and ITA assessments and conduct careful human alignment check to ensure the quality of annotation. Finally, evaluated our model's 's performance by calculating the similarity between its output and the annotated ground truth labels. This rigorous evaluation framework allows to directly measure the effectiveness of our approach. We provide detailed description in section A.3
Table -oslloiion-follooingg generation ability on the GenEval++ + benchmark, evaluated by GPT-4.1. Bold indicates the best result. Findicates our framework with only the reasoning stage. Green arrows indicate improvement over the BAGEL baseline.
Method
Color
Count
Color/Count
Color/Pos
Pos/Count
Pos/Size
Multi Count
Overall
GPT-4o OpenAL
2025
0.900
0.675
0.725
0.625
0.600
0.800
0.850
0.739
FLUX.1 1- 1-Kontext (Labs
2024
0.425
0,500
0.200
0,250
0.300
0.400
0.325
0.343
FLUX. I-dev (Labs
2024
0.350
0.625
0.275
0.200
0.225
Janus-Pro Chen ct al
2025
0.450
0,300
0.125
0,300
0.075
0.350
0.125
0,246
T21-RI
Jiang al 2025
0.675
0,325
0,200
0,350
0.075
0.250
0,300
0,311
Echo-4u
2025
0.800
0.775
0.625
0.800
0.625
BAGEL Deng et al.
2025
0.325
0.600
0.250
0.325
0.250
0.475
0.375
0.371
BAGEL Ourst
0.500
0.650
0.600
0,650
0.550
0.600
0.600
0,593
10.22
BAGEL+
0.675
0.725
0.750
0.575
10.32
Table 2: Evaluation of understanding capabilities our proposed ITA benchmarks. All scores reported as accuracy (%). tindicates our framework with only the reasoning stage. Green arrows indicate improvement over the BAGEL baseline.
ITA
Color
Count
Color/Count
Color/Pos
Pos/Count
Pos/Size
Multi Multi-Cuunt
Overall
BAGEL
60.63
58.54
45.42
63.54
53.96
80.83
61.50
60.60
BAGEL Oursf
60.42
59.38
47.71
63.75
55.63
81.46
63.96
61.76 11.16
BAGEL Ours
69.58
67.50
69.79
72.29
76.04
83,33
75.00
73.37 112.77
The results in table T the enhanced generation capabilities of our framework. Even compared with the SOTA method Echo-4o (Ye et al., (025) which finetunes related data, our method still achieves 1% point overall improvement. And in complex situation like Multi Coutt, our method is significantly better then Echo-4o.
As shown in table and table 3 our approach also yields substantial gains in the ITA and VQA understanding tasks. Notably, the reflection-refinemeet process is critical. While baseline with only the reasoning stage improves GenEval++ scores, our full framework adds a further improvement of nearly 10% points. This effect is more dramatic in understanding tasks, where the reasoning-only

---

Published as a conference paper at ICLR 2026
Table 3: Evaluation of VQA capabilities. All scores are reported as accuracy (%). findicates our framework with only the reasoning stage. Green arrows indicate improvement over the BAGEL baseline.
VQA
Color
Count
Color/Count
Color/Pos
Pos/Count
Pos/Size
Multi Couni-Count
Overall
BAGEL.
91.74
79.30
88.28
82.93
85.10
92.45
86.48
BAGEL Oursf
91.67
76.12
88.76
78.71
83.29
84,98
93.45
86.72
10.24
BAGEL + Ours
93.95
84.63
91.15
84.09
86.06
86.54
94.50
89.63 13.15
One frog jumped in of three suitcases, showcasing playful moment in the scene.
Three ducks quacked by pond, gathered at the side, basking in the serene water's reflection.
Evidence of currently present invisible elephant, such as large footprints in the mud, broken branches, and disturbed water pond
A picture of birthday featuring colorful mermaid tail design, with the text on it: "HAPPY", "BIRTHDAY", "OUR", "MERMAID".
miniature diorama scene using photography technique, depicting chibi-style scene of "Sun Wakonn's 's battle with the White Bone Spirit"
HAPPY
BATTLLY
Bagel
CUR-..
MERMAID
Ours
Figure 5: Qualitative comparison between Bagel and our results.
baseline offers minimal gains (1.16 on ITA, 0.24 on VQA). whereas our full framework achieves significant improvements (12.77 on ITA, 3.15 on VQA). These findings highlight that the reflection- refinement stage is essential for improving generation and is the key to unlocking the model's s understanding abilities. We also provide some qualitative results in fig. For more detailed visualization of the multi-round generation process, please refer to the section A.6
3.2 ABLATION STUDIES
Influence of Trajectory Length. We investigate the impact of trajectory length on performance during both training and inference. For training, detailed in table [ trajectory length of two (one reasoning stage and one reflection-refinmment stage) achieves an optimal balance between computational cost and performance.
For our inference-time analysis, we evaluate the impact of trajectory length by setting maximum number of reflection-refiemment turns. For each sample, the evaluation terminates in one of two ways:
Table 4: Impact of training trajectory
Table 5: Cross-topic evaluation GenEval++. Models are trained on specific category and tested on all cate- gories.
length on generation (GenEval++ Vali- dation Reward) and understanding (ITA). "RR" denotes a stage.
Test
Counting
Color
Position/Size
Train
Rollout Stage
GenEval++
ITA
Pretrained
62.71
60.63
80.83
Reason
0.654
62.83
Counting
71.25
60.63
81.46
Reason 1x RR
0.729
74.49
Color
62.50
66.25
81.25
Reason 2x RR
0.732
74.76
Position/Size
60.00
60.00
83.33
7

---

Published as a conference paper at ICLR 2026
GenEval
GenEval++
TIF
1.02
1.725
0.93
0.39
1,700
0.675
0.689
0.652
0.689
85.0
81.2
$1.9
02.6
82.0
1.755
0.97
0.55
0.956
9,690
93.0-
0.944
3.25
790-
75.9
0.593
1630
750-
6.41
Mi
0.32
maccning
neeriing 12480
retoning 26
renning 30
metoning 44
Inkootng 2 1x:8
anbosboy Rd
romening Sr
rescoting
newarong J.
Artoning 28
Wokiing 3r
wesoting 44.08
Figure 6: Inference-time scaling effect across the GenEval, GenEval++, and THI benchmarks (left right). Performance is shown as a function of the maximum allowed reflection-refieemen turns.
GenEval++ Reward
GenEval++ VQA Accuracy
0.500
0.7
Ours wio fect Refire Training
0.6
0.890
0.5
6.4
0.880
0.875
Ours
0.2
Ours wie Refiect Aetine Training
0.865
200
Staps
Stops
Figure 7: The evolution of generation and understanding abilities in the training process. The left figure shows the generation accuracy (measured by Qwen-2.5-WL-722B) and the right figure shows the VQA accuracy.
either the model signals completion by outputting "No further edit needed," or it is stopped upon reaching the predefined maximum number of turns. illustrated in fig. 6 the results across the GenEval (Ghosh et al.) 2023), GenEval++ (Ye et al. (025). and TII (Wei et al. (025) benchmarks show a consistent pattern. The most substantial performance gain occurs after the first reflection- refinement turn. While further improvements are observed with additional turns, the performance generally saturates, reaching peak when the trajectory length extended to four or five turns.
Capabilities Evolution During Training. fig. demonstrates how our proposed framework drives the simultaneous improvement of generation and understanding capabilities. In the first 150 steps, our model's generation accuracy follows trajectory similar to the reflection-free baseline, while the VQA accuracy shows minimal change. This suggests that the initial training primarily focuses on basic generative mapping without deepening internal understanding. However, beyond the 150-step mark, the reflection-refinemenn mechanism begins to yield observable results. At this point, the model's understanding ability starts rise noticeably. This gain in understanding does not occur in isolation; it is directly associated with a subsequent acceleration in generation accuracy, allowing our model achieve superior performance relative to the baseline. Ultimately, the comprehensive results validate the effectiveness of our framework in mitigating the optimization dilemma and demonstrating that explicitly integrated understanding is key to unlocking superior, unified multimodal performance.
Generalization of co-evolutionary Learning, A key question is whether the co-evolution of un- derstanding and generation capabilities generalizes to broader domains beyond GenEval++. While rigorously measuring this co-evolution would require specialized evaluation protocols, as we con- structed for GenEval+++, we can assess the impact on general generation capabilities. Our results on the TIIF benchmark ( demonstrate that our framework yiclds significant improvements in general-domall setting, suggesting that the benefits of our approach do transfer.
To further probe the nature of the learned understanding, we conducted cross-topic experiment within EvnEval++ We trained our model on specific attribute categories (e.g., color, counting) and evaluated its understanding
all categories. The results, presented in table [ indicate that the improvements in understanding are localized to the domains the model was trained on. This finding suggests that current framework leams domain-specific understanding. While this specialized
8

---

Published as a conference paper at ICLR 2026
Table 6: Quantitative evaluation results on THF testmini benchmark (Wei et al. 2025). Results are evaluated by Qwen VL2.5-72B. indicates our reproduced results using the official repository and checkpoint. The best results are bolded.
Rasic Fallowing
Advanced Following
Model
Overall
Designer
Avg
Autribte
Reiation Reasoning
Avg
Attrilute +Relation
Attrihure +Reasoning
Relation +Reasoning
Style
Text
GPT-4o OpenAl
2025
84,19
85:30
81.00
86.15
88.74
81.04
81,95
50.03
80.58
76.67
92.76
89.55
H-UX.1-des Labs
2024
66.24
74.41
72.50
79.20
72.52
60.72
66.76
61.26
56.00
63.33
44.49
74.63
FLUX.1 1-Pro Labs 2024
63.75
70.00
08.51
75.66
64.03
70.69
62.34
64.05
63.00
34.39
09,94
Tamus-rro778
Chen of al. 2005
65.38
74:99
74.50
73.69
76.77
61.77
65.71
62.01
61.16
43.33
38.46
79.48
Jiang
67.61
$1.14
79.81
70.10
50.00
32.13
RAGEL Deng al. 2025
70.97
78.16
78,00
80,24
76.25
68.23
73.37
64.36
68.92
80,00
40.72
76,87
BAGEL w/ self Coll Deng 2025
68.06
77.63
75.00
78.55
79.33
71.24
77.65
09:77
72.93
09:93
26.24
09.78
RAGEL
85.07
82.61
88.58
78.52
76.96
77.43
93.33
72.40
82.84
knowledge is highly effective, developing methods to foster more generalized understanding remains an important direction for future research.
4 RELATED WORK
Unified Large Multi-modal Models. Unifying and utilizing multi-modal representations' is a central goal in the development of large language models. Early work, such Chameleon (Team] 2024), unified image and text by utilizing discrete tokens for both generation and understanding, employing unified next-token prediction paradigm. Later works (Wu et al.) 2025b Chen et al.) [2025] Wu et al.. 2024b; Ma et al.) 2025 Tang et al.] 2025 Lin et al., 2025 Wang et al. (024) extended this paradigm with improved tokenizers, decoupled encoders for generation and understanding. and even lossless continuous tokens. In contrast, Wu et al. (2024a) unifies the representation of different modalities in the latent space, utilizing different encoders and decoders for each modality. More recent work (Zhou iet al., [2224] Xie et al., 2024 2025) Deng et al. (025) has further bridged the modality gap by employing different tasks within the same model: next-token prediction for understanding and a diffusion objective for generation. Among these, BAGEL (Deng et al.) (025) has emerged as particularly powerful model, demonstrating significantly improved performance on both generation and understanding tasks. In this paper, also focus unified multi-modal modeling, but from task-oriented perspective. Specifically, we aim to unify tasks by framing understanding as a subtask of generation. We adopt BAGEL as our baseline model to ensure convincing evaluation.
Reinforcement Learning for Multi-modal Models. Reinforcement Learning (RL) has emerged powerful paradigm for enhancing the reasoning capabilities of large language models (Jaech et al.
2024 Shao et al.] 2024). This approach enables models to transcend the limitations of imitation learning. where they merely replicate demonstrated patterns. Instead, RL empowers them to autonomously discover and optimize complex, multi-step generative strategies. Early works [Tam al. (2025); Zhou al. (2025) aimed to adopt the RL paradigm to image understanding. which still Tocused on text level learning. Recent works have begun to leverage RL to improve text-to-
image synthesis (Jiang al. 2025 Zhang al., 20256) Duan et al., 2025 Liu et al., 2025). For instance, GoT-RI (Duan et al. 2025). ), built on the next-token prediction paradigm, proposes learning
a detailed semantic plan and layout for image generation. T21-R1 (Tiang al., 2025) further employs
reinforcement learning to jointly train both text and image tokens. FlowGRPO (Liu et al. 2025)
extends this approach to diffusion models by applying Generative Reward Process Optimization (GRPO) to the series of denoising steps. Since our work builds on BAGEL, which generates text via discrete next-token prediction and images via a diffusion process, we adopt both GRPO and FlowGRPO to jointly optimize both processes. However, focus is recasting understanding as component of generation. We therefore compose multiple tasks to construct the generation process, resulting in more complex and thorough framework.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
This paper analyzed the key trade-off between generation and understanding in multimodal models, identifying the conflict may arise from their competitive optimization objectives. To navigate this challenge, we proposed the Reason-Reflec-Refine (R3) framework. R3 re-conceptualizes generation as multi-step process where the model explicitly leverages its understanding capability to iteratively refine its output. Our experiments indicated that by integrating reflection into the generative flow, R3 helps ease the optimization dilemma, leading stronger generation results and improved understanding ability related to the generative task. This framework offers valuable insights for designing eext-generttion unified multimodal models.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
LW is supported by National Science and Technology Major Project (2022ZD01 14902) and National Science Foundation of China (NSFC92470123, NSFC62276005),
REFERENCES
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.1785 2025.
Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3043-3054, 2023.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining, arXiv preprint arXiv:2505. aate:22555.44883. 14683, 2025.
Runpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, et al. Dreamllm: Synergistic multimodal comprehension and creation. arXiv preprint arXiv:2309. 1499, 2023.
Chengqi Duan, Rongyao Fang, Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng. Hongsheng Li, and Xihui Liu. Got-rl: Unleashing reasoning capability of mllm for visual generation with reinforcement learning. arXiv preprint arXivv2505, 2025.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14700 2020.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, al. Openai ol system card, arXiv preprint 1.Xi::2412.1672000 2024.
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and loken-level cot. arXiv preprint arXiv:2505.00703, 2025.
Black Forest Labs. Flux.
com/b/.ccccccccaccccccc forest Labs/flub forest-Labs/Elux 2024.
Junzhe Li, Yutao Cui, Tao Huang, Yinping Ma, Chun Fan, Miles Yang, and Zhao Zhong. Mixgrpo: Unlocking flow-based grpo efficiency with mixed ode-sde. arXiv preprint arXiv:2507.21802, 2025.
Weixin Liang, Lili Yu, Liang Luo, Srinivasan Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, al. vixture-of-transfoome architecture for multi-modal foundation models. arXiv preprint (..i:::24/1004999 2024.
A sparse and scalable
Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangho Yu, Shaodong Wang, Yunyang Ge, al. Uniworld: High-resolution semantic encoders for unified visual understanding and generation. arXiv preprint arXiv:2506.03/1 03147, 2025.
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan. Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.054700 2025.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 2022.

---

Published as a conference paper at ICLR 2026
Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang. Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiao- juan Qi. Unitok: A unified tokenizer for visual generation and understanding. arXiv preprint
arXiv:2502.20321, 2025.
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden Mijeen-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23-40. Springer, 2024.
OpenAl. httt:s///ppeaaa..coo///eeerrrrrr -o 2025.
40 soooimae--aa generation
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Hao Tang, Chenwei Xie, Xiaoyi Bao, Tingyu Weng, Pandeng Li, Yun Zheng, and Liwei Wang. Unilip: Adapting clip for unified multimodal understanding, generation and editing. arXiv preprint arXiv:2507.23278, 2025.
Chameleon Team. Chameleon: Mixed-modaa early-fusion foundation models. arXiv preprint arXiv:2405.098/8. 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Bean-Baptise Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint vrXiv:2322.11805 2023.
Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.07491, 2025.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869, 2024.
Xinyu Wei, Jinrui Zhang, Zeqing Wang, Hongyang Wei, Zhen Guo, and Lei Zhang. Tiif-bench: How does your 12i model follow your instructions? arXiv preprint arXiv:2506.022 2025.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint 2rXiv:2508.02324 2025a.
Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19977, 2025b.
Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal 11m. In Forty-first International Conference on Machine Learning. 2024a.
Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409. 044099, 2024b.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijic Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12:: 12528, 2024.
Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal models. arXiv preprint 2506::2500..5644 2025.
Junyan Ye. Dongzhi Jiang, Zihao Wang, Leqi Zhu, Zhenghao Hu, Zilong Huang, Jun He, Zhiyuan Yan. Jinghua Yu, Hongsheng Li, et al. Echo-4o: Harnessing the power of gpt-4o synthetic images for improved image generation. arXiv preprint arXiv:2508.n 09988, 2025.
12

---

Published as a conference paper at ICLR 2026
Jiaxing Zhang, Xinyi Zeng, and Hao Tang. Resolving task objective conflicts in unified multimodal understanding and generation via task-aware mixture-of-experts. arXiv preprint arXiv:2506.03591, 2025a.
Yu Zhang, Yunqi Li, Yifan Yang, Rui Wang, Yuqing Yang, Dai Qi, Jianmin Bao, Dongdong Chen, Chong Luo, and Lili Qiu. Reasongen-rl: Cot for autoregressive image generation models through sft and arXiv preprint arXiv:2505.24885, 2025b.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408. 10039, 2024.
Hengguang Zhou, Xirui Li, Ruochen Wang. Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1- zero's" aha moment" in visual reasoning 2b non-sft model. arXiv preprint arXiv:2503.05132, 2025.
13

---

Published as a conference paper at ICLR 2026
A
APPENDIX
A.1
CLARIFICATION ON THE USE OF LLM
During the preparation of this manuscript, Large Language Model (LLM) utilized tool enhance readability and correct grammatical errors. The authors carefully reviewed and edited all Al-generated suggestions to ensure the final text aligns with their original intent and arguments. The intellectual content, analyses, and all arguments presented in this paper are solely the work of the human authors, who take full responsibility for the final content of the publication.
A.2 RL TRAINING WITH GRPO
Our methodology leverages Group-Relative Policy Optimization (GRPO) to refine two key com- ponents: the Chain-of- Thouuhh (CoT) generation within our Reasoning and Reflect stages, and the denoising process for image generation and editing. For the text-based CoT policy, we employ the standard GRPO algorithm (Shao et al. 2024). For the diffusion model, we adopt the framework of FlowGRPO (Liu et al. 2025). Below, we provide technical overview of these reinforcement learning techniques.
A.2.1 GROUP-RELATIE POLICY OPTIMIZATION (GRPO)
Group-Relative Policy Optimization (GRPO) Shao 2024) a policy gradient algorithm derived from Proximal Policy Optimization (PPO). It stabilizes training by normalizing advantages against group of responses sampled from the policy. This approach reduces the variance of advantage estimates, leading to more consistent and effective policy updates.
The optimization process begins by sampling a group of responses {o:1 for given prompt q using the current policy Teai Each response o, is evaluated to obtain a reward R, The group-relative advantage A,, for the i-th response is then calculated by standardizing its reward against the statisties of the entire group:
R, mcan ( (1
(td
where 5 is a small constant added for numerical stability.
The policy wy then updated by maximizing clipped surrogate objective. This objective is regularized with KL-divergence penalty to prevent large deviations from reference policy Tref (typically the initial supervised fine-tuned model):
JGRPO (0)
E (q.a)>D,
fort Ma
lod1
min(r:a(0) A.,, (l)p(r..(0), 1 - E, 1 e) A..)) - DKL(TToll
los|
Here, (0) is the per-token importance sampling ratio between the new policy and the old policy TB
To (oz, q, 01,ce)
(0,)(0)
Toau (Oi, g, 0i,et))
A.2.2 FLOWGRPO FOR DIFFUSION MODEL OPTIMIZATION
To apply policy optimization to our diffusion model, we utilize FlowGRPO (Liu et al. 2025 which adapts the GRPO framework for continuous state-space models trained via flow matching.
Flow Matching Preliminaries. Our diffusion model is built upon Rectified Flow (Liu et al. 2022
which defines linear interpolation between data sample vn X, and noise sample 31 Xi as x = (1 1) + Lxy for E [0, 1]. A velocity field network (g !) is trained to predict the vector field æ1 zo via the flow matching objective:
(0) E4,z0.21 [1[(2] xo) vo ((1 t)xr + Lx, 1)
14

---

Published as a conference paper at ICLR 2026
MDP Formulation and Policy Optimization. The iterative denoising process of diffusion model can be naturally framed as a Markov Decision Process (MDP). At each discrete step t, the state is 8, (c, t, 24), comprising the conditioning prompt c, the current time and the noisy sample 2;- The action corresponds to generating the subsequent, less noisy sample 3 1 ~ To (o [2,, c). terminal reward R(xo,C) c) is assigned only at the final step = 0) upon generating the complete sample 20.
Under this MDP formulation, FlowGRPO generates group of G images W for given prompt The advantage for the i-th trajectory is computed relative to the group's 's final rewards:
R(x mean (eam(((((2))))))
A
sud( ))))))))))))))))))) +
Since the reward is terminal, the advantage A is constant across all timesteps for a given trajectory i. The FlowGRPO objective is then:
JFWW-CRRO (0) (at
min (ri(0)A',, 1 3BDL (Toll
po po(xi_11a(.e) reat (= m)a))
where the importance ratio is r((
Stochastic Differential Equation for Exploration. Reinforcement learning requires policy exploration, yet the generative process defined by an Ordinary Differential Equation (ODE), dr. vo (1,, 1)dl, is deterministic. To facilitate exploration, FlowGRPO converts the ODE into corresponding Stochastic Differential Equation (SDE):
of (It (1 (z)) dt + ordwi,
dx,
Up Iz,
21
where dw, represents the increments of Wiener process and controls the noise magnitude. Discretizing this SDE with the Fuler-Maruyama method yields the update rule for the generative sampling process:
Ar Tr. vo( 1)
o7 21
(x. + +1- ())a((e,))) 11) AL o.VAlz > ~0011
The noise schedule is given by o, = a t/(1 t). where a a scalar hy hyperparameter that adjusts the level of stochasticity. This SDE-based sampling enables the policy to explore different generation paths, which essential for effective RL-bbased optimization.
Efficient Training with MixGRPO (Li al., 2025). To mitigate the computational demands of SDE-based sampling during policy training, we adopt the MixGRPO (Li et al. 2025) strategy. This approach combines SDE and ODE sampling in denoising process, substantially reducing the computational burden while maintaining strong model performance.
A.3 EXPERIMENTAL DETAILS
A.3.1 GENERATION BENCHMARKS
To validate the effectiveness of our proposed Raason-ReflectRReiin framework, we conducted experiments three widely-used text-to-image generation benchmarks.
GenEval.
The GenEval benchmark (Ghosh al. 2023 designed to assess model's capability
to generate images with complex compositional requirements, including object counting, spatial relationships, and attribute binding. For training, prompts were generated using the official GenEval scripts, which construct prompts by randomly combining predefined templates. We rigorously excluded any prompts present the official test set from our training data. The reward signal for our reinforcement learning process is derived from the soft reward function proposed by FlowGRPO (Liu jtt al. 2025), which was specifically designed for this benchmark. The result is presented in table method reaches state-of-the-art with score of 0.962.
15

---

Published as a conference paper at ICLR 2026
Table 7: Hyperparameters for Different Datasets
Hyperparams
GenEval
GenEval++
THF
Learning Rate
5e-6
5e-6
5e-6
Batch Size
16
16
16
Group Size
16
16
16
Training Steps
400
300
200
CE Weight
1
1
MSE Weight
2
Temperature
0.9
0.9
0.9
Number Timesteps for Reasoning
10
15
15
Number Timesteps for Edit
20
20
20
CFG for Text
4
4
CFG for Image
1.5
1.5
1.5
KL for Image
0.005
0.005
0.005
KL for Text
0.0005
0,0005
0,0005
GenEval++. GenEval++ (Ye et al. 2025 is more challenging extension of GenEval, featuring more complex instructions and employing advanced Vision-I Language Models (VLMs) for a more robust evaluation. Following similar procedure to GenEval, training prompts generated by randomly combining elements from the official templates, ensuring no overlap with the test set. For this benchmark, we employ Qwen-2.5-VL-72B the reward model.
The TIF benchmark (Wei al. 2025 is designed systematically assess model's ability to interpret and follow intricate, fine-grained textual instructions. comprises 5,000 prompts categorized into three difficulty levels, enabling nuanced evaluation of critical capabilities such as attribute synthesis, reasoning, and style control. For our experiments, we utilize the official training prompts for reinforcement learning and report performance on the short-prompt version of the test-mini evaluation set. The reward signal is calculated based on list of yes/no questions with ground-truth answers provided by the benchmark. The reward defined the ratio of correct answers:
THE.
Ncomect
Nivial
Given model generated image, we utilize Qwen-2.5-VI--72B as the VI.M to answer these questions and compute the reward.
A.3.2 UNDERSTANDING BENCHMARKS
central hypothesis our work is that structuring the generation process through iterative reasoning with reinforcement learning not only improves the final output but also enhances the model's 's fun- damental multimodal understanding capabilities. To empirically validate this claim, we designed two novel downstream evaluation tasks to measure the change in the Bagel model's comprehension abilities before and after training with our Reason- Reason-Reflect Reflect-Reeeeeeee Refine framework. For both tasks, we use a powerful proprietary VLM, Gemini 2.5 Flash, establish high-quality ground truth for comparison.
Compositional Visual Question Answering (VQA). This task is designed to directly probe the model's ' ability to accurately perceive and verify the complex compositional elements it was trained to generate, such object counts, attributes, and spatial relationships.
Data Construction. We first utilize the Bagel model to generate set of images based on the test prompts from the GenEval++ benchmark. Subsequently, we leverage the structured metadata of these prompts to automatically construct corresponding set of factual, closed-ended (yes/no) questions. For instance, for a prompt describing "five cats," we generate the question, "Are there exactly 5 cats in the image?".
Evaluation Protocol. Ground truth is established by querying Gemini 2.5 Flash with these image- question pairs and recording its answers. We conduct careful human alignment check to ensure a high-quality annotation. We then evaluate the accuracy of both the baseline (pre-trained) Bagel
16

---

Published as a conference paper at ICLR 2026
Table Text-to-image generation ability on the GenEval benchmark (Ghosh et al. 2023). The best scores are bolded.
Method
Single object
Two object
Counting
Colors
Position
Color attribution
Overall
FLUX, 1-dev (Labs. 2024
0.99
0.81
0.79
0.74
0.20
0.47
0.67
Owea-Image-RL RL Wu ot
2025a
1.00
0,95
0.93
0.92
0.87
0.83
0.91
FlowGRPO (Lu
1.00
0,99
0.95
0,92
0.99
0.86
0,95
BAGEI DDng al.
2025
0.99
0.94
0.81
0.88
0.64
0.63
0.82
BAGEL + Ours
1.00
1.00
0.95
0.95
0.98
0.89
0.96
model and our final (RL-trained) model on this VQA set. The primary metric is the accuracy of each model's answers when compared against the Gemini 2.5 srash-geeerated ground truth. We conduct a careful human alignment check to ensure a high-quality annotation.
Image- Sanae-Teex-Alligg -elt--lggnmennt (TTA) This task assesses a more holistic and nuanced form of comprehen- sion by evaluating the model's ability to function reliable "judge" or reward model, critical capability for advanced AI systems.
Task Setup. Using the same set of images generated for the VQA task, prompt the Bagel model to perform an evaluation. Specifically, we employ the official GenEval++ evaluation template, which instructs the model assess the quality and prompt-alignment of a given image.
Evaluation Protocol. We first obtain a ground-truth set of evaluations by having Gemini 2.5 Flash perform the same assessment on all image-prompt pairs. We conduct careful human alignment check to ensure high-quality annotation. We then compare the judgments made by the baseline and the final trained Bagel models against this ground truth. The primary metric is the agreement rate between Bagel's judgments (e.g., its scores categorical decisions) and the assessments provided by Gemini 2.5 Flash. This allows us to quantify the improvement in the model's ' ability to discern high-quality, prompt-adie adherent generations.
A.3.3 IMPLEMENTATION DETAILS
We detail the specific training procedure of the R3 framework. the Reasoning stage, we utilize a prompt batch size of 16 and sample 16 responses for each prompt. The reasoning text is sampled with temperature of 0.9, while the diffusion SDE sampling uses a noise parameter transition to the Reflect-Refing stage, select subset of 16 samples from the total pool of 256 candidates (16 prompts x 16 responses). Crucially, we employ a sampling strategy to ensure that approximately 20% of these selected instances "perfect" samples (i.e., achieving reward of 1). For the subsequent Reflect-Refine stages, we maintain a text sampling temperature of 0.9 but adjust the editing diffusion noise parameter for reasoning and another for reflect-refine. Additional training hyperparameters are summarized in Table 7
= 0.7. To
1.0. The training trajectory length is set with round
A.4 EFFECT OF RL TRAINING
To clarify the effect of RL training, we conduct an ablation study comparing Bagel (without RL training) against our RL-trained model using the R3 framework. The results are presented in table 9
Inference Strategy
Bagel
Ours
Reasoning only
0.399
0.593
Reason+RR
0.432
0.675
Reason+RR x 2
0.436
0.689
Reason+RR x 3
0.439
0.682
Reason+RR x 4
0.439
0.689
Reason+RR 0.439
Table 9: Inference performance on GenEval++ comparison between Bagel and R3 (ours) under same inference strategies.
17

---

Published as a conference paper at ICLR 2026
These results demonstrate the effectiveness of RL training:
1. Achieve higher performance ceiling: Our RL-trained model achieves substantially better perfor- mance across all iteration counts (0.593 vs. 0.399 with reasoning only, and 0.689 vs. 0.439 at convergence). indicating that RI. fundamentally enhances the model's I' capability to understand and generate improved responses.
Improve efficiency: Our model converges to near-optimal performance within Reflection- rounds (0. (0.689), whereas Bagel requires 3 rounds to reach its plateau (0.439). This demonstrates that RL training not only clevates the performance ceiling but also accelerates convergence, making the approach more computationally efficient in practice.
Notably, the performance gap between our model and Bagel increases with the Reflection- Refine framework, suggesting that RL training better equips the model to leverage iterative refinement effectively.
A.5 COMPUTATIONAL COST ANALYSIS
key practical concern for iterative refinement is computational overhead. We provide a computa- tional cost analysis as follows:
Adaptive Inference. Unlike fixediteration approaches, R3 learns to self-terminate when generation quality is satisfactory. On GenEval+++ the distribution of refinement iterations shows efficient resource allocation: 45% prompts finish immediately (0 refinement cost), 26% require refinement turn, 14% need 2 refinement turns, and only 15% require 3+ refinement turns. This adaptive behavior significantly reduces average computational cost.
Inference Latency. We measure wall-clock time on single NVIDIA H20 GPU (batch size 1, 512x512 resolution). The initial Reasoning stage takes 20-25s. Each Reflect-Refine turn requires 25-35s, with Reflection (text) accounting for 5-10s and Refinement (image) for 20-25s.
A.6 VISUALIZATION ON MULTI-ROUND GENERATION
We demonstrate the multi-round editing process of proposed R3 framework in figs. 8 15 The figure details each stage, including the initial Reason stage's planning text, the subsequent reflection text, and the resulting refined image.
18

---

Published as a conference paper at ICLR 2026
the pond, ethinte The model should generato an image of a boy standing neara pond, not under stands near the enge of colm, reflactive pond under clear sky. surrounded by lush greenery. The scene is ultra realistic, with warm naturl lighting sharp datalls of bay clothing and textured water sur faces </think>
boy on
tree.
The comprehensive prompt Ayoung
<thinke description states "The boy under tree:" the provided image, the boy standing under a tree near the pond, which contradicts the target deseription. To align the image terget description,
the pond,
under which boy standing be minoved or roplaced with an open ares, ensuring the boy appears on overheed, The instructions should
the side of the pond without specify editing the out of scene, leaving the boy standing beside the pond in an open environment, <ithink Modify the imegge show the boy standing directly the edge the pond without any above him; remove the tree canopy and replace it with an open sky or beckground, ensuring the boy appears beside the water setting. Final edit: Remove the tree branches and follage from above boy, leewing him standing on the pond'ss edge in open-air environment with tree overheed.
boy be "on the pond, not under tren:" In the provided image, the boy is already standing beside the pond, which aligns with the location requirerment. There is no tree under which the boy standing, fulfilling the "not under a tree" candition. Therefore, no further editing operations are nacessary to conform target description. </think>> No further edit needed; existing image atready depicts the boy standing on the side of the pond without being under tree, matching the description.
<think description states
Figure 8: An illustration of our model's two-stage generation process. For the prompt "The boy is on the side of the pond, not under a tree," the model's initial reasoning stage erroneously generates a tree above the boy. The subsequent reflect-refine stage corrects this error, after which the process terminates automatically.
Ayellow grasshogper hops the wooden the yellow sun sploshes warmth ecross the field.
<think> The model should geneesae image yellow grasshopper wocden tence, with a bright yellow comprehenaive promat le: A vibrant yellow grasshopper leaps near rustic
shining over field. The sunlit field, bathed yellow sun overneed, ultre-realistic, warm-toned, dynamic lighting. </think>
wooden
golden from glowing
<thinke The description specities shows grean grasshoppar. To allgn the torget description, grasshosper's color should be ed edited to white maintaining the rest of image's elements, and hold lighting, as they already metch the description. </hinkk> Modify grasshopper appear yellow while preservingthe fence, glowing yollow sun, and womn taned field lighting from the original imags. Final edit: Updane the graashogper's 's color to yallow, ensuring consistency the
sehoppe,, but the
wonden
yollow
wooden
description a "yellow greeshopper," while retaining all other visusi components of scene as originally presented. Final Output: A yellow grasshopper hops under the wooden as the yellow sun the warmth, metching tanget description.
<thinks The description matches the arovided image perfectly, as shows yellow gresshopper under a woaden fence with a yellow sun casting warmtth over a field. No further aditing is raquired to align the image with the target description, <hthikk No further edit needed, the existing image already corresponds to the description: "A yellow grasshooper hops under the wooden fance the yellow sun splashes warmth across the holld"
Figure 9: An illustration of our model's two-stage generation process. For the prompt "A yellow grasshopper hops under the wooden fence as the yellow sun splashes warmth across the field," " the model's initial reasoning stage generates grasshopper with an incorrect color. The subsequent reflect-refine stage corrects the color, after which the process terminates automatically.
19

---

Published as a conference paper at ICLR 2026
picture of band performing stage, the on it: SSIMPLY, "Bullett "Prooft "Nothing,, "SAUGER: <thinle The model should generate stage, sccompanied by reading 'SIMPLY
band performing Nothing SAUCER." The comprehendive prompt is: Live bend pertorms en dimly stage dramatic lighting overlay in bold, slylized font reads "SIMPLY Bullet Proof Nothing SAUCER," avoking punk rock nestheric, <ithinko
SIMPLY BULLET PROCD NOINISERY
musicians playing instruments.
ethinic The target description calls for taxt charges: "SIMPLY BULLET PROOF NOTHING SAUCER:: The original image has 'SIMPLY BULLET PROCD NOINISERY: to align
torget, replace the cxisting with the specified words, enauring proper capitalization a nd ling. layout and style
should remain consistent original image's design. </think> Modify the text overlay to read "SIMPLY BULLET PROOF NOTHING SAUCER" in bold, capital letters matching style origgnal imagle, ensuring aligrmmnt with target description's requirements. Final edit: Update the text to "SIMPLY BULLET PROOF NOTHING SAUCER" in font and size consistent with the original imsge's design, preserving visual cohesiin.
SIMPLY BULLET PROOF NOTHING SAUCER
ethinio The target description stotes the text should read "SIMPLY BULLET PROOF NOTHING SAUCER." The providedd image already matches deseription, stags. Therefore, no further editing operation are required beyond what already presentin the image, </thinka No further edit needed: the image already eligns with target description text ("SIMPLY BULLET PROOF NOTHING SAUCER' and bend perform ing on stage. Finel Oulput: The existing image meets requirements the target desoription without any additional i edits.
text the band performing
Figure 10: An illustration of our model's two-stage generation process. For the prompt 'A picture of band performing on a stage, with the text on it: "SIMPLY", "Bullet", "Proof", "Nothing", "SAUCER"", the model's initial stage fails to correctly render some of the text. The subsequent reflect-refine stage successfully corrects the text to match the prompt, after which the process terminates automatically.
picture of bell resting court, the it "QUICK", "STARTT "IENNIS" "BEFORE; "SCHOOL; "CLUB: <think The model should generste
tennis bell overlay stating "QUICK START TENNIS BEFORE SCHOOL CLUB" The comprehensive prompt is: vibrant temii ball rests on a bright sunlight,
QUICK START TENNES SCHOOL
green clay court
sharp
contrast spell out "QUICK START TENNIS BEFORE SCHOOL CLUB, with dynamic typography and a sporty, cnargetic atmosphere, <ithinke
<thinko The tanget description requires text changes to "QUICK START TENNES BECTIIS SCHOOL" into "QUICK START TENNIS BEFORE SCHOOL
instructions must replace existing text the tonnis bell with the specitted words, ensuring alignmant with the targot description. This riquins aditing the text content the ball while preserving
mage's structure and lightingg <tthink> Modify the text on the tennis to wad "QUICK START TENNIS while maintaining
rest
QUICK START TENNIS
BEFORE SCHOOL
image's original
lighting and court background. Ensure the font style and size match the existing text lyyout. Final edit: Update the text on the tennia holl display "QUICK START TENNIS style and size as the original text, preserving the image's ovesall apaearance.
SONOL
SCHOOL CLUB" the same
<think
The target description matches the text already present on the ball in provided linage. Therefore, further editing operations am required to align the image with the target description.
</think No further edit needed, reada "QUICK START TENNIS BEFORE SCHOOL CLUB," which matches the targel description.
text the temnis ball siresdy
Figure 11: An illustration of our model's two-stage generation process, showing limitation. For the prompt requiring specific text on a tennis ball, the initial stage fails to render all the words. The reflect-refine stage adds most of the missing text but introduces minor spelling error ("CLUB"). Despite this inaccuracy, the model terminates the process without attempting further corrections.
20

---

Published as a conference paper at ICLR 2026
Apotoo white bench, yellow stop sign, pink
<thinio The model should depict an outdoar umen scone teaturing white bench, yellow sigh, and pinknecktie lying
ground nearby, surrounded by naturl urban environment Like grass stone. The answer image description: small outdoor urben setting shows a white woodan
stone- paved emidst overgrown gress and scatlered debris. A yellow stop sigh positioned neerbyy leaning slightly against worn stone wall, A pink nacktie lies diacerded on the rocks,
ground beside the berch, surtounded scattered twigs and avaking sense of abardonment or forgotten objects in a rustic setting. e/think
ethinkc Okay, start by comparing orig Image the target description. The target requires white bench, yellow stop sign, and pink og Image: The banch: The current banch white, so that matches the first requirement. 2. The signt The current sign yellow and shaped like a hexagon, not a sign. ssop sign ln typically octagonal. So the sign must changed octagonal yellow. 3. The tie: The pink tie is almady visible lyknn on the ground, so no change needed here. Therefore, the editing task involves replacing the hevagonal sign with an octagonal yollow stop sign, while keeping the rest unchanged. </think>- Edit the sign replace the hesagonn with octegoonal yellow stop sigh, while preserving white
Lookingat at
STOP
pink
<thinks Okay, start by comparing orig nat image the target description. The targetraquiee requires white bench, yellow stop sign, and pinktti. Looking at the existing image: The bench already white, that matches the first requirement. 2. The stop sign yellow, which correct. The bi: The current object is pink, so na change needed here. Therefore, further edits are required all elemments sligh perfectlly with target desorpttin, <ithinke no turther ecit needed
Figure 12: An illustration of our model's two-stage generation process. For the prompt 'A photo of white bench, a yellow stop sign, and a pink tie," the model's initial generation fails to correctly render the stop sign. The subsequent reflect-refine stage successfully generates the object specified, and the process terminates automatically.
photo larger computer mouse on the above and smaller sandwich on the below.
where larger computer placed on higher surface, such as a ledge shelt, while smallar sandwich lies on the ground below, illustrating contrest in scale and contect. answer image description: tustic outdoor setting festures a larger computer mouse perched stop a stone ledge or weathered wooden sholt, while small sandwich lies on the rough ground below amidst scettered debris twigs fallien placament, </thinke
<think The model should depict
creating contrest
<think Okay, let's start by looking at the current scene. The existing setup shows small computer mouse perched stop a stone ledge, while tiny sandwich liss below on the ground, The torget requires the mouse appear larger situated
while the sardweich becomes smaller and lower. First, the mouse must be enlarged pr proportionally reiative its ourrent size. Enlarging it would make it dominata more upper portion of frame, perhaps covering a larger ledge the sandwich should appear sma lier, probably by scaling it down relative above. The positioning stays consistent: mouse the ledge above, sarddwich on the the ground below. </think Edit the image to scale the mouse langer, occupying more of the upper ledge, while reducing the sandwich and placing contrast.
the
ground
ethinke Okay, lot me start by analycing the current Image. The question asks If the existing setup matches the torget piioription: "lerger computer mouse above, smalier sandwich below. current scanaria shows mouse on ledge above, and is small sandwich on the ground below, The existing mouse appears proportionalyy large milativa seems correct size contrast. New contimm the arrangeenn mousa is top, sandwich hottom, No further edits required hare. </hinko no further edit neaded
sandwich,
placement
Figure 13: An illustration of our model's two-stage generation process. For the prompt 'A photo of a larger computer mouse on the above and a smaller sandwich," the model's initial generation fails to correctly depict the specified size relationship. The subsequent reflect-refine stage adjusts the relative sizes of the objects to match the prompt, after which the process terminates.
21

---

Published as a conference paper at ICLR 2026
photo of white cat, (reen bench, red elephant, and purple sandwich <thinio The model should depict an outdoor scene featuring a white cet resting on the ground, gmen park bench, a small red elophant statte, and purple sandwich y'ngteerrby
set in natural or urtan environimnnt garden area. The a nnseer image description: A small outdoor sotting a a white green park bench, while small alephant statue sits on stone concrete surface nearby. purple sandwich lies discarded on the ground beside the cat, surrounted scattered dabris lkk falion small twigs, evoking an urban park etmospheree <ttinkk>
cat
nearthe
<thinks Okay, let me sta it by comparing the current image to the target de scription. features white cat lying the ground, green the beckground, a purple eleghant toy the right, and a small loy-like sancwich the foreground. The target requires white cat, green bench. red elephant, and purple sandwich. Let's break down each component: White
bench
The catis is
alrandy white and hatches tha requimeeent 2. Grean banch: The existing banchis is change here. 3. Elsphent:: The current elephent purple, Terget requires red, so need to change elephant color to red. Sandwich: The current object is a emall toy sandwich, which agpears yellowsssslliiobbbrrooe replace the existing object elephent to be red and replace the foreground with purple sandwich. Keep the white green bench, and other elements unchanged.
Target requiress
purple sandwich,
purple one. </think>
<think> Okay, lot me start by comparing the current image to the torget festures white
the ground in front green park berich. Next to the cat is red elephant toy, and the foreg orndr lies a purple object resembling a sandwich. Now, the target requires white cat, green bench, red elephant, and purple sandwich. First, confirm matches (white), which is correct. Next, check the be nch color: the existing bench gmen, which fits. The elaphant color red, motching the target. Now the sandwich part: currettoregroum os purple, so that's correct too. No further edits are needed hore bezause all nlements match the desired components. </thinie further edit needed
description. The
An illustration of our model's two-stage generation process. For the prompt "a photo of a white cat, a green bench, a red elephant, and a purple sandwich on the below," " the model's initial generation renders both the elephant and the sandwich with incorrect colors. The subsequent reflect-refine stage corrects the colors of both objects, and the process terminates automatically.
Figure 14:
A pphoto of two suitcases on the above and three donuts the below. <thinks model should dapict an outdoor urban setting where two suitcsses are placed on alodge or elevated surface, while three donuts li scattered on the ground below, surrounded by netural slightly worn environment for image descriptian; A rustic outdoor scane showcasing two vintage sultcases perchad stop ledge amidst rough urban on the ground below, surrounded by scattered debris fallon leaves small twigo. evoking sense of abandonment and time worn charm. </think>
The
three donuts haphazardly
<thinic Okey,, let's start by comparing onginal image the target desorption.. The tanget requires two suitcases on the upper part and three donuts on the lower, Looking the existing setup: Upper part: current image shows stacked suitcases, matches the requirement of two cuitcases on the above. So no change needed here. displays four donuts
Lower pert: The current requires threa donuts on the below: Therefons, editing step would involve removing the extra donut, leaving only three visible. </think> Edit the image to remove the extra donut on the for right, leaving three donutsin foreground arrainged similerly, while preserving the two stacked suitcases on the ledge above unchanged.
The larget
image the target description. terget requires two suitcsees the top and three donuts at the bottom, Looking at the original image: The part two suitcases stackind correctly, so the uppar portion matches the requirement. Now the bottom: there are four donuts shown, while target requires three. Therefore, mustremove the extra donut, leaving three remaining at the </think
ethinko Okey,,
by comparing the
image remove the extra donut cionuts remaining on the bottom, while preservinng ng the two sultcases on the top unchanged.
right, leeving three
<thinkc Okay, let's The target requires suitcases the upper and three dinnuts on the lower. Looking at the current image: The upper part already shows two suitoases stacked correctly top of the other. Now check the lower part shows three donuts lying
ground, which matches targel requirement three donuts bottom. No further edits are needad here bacause both uppar and lower sections align partectly. Tharatoro, contirm no further edits required. </thinle turthar edit noedied
An illustration of our model's three-stage generation process. For the prompt "A photo of two suitcases the above and three donuts on the below," " the initial generation incorrectly produces four donuts. In the first reflect-refine stage, the model identifies the count error but fails to correct it. A second reflect-refine stage is initiated, where the model successfully edits the image to show the correct number of donuts, after which the process terminates automatically.
Figure 15:
22

---

Published as a conference paper at ICLR 2026
A.7 EXTENSION OF R3 FRAMEWORK TO MAZE NAVIGATION TASK
To demonstrate the versatility of our proposed R3 Geeason-ReflectRefiine framework beyond text- to-image generation, we apply the complex task of Maze Navigation. In this task, the model is given an initial image of maze and is required to plot solution path from the entrance to the exit. Generating the entire path in single forward pass is challenging for complex mazes; therefore, the iterative nature of our R3 framework is particularly well-suited for decomposing this problem into sequence of manageable sub-steps.
Our training strategy consists of two stages. First, we perform Supervised Fine- Tuning (SFT) on a dataset of random trajectories within various mazes. This initial stage equips the model with the fundamental capability of state transition-that is, generating the subsequent maze image based on its current state and given movement instruction. Subsequently, we employ Reinforcement Learning (RL) to optimize the framework, enabling the model to learn effective policy for generating sequence of movement instructions that successfully navigates the maze from start to finish.
23

---

Published as a conference paper at ICLR 2026
navigation agent. current image, you
1. Understand the maze elements: Vellow star: starting point - Red flag: destination -Buee circle: current position BBlakk lines: currendly exploring path
explored
paths
Follow the navigation rules: Move to the next intersectin//unctee /uunciionn in each step When peth leeds to dead end, mark it gray. and return to the
previous intersection/unntiinn
analyze mare, instruction (starts with <ection> and ends with s/action>), eug <action>Go wgydoww//ell//iggiit along the mure path.s/actions and output "Reached the destination/nc Dntraationyoeeemmnntio />"
state, you
next movement instruction.
movement
fllg destination"
There no road
Go right
maze path
Go right
let's
Go along the maze path
Go left maze path
Go up mase path
There ahead, let's go back
down maze path
Go right along the maze path
There is no road ahead, go back
Go left along the
along the
Go left maze path
Go left
Go up along the path
along maze path
Go maze path
Reached the destinations termination]]
Figure 16: An illustration of maze navigation results. The model is tested 12x10 maze, where it demonstrates the ability to accurately identify valid directions during the navigation process. When faced with multiple paths dead ends, the model can effectively backtrack. Furthermore, it automat- ically recognizes when the destination has been reached and terminates the process accordingly.
24


Reviews:
[{"review_id": "PlhRGlMJvN", "reviewer": "Reviewer_xGzx", "summary": "This paper tackles the \"optimization dilemma\" in multimodal models, where improving generative capabilities often degrades understanding, and vice versa . The authors posit this conflict arises from competing training objectives for finite model capacity . They propose the Reason-Reflect-Refine (R3) framework, which recasts generation as an iterative \"generate-understand-regenerate\" process. The model first Reasons to create a plan and initial image , then iteratively Reflects on its output (using its understanding) and Refines the image until it aligns with the prompt. The framework is trained end-to-end with reinforcement learning, using a novel Tree-RL strategy for improved stability . Experiments show that R3 not only enhances generation quality but also improves the model's understanding capabilities (e.g., counting accuracy), effectively mitigating the conflict", "questions": "Overall, I find this method to be simple, effective, and intuitive. I will take the opinions of the other reviewers into consideration as well.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Important Problem: The paper addresses the \"generation vs. understanding\" trade-off, a critical and widely recognized challenge in developing unified multimodal models.\n\n\n\nNovel & Intuitive Framework: The R3 framework is a novel solution that embeds understanding as a functional sub-process within generation, creating a synergistic loop rather than a competitive one .\n\n\nStrong Empirical Validation: The core claim is well-supported. Experiments show simultaneous improvements in generation (GenEval++) and dedicated understanding tasks (VQA/ITA), with clear gains in skills like counting .\n\n\n\nEffective Training Strategy: The proposed Tree-RL strategy (Fig. 3) is a solid technical contribution, demonstrating superior stability and reward over standard full-trajectory RL (Fig. 4) .", "weaknesses": "1. The new VQA and ITA benchmarks rely on \"ground truth\" labels generated by Gemini 2.5 Flash. Thus, high scores may reflect better alignment with Gemini's judgment rather than an objective, absolute improvement in understanding.\n\n2. I'm curious whether this method is effective for image editing tasks, especially when spatial understanding is required.\nFor example, suppose there's an image with four people, and my prompt is: \"Move the third person behind the second person,\" instead of simply arranging all four people side by side.\nCan the model correctly interpret and execute such spatially grounded instructions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uicQUri2g2", "reviewer": "Reviewer_r2cT", "summary": "This paper addresses the trade-off between generation and understanding capabilities in multimodal models by proposing the Reason-Reflect-Refine (R3) framework. The authors argue that this conflict arises from competing optimization objectives and propose decomposing generation into a multi-step \"generate-understand-regenerate\" process. It proposes R3 (Reason–Reflect–Refine): turn single-shot image generation into a multi-stage loop that (i) plans, (ii) self-evaluates alignment of the produced image to the prompt, and (iii) edits until a learned stop condition (“No further edit needed”). Training alternates policies for the Reason and Reflect-Refine stages with a tree-RL scheme (GRPO for text; FlowGRPO for diffusion) and stage-wise rewards from a VLM judge. Experiments (GenEval/GenEval++/TIIF) report sizable gains in instruction-following and smaller but non-trivial gains on bespoke understanding tests (VQA/ITA built over model-generated images and VLM judgments).", "questions": "- Can you report human A/B on a held-out prompt set and the cost per image and effectiveness per RR turn? \n\n- What is the exact “immediate rollout” procedure (sampling temperature, selection by reward diversity, replay buffer size), and how sensitive are gains to it?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Well-motivated problem: Recasting generation as generate→understand→regenerate is a well-motivated method aligning with the philosophy of multiple prior COT-related methods. The paper explains the loop clearly and gives a concrete stop rule. \n\n- Optimization design: The tree-RL split reduces variance compared to end-to-end trajectory optimization; stage-specific rewards and formatting checks are well-motivated.\n\n- Empirical gains: On GenEval++ (GPT-4.1 judge), BAGEL+R3 improves overall score vs. BAGEL and edges Echo-4o; understanding proxies (ITA/VQA) also rise. The inference-time scaling plot shows most gains after the first reflect-refine, with saturation after ~4–5 turns—useful for deployment budgets.", "weaknesses": "- The whole framrwork idea does not look particularlu novel to me: Self-correction and iterative refinement for generation have been extensively studied.\n\n- I am abit concerned about the practicability of this work. In practice, developers tend to develop a single-shot generation instead of allowing multiplt RR turns, which is expensive. \n\n- Evaluation validity: Generation quality relies on GPT-4.1 as the arbiter for GenEval++ and understanding relies on Gemini 2.5 Flash to create “ground truth” for VQA/ITA—both proprietary VLMs that may share alignment biases with the reward/eval models. This risks overfitting to judge preferences and limits claims about human-perceived quality. Human or crowd-sourced evaluations are highly recommended to validate the effect outside model-judge ecosystems.\n\n- Because reflection quality and termination are rewarded via VLM scores and format checks, the system could learn to optimize the judge (e.g., produce edits that increase the VLM’s “alignment” without visibly improving images, or prematurely stop when the judge saturates). A test where the judge is swapped could be helpful.\n\n- Generalizability: Authors themselves observe domain-specific improvements (training on “counting” helps counting much more than other attributes). That tempers the central claim that R3 “reconciles” the optimization dilemma broadly. Evidence beyond compositional instruction following (e.g., style/photorealism, long-prompt semantics) is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bSusf6fImx", "reviewer": "Reviewer_c1QJ", "summary": "This paper discusses the long-standing trade-off between generation and understanding in multimodal large models. The authors argue that these two abilities compete for model capacity due to misaligned optimization objectives. To overcome this, they propose the Reason–Reflect–Refine (R3) framework, which reconceptualizes generation as a multi-step process: the model first reasons about the input prompt, then reflects on whether its generated output aligns with the intent, and finally refines the output accordingly. A Tree-based Reinforcement Learning strategy and stage-wise rewards enable stable optimization.", "questions": "Please see the weakness.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* This paper demonstrates the trade-off between understanding and generation, where fine-tuning for one capability degrades the other, and naive co-training yields negligible gains.\n* The Tree-RL strategy and stage-wise reward formulation stabilize training.\n* The results demonstrates consistent improvements across multiple benchmarks, including newly introduced VQA and ITA evaluations.", "weaknesses": "* The experiments does not seem sufficient. Many other unified understanding and generation models are missing from the comparison. Furthermore, the comparison against proprietary models should not be limited to just one; others, such as Gemini 2.5 Flash or Gemini 2.0 Flash, should also be considered.\n* The proposed method performs worse than GPT-4o on both the GenEval++ and TIIF benchmarks, and this performance gap should be discussed and explained in more detail.\n* The reward model design is more empirial and heuristic. The optimization may still bias toward the reward model rather than true bidirectional alignment with understanding and generation. More theoretical analysis is needed to explain the underlying optimization alignment mechanism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MGTtCPfviA", "reviewer": "Reviewer_k1uR", "summary": "This paper presents R3, a Reason-Reflect-Refine framework for jointly improving unified multimodal model's understanding and generation ability. The proposed framework iteratively reasons on the generated image and refines it based on its own feedback, and further uses RL to finetune the model for better conducting R3.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed framework significantly improved the model's instruction following generation power as well as the understanding power.\n- The use of RL to further improve the R3 power gives significant improvements in the designed scores.\n- Such an approach may lead to more advanced multimodal reasoning paradigms.", "weaknesses": "- My major concern is that the effect of RL is unclear. Based on results in the paper, RL seems improved the average understanding and generation capability in one Reflection-Regenerate step. But it is unclear whether 1) RL improved the highest possible performance given an unlimited Reflection-Regenerate round budget, OR 2) RL reduced the number of Reflection-Regenerate rounds to have a converged performance.\n- The proposed framework is only tested on one single model, BAGEL. It's necessary to test its generalizability over different MLLM backbones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)