MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoning
Authors: Yapeng Mi
Yanpeng Zhao
Hengli Li
Chenxi Li
Huimin Wu
Xiaojian Ma
Song-Chun Zhu
Ying Nian Wu
Qing Li
Venue: ICLR 2026 Poster (2026)
Keywords: Image Generation
Test-Time
Latent Reasoning
Primary area: generative models
TL;DR: A training-free test-time latent reasoning method that jointly optimizes text and image latent representations for state-of-the-art multimodal generation.

Abstract: Reasoning-augmented machine learning systems have shown improved performance in various domains, including image generation. However, existing reasoning-based methods for image generation either restrict reasoning to a single modality (image or text) or rely on high-quality reasoning data for fine-tuning. To tackle these limitations, we propose MILR, a test-time method that jointly reasons over image and text in a unified latent vector space. Reasoning in MILR is performed by searching through vector representations of discrete image and text tokens. Practically, this is implemented via the policy gradient method, guided by an image quality critic.
We instantiate MILR within the unified multimodal understanding and generation (MUG) framework that natively supports language reasoning before image synthesis and thus facilitates cross-modal reasoning. The intermediate model outputs, which are to be optimized, serve as the unified latent space, enabling MILR to operate entirely at test time. We evaluate MILR on GenEval, T2I-CompBench, and WISE, achieving state-of-the-art results on all benchmarks. Notably, on knowledge-intensive WISE, MILR attains an overall score of 0.63, improving over the baseline by 80%. Our further analysis indicates that joint reasoning in the unified latent space is the key to its strong performance. Moreover, our qualitative studies reveal MILR's non-trivial ability in temporal and cultural reasoning, highlighting the efficacy of our reasoning method.

Published as a conference paper at ICLR 2026
MILR: IMPROVING MULTIMODAL IMAGE GENERA- TION VIA TEST- TIME LATENT REASONING
Yapeng Mi Yanpeng Zhao Hengli Li2.3
Chenxi Lia.4
Huimin Wu? Xiaojian Ma² Song-Chun Zhu
Ying Nian Wu
Qing Li
University Science and Technology of China
State Key Laboratory of General Artificial Intelligence, BIGAI
PPeking University "Tsinghua University
University of California, Los Angeles
Project: https:/spatiigen.giiie http://paagggniiiiiuubb/miiioooo
( Code: https:/ggihub.com/ppatggnnn
ABSTRACT
Reusoning-augmenteed machine learning systems have shown improved per- formance in various domains, including image generation. However, existing beasoning-basee methods for image generation either restrict reasoning to single modality (image text) rely on high-quality reasoning data for fine-tuning. To tackle these limitations, we propose MILR, test-time method that jointly rea- image and text in unified latent vector space. Reasoning in MILR is performed by searching through vector representations of discrete image and text tokens. Practically, this is implemented via the policy gradient method, guided by an image quality critic. We instantiate MILR within the unified multimodal understanding and generation (MUG) framework that natively supports language reasoning before image synthesis and thus facilitates cross-modal reasoning. The intermediate model outputs, which are be optimized, serve the unified la- tent space, enabling MILR to operate entirely at test time. We evaluate MILR on GenEval, 12I-CompBench, and WISE; achieves state-of-the-ar results on all benchmarks. Notably, on knowledge-intensive WISE, MILR attains overall score of 0.63, improving over the baseline by 80%. Our further analysis indicates that joint reasoning in the unified latent space is the key to its strong performance. Moreover, our qualitative studies reveal MILR's 's nontrivial ability in temporal and cultural reasoning, highlighting the efficacy of our reasoning method.
A photo of three kites.
A photo of three kites, each with distinct color and shape, flying in the sky.
Three vibrant kites sour in the sky sense of depth and movement.
The colorful image depicts three kites and the red kite is at the bottom.
[ (2 Llrigu
[z00,
Unified MUG Model
21 12
36 77
43 53
12 zi0g
[2 2toip
Figure 1: Latent reasoning of MILR. The black solid line denotes extracting the output vector rep- resentations x of the tokens (t) and image tokens (1)) be optimized, and the black dashed line denotes decoding from the optimized latent vectors z where (z(1) 2(0)
Contact: 7adiyapeng78@gmall com, yannzhao. yarrzhao.ed@@maill dylan. wyylan.liqingen www.llggiiiggnini.mmmmmmmmmmmeem com. t: Project Lead. & Corresponding Author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Text-guided image generation is the task of synthesizing an image conditioned on a given text in- struction. In recent years, the field has witnessed transformative progress: moving from generative adversarial models (Goodfellow et al., 2014) to autoregressive and diffusion approaches (Sun et al., 2024: Esser et al., 2024; Black Forest Labs, 2024; Chen et al., 2025; Wu al., 2025a). However, traditional models are limited in generating images in single-shot fashion and thus are unable to resolve potential defects (Huang et al., 2025a; Niu et al., 2025). Inspired by the success of reasoning- augmented L.LMs-
as OpenAl ol (OpenAl et al., 2024) and DeepSeek-R1 (DeepSeek-A) et al.. 2025)-that can reflect on and refine their thoughts, and answer accordingly, recent works have attempted to endow image generation models with reasoning ability (Guo et al., 2025; Fang et al., 2025; Zhang et al., 2025b; Wu et al., 2025b; Chern et al., 2025).
Reasoning augmmettdd image generation models typically perform reasoning in two spaces: lan- guage and image spaces. Language reasoning primarily involves refining instructions to make them more comprehensible to the model (Wu et al., 2025b; Li al., 2025b), and image reasoning it- erates on the generation guided by a quality metric (Guo et al., 2025; Zhuo et al., 2025). Early works implement reasoning either in the language or in the image space, lacking mechanism for synergistic reasoning across the two spaces. To fill the gap, recent works resort to unified multi- modal understanding and generation (MUG; Jiang al. (2025a); Duan et al. (2025); Zhang et al. (2025b)), which natively supports language reasoning before generating images (referred to as mul- timodal image generation), facilitating cross-moda image-text reasoning. Despite the success, these approaches require carefully curated reasoning data and depend on model fine-tuning, rendering them complex and costly to develop in practice.
To address these limitations, we propose Multimodal Image generation via test-time Latent Reason- ing, dubbed MILR. Our core idea is to reason a unified latent vector space that encodes both text and images, starkly different from previous methods that explicitly reason over raw images and text. We build MILR upon a Transformer-basee represented by the intermediate output vectors. Since the shared latent space is modality-agnosee it provides a unified view of multimodal reasoning, reducing the modality gap and improving the overall efficacy of cross-modal reasoning.
MUG model and choose the unified latent space
Reasoning with MILR involves finding the best vector representations of image and text tokens that lead to improved image quality. We implement it using the policy gradient method (Williams, 1992), where, at test time, the reward is computed by scoring the compatibility between the generated image and the given instruction. Crucially, gradients are only back-propagated to the cross-modal latent representations (i.e., the intermediate model outputs) obtained from the forward pass (see Figure 1), without altering any model parameters, and thus making MILR a test-time latent reasoning method.
We evaluate MILR on three widely-used image generation benchmarks: GenEval (Ghosh et al., 2023), T2I-CompBench (Huang et al., 2023), and WISE (Niu et al., 2025). MILR established new state-of-the-ant results across all benchmarks. Notably, it achieves an overall score of 0.95 on GenEval, matching the best training- baased model and outperforming the best test-time-scaling method by 4.4%. On the more challenging WISE benchmark, MILR obtains an overall score of 0.63, surpassing the strongest baseline model by 16.7%. Our further analysis reveals that the best performance of MILR is driven by its ability to perform joint image-text reasoning in unified latent space. This unique advantage also makes MILR successful in instructions that involve challenging cultural and temporal reasoning.
To summarize:
We introduce MILR, a test-time performing joint image-text reasoning in a unified latent space.
method that improves image generation by
We demonstrate the effectiveness of MILR by showing that it achieves superior performance across three different benchmarks for image generation.
We conduct a comprehensive analysis, perform various ablation studies, and discuss potential limitations of MILR.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORKS
Reasoning-Augmenteer Image Generation. Remarkable progress has been achieved in aassoing--uggeeninn LLMs (Wei et al., 2022: DeepSeck-Al et al., 2025), but effectively integrating reasoning into conventional text-guided image generation models remains challenge (Betker et al., 2023; Sun et al., 2024; Esser et al., 2024; Zhao et al., 2024; Chen et al., 2025). With the develop- ment of unified multimodal understanding and generation that can generate images and text in an interleaved manner, many works have explored reasoning for text-guided image generation. The pre- dominant paradigm uses causal language modeling to unify image and text generation as next token prediction, enabling the model to make plans via chain-of-thouugh before generating images (Fang al., 2025; Deng et al., 2025; Xiao al., 2025; Cai et al., 2025; Guo et al., 2025; Jiang et al., 2025a; Duan et al., 2025). Another line of research focuses on reasoning over images via the test-time scal- ing strategy (Li et al., 2025b; Zhuo et al., 2025). These works typically rely on an external critic model to provide feedback on further improvement. Unlike all these methods that explicitly perform reasoning over raw images and text, we use test-time optimization refine the latent representations of image and text tokens, leading to a unified mrosl latent reasoning method.
Reasoning in The Latent Space. Differently from explicit reasoning via a chain of thoughts (Wei et al., 2022), latent reasoning refers to an implicit reasoning mechanism applied to the latent states (e.g., intermediate outputs) of the model. As in Transformer-based models, latent reasoning is typ- ically implemented as spatial and temporal recurrences. Spatial recurrences implicitly deepen the model by iterating over the latent states between the Transformer layers (Hao et al., 2024; Cheng & Van Durme, 2024; Zhang et al., 2025a; Shen et al., 2025), while temporal recurrences refine the latent states through iterations across input tokens (Dao & Gu, 2024; Geiping et al., 2025). Under the recurrences is the idea of scaling up test-time computation for inference; however, this requires the recurrent modules to be pre-trained. In similar vein to those, we iterate over the unified image-text latent states only at test time, without introducing and updating any model parameters.
Reinforcement Learning for Reasoning, Reinforcement learning (RL) has been the key to elicit- ing the reasoning ability of large language models (DeepSeek-Al) et al., 2025; OpenAI al., 2024). Inspired by the success of GRPO, which is an improved RL algorithm over PPO (Schulman et al., 2017) and is used in DeepSeek-RI (Shao et al., 2024), researchers have extended it to the multi- modal understanding domain, including visual question answering (Huang et al., 2025b; Liu et al., 2025b). image generation, RI has also been shown to effective (Jiang et al., 2025a; Tong et al., 2025; Jiang et al., 2025b; Pan et al., 2025b; Duan et al., 2025; Pan et al., 2025a; Liu et al., 2025a; Xue et al., 2025; Xiao et al., 2025), but has been used as a training-time optimization method. Unlike them, employ simple algorithm, REINFORCE (Williams, 1992), for test-time optimization.
METHOD
3.1 REASONING-AUGMENTTI MULTIMODAL. IMAGE GENERATION
We are interested in a framework for reasoning -wwgnineed image generation that natively supports language reasoning before image synthesis. A typical implementation is through unified multimodal understanding and generation (MUG; Chen et al. (2025); Deng et al. (2025)). Suppose the language token sequence := f1:M t,, f2,... ... tM, the image token sequence v U1:N U1, U2,--. UN, and the given instruction c, MUG defines multi-modaa modal image generation as an autoregressive gener- ation process:
N ptt, v(c) = II pv, Ui: t, c)
M II p(tma (tmmm m==
(I)
The generation of depends on the reasoning via language tokens t, which are generated from By analogy to textual reasoning, we refer to the generation of v as visual reasoning. To produce the final image, is further passed through a pixel decoder: V, p(-lt, c). Since focus on discrete image tokens produced by pre-trained discrete VAE (Chen et al., 2025), the pixel decoder p(-It, v, c) becomes deterministic given
Let R(V,,o c) denote a reward model that scores the compatibility between the given instruction c and the final image Vy, the goal of test-time reasoning-augmenteed image generation is to find an
3

---

Published as a conference paper at ICLR 2026
Final Text: Three colorful birds perch side by side branch-greet green the left, red in the center, and blue on...
Final Image:
Imagge Decuder
4
f
EM
V1
1/
4
VN
Text Token Decoder
Image Token Decoder
Reward
Unified Latent Space
Model
(t)
(L)
(t) M
(v)
(v)
(v) M
(v)
Reward
1
3
4
M+1
M+2
M43
M+4
M+N
Unified Multimodal Generation Model
Instruction: A photo of three birds
Figure 2: Overview MILR. MILR performs test-time latent reasoning in a unified latent space; uses policy gradients to iteratively refine text image latents ( z((), guided by a reward model. The reward model scores each generated image conditioning on the instruction.
optimal pair (t", v*) that maximizes the expected reward under p(-It, v, without modifying any model parameters:
arg mag t.v
In the case of MUG, the model itself can act as the reward function because of its multimodal understanding ability; alternatively, any off-the-shelf model that has such capability can serve the same role (see discussion in Section 4.4). Due to the infinite search space, the problem defined by Equation 2 is generally intractable.
3.2 IMAGE GENERATION VIA TEST-TIME LATENT REASONING
3.2.
MULTI-MODAL LATENT REASONING
Rather than searching over discrete image and text tokens, we propose searching in the unified latent vector space, that is, searching over their continuous vector representations. As MUG, these vectors correspond to the intermediate model outputs at respective token positions. They lie in a vector space that encodes both image and text tokens, offering a unified view of visual and textual reasoning' and facilitating cross modal reasoning (see Figure 2).
Formally, denoting the latent representations of image and text tokens by 2 (t) ((t) 21.M> respectively, where z(v), 2()) IR are the outputs from the same Transformer layer and thus lie shared d-dimensional vector space, we can rewrite Equation 2 as:
21:N and ((
z arg max Eyypt [R(V,,c))
(3)
z
where z = (2: (()) indicates the multimodal latent representation of the token sequence [t, v]. We refer this optimization problem as multimodal latent reasoning.
Given the optimal z from specific model layer, to produce the final Vy. we need to continue the forward pass until it is decoded into discrete tokens v). Thus, the pixel image generation becomes:
p(Vllz", = p(V/tt, v. c)p(t, vlz"),
where p(t, v/zz) represents the remaining forward pass of MUG starting with z".
We refer "reasoning" iterative updates in the latent space rather than text-hased chain-of-thoughh

---

Published as a conference paper at ICLR 2026
3.2.2 GRADIENT-BASED OPTIMIZATION FOR LATENT REASONING
In general, the problem defined by Equation 3 does not admit closed-form solution, so we resort to REINFORCE (Williams, 1992), a policy gradient optimization method. We note that it has been applied to fully textual reasoning in language tasks (Li et al., 2025a), but we, for the first time, extend it to unified multimodal latent reasoning for image generation.
With REINFORCE, we can formulate the cross-modal optimization process as:
z k+1 + z" + [(z") J(z") = Evyro(- [R(V, V) vlg()( )]]
(5)
(6)
where 17 is the learning rate. For efficiency, we choose as z the outputs of the last Transformer layer, Le., the inputs to the final modal-specific decoding heads. Moreover, we approximate gradients J(z) using a single sampled pair (t, v). The gradients are back-propagated only to the model outputs z, without altering any model parameters, and thus making MILR a test-time reasoning method.
Naively, we would optimize all M + N latents in 21:M++N but searching using only the guidance of reward model potentially biased, and it does not leverage the generative capacity of MUG for better exploration; instead, we optimize only the first A.M (where X E (0, 1) latents for text. After decoding them into discrete tokens, we complete textual reasoning via the standard autoregressive generation conditioned on them. As we will later see, this simple strategy strikes a good balance between efficiency and performance (see Section 4.3.1). For visual reasoning, we adopt similar strategy and optimize the first N, (where A E (0, 1) latents. 3 This is further supported by the observations of Hu et al. (2025): the first few tokens govern the global structure of the image, while the remaining tokens primarily influence high-frequency details."
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Benchmarks and Baselines. We conduct experiments on three benchmarks widely used in image generation: GenEval (Ghosh et al., 2023), T2I-CompBench (Huang et al., 2023), and WISE (Niu et al., 2025). We compare MILR against the following three sets of baselines:
Nea-rnngg models refer to the classical image generation models that synthesize images from given instruction in single shot, without refining the instruction. We consider diffusion models such as FLUX. -dev (Black Forest Labs, 2024), DALL-E 3 (Betker et al., 2023), and SD3-Medium (Esser al., 2024), autoregressive models such LlamaGen (Sun et al., 2024) and Emu3 (Wang et al., 2024), and hybrid autogressive-diffusee models such as BAGEL (Deng et al., 2025) and GPT-4o (OpenAI, 2025).
Training-based reasoning models refer to the image generation models that acquire the reason- ing ability through training. including GoT-R1 (Duan et al., 2025), T2I-R1 (Jiang et al., 2025a), Flow-GRPO (Liu et al., 2025a), and GRPO- and DPO-tuned Janus-Pro (Tong et al., 2025).
Test-time reasoning models refer to the models that admit reasoning through tailored inference strategies, including Reflect-DiT (Li et al., 2025b) and ReflectionFlow (Zhuo et al., 2025), which uses language feedback, and Best-of-N and PARM (Guo et al., 2025), which rely on search.
Hyperparameters and configurations. We choose Janus-Pro (Chen et al., 2025), autoregressive model, as the MUG model. For the portions of text and image tokens that are optimized, we perform grid search
validation split sampled from GenEval and empirically set X 0.2 for text and X, = 0.02 for images (see Figure 5). We use the Adam optimizer (Kingma, 2014), where the learning rate is empirically reward model, following previous work (Liuu et al., 2025a; Jiang et al., 2025a; Tong et al., 2025). We
0.03. For each benchmark, we use its own evaluation toolkit the
We experimented with different optimization strategies, see details in Appendix A.I.
'In each iteration, the text token number M may vary while the image token number remains unchanged. 'Rather than optimizing a prefix of image token sequence, we can optimize random subset of image tokens, but this strategy proves worse than prefix optimization (see Table 5 in Appendix A.
5

---

Published as a conference paper at ICLR 2026
Table 1: Results on GenEval. The best score is in bold and the second best is underlined.
Method
Single Obj.
Two Obj- t
Counting
Colors
Position
Binding
Overall
Non-reassoing Madels
LlanaGien (Sun al., 2024)
0.71
0.34
0.21
0.58
0.07
0.04
0.32
(Wang 2024)
0.34
0.21
FLU.X.I-dev (Black Forest Labs, 2024)
0.98
0.79
0.73
0.77
0.22
0.45
0.56
DALL-E 3 (Belker al., 2023)
0.96
0.87
0.47
0.83
0.43
0.45
0.67
SDD--Medium (Esser
2024)
0.99
0.94
0.72
BAGEL (Deng et al., 2025)
0.99
0.94
0.81
0.88
0.64
0.63
0.82
GPT-4o (OpenAl, 2025)
0.99
0.92
0.85
0.91
0.75
0.66
0.85
Training-hant based Reasoning Models
GoT--1 (Duan et ol., 2025)
0.99
0.94
0.50
0.90
01,46
0.68
0.75
(Biang et 2025a)
0.99
Flo GRPO Too-GRPO (Liu et al., 2025a)
1.00
0.99
0.95
0.92
0.99
0.86
0.95
ReasonCien-R R (Zhang et ol., 2025h)
0.99
0:94
0.62
0.90
01.84
0.84
0.86
Janus- Pro-7B(+GRPO) (Tong al., 2025)
Janus-Pi PaossPrr-77i Pto-BB(-DPO) (Gio al., 2025)
0.99
0.89
0.65
0.92
0.82
0.72
0.83
Test Reasoning Models
Reflect-DiT (1. at,l. 2025b)
0.98
0.80
0.81
RelloctionFlow (Zhoo et al., 2025)
1.00
0.98
0.90
0.96
0.93
0.72
0.91
Pouus-Po-7BB(-ee- Enhanced Reasoning)
0.98
0.1
0.55
0.67
0.79
0.99
0,96
0.89
0.93
0.92
0.80
0.91
Janus-Pro Pro-PBo- (BPRAMM (Goo et al., 2025)
1.00
0.95
0.80
0.93
0.91
0.85
0.91
Janus-Pro-IB (Chen et
0.98
0.51
0.56
Janus-Pro- Panus-Po-IB+MILL.
1.00
0,91
0.78
0.92
0.86
0.86
0.89
Janus-Pro 7B (Chen al. 2025)
0.98
0.85
0.56
0.89
0.77
0.64
0.78
Janu--Po-7B-MILR
0.90
Table 2: Results on T21-CompBench and WISE. The best is in bold, and the second is in underlined.
T2I-CompBench
WISE
Color f
Shape f
Texture f
Spatial 1
Non-Spatial 1
Complex. f
Overall f
Avg ?
Non-reasoning Models
PixArt-o Chen al. (2023)
0.6690
0.4927
0.6477
0.2064
0.3197
0.3433
0,4465
0.47
FLUUX.I-Be Black Forest Labs (2024)
0.7407
0.5718
0.6912
0,2863
0.3127
0.3703
0.4957
0.50
DALL.BB Betker et (2023)
0,7785
0.6209
0.7036
0.2865
0.3003
0.3773
0.5112
SD3-Medium Esser al. (2024)
0.8132
0.5885
0.7334
0,3200
0.3140
0.3771
0.5244
0.42
Show-o et al. (2024)
0,5000
0,4100
0,4600
0,2000
0.3000
0.2900
0,3700
0.30
RAGEL Deng el. (2025)
0.8027
0.5685
0,7021
0,3488
0.3101
0.3824
0.5191
0.52
Truining-haae basnd Reasoning
T21-I liang et al. (2025a)
0.81.30
0,5852
0,7243
0.3378
0.3090
0.3993
0.528
0.54
GoT-RI Duan al. (2025)
0.5549
0.330
Janus-Pro Prou-PPo-BB+GRPPOO Tong et al. (2025)
0.7721
0,5366
0.7317
0,2869
0.3087
0.3697
0,5010
Test-time Reasoning Moullls
Show-o + PARM Guo et al. (2025)
0,7500
0.5600
0.6600
0.2900
0.3100
0.3700
0,4900
Janus Pro-TBeTTTextt Enhunced Remoningg)
0.7087
0.4419
0.5821
0,2547
0.3072
0.3761
0.4459
0.46
0.7089
0.4925
0.7089
0.3542
0.3262
0.3721
0.4938
0.52
Janus-Pro-1B Chen et (2025)
0.3411
0,2261
0.2696
0.0968
0.2808
0.2721
0.2478
0.26
Banus-Pro-IB-MI1
0.2796
0.4177
0.2796
0.2622
0.2613
0.3512
0.40
Januc-Pro-7B Chen et (2025)
0.6359
0.1528
0.4936
0,206
0.3085
0.3559
0.3921
0.35
Jumus-Pro7B-MMLL
0.5117
0.4613
conduct all experiments on single NVIDIA A100 80GB GPU. A discussion on model efficiency can be found in Appendix A.3.3.
4.2
MAIN RESULTS
MILR achieves state-of-the-an results on GenEval, one of the most widely-used benchmarks for image generation (see Table 1). It improves over the base Janus-Pro-7B by 0.17, with the largest increases obtained from Counting (+0.34), Position (+0.21), and Attribute Binding (+0.27). No- tably, MILR surpasses frontier non-reasoning models such as SD3-Medium, BAGEL, and GPT-4o (+12%). Compared with training-based reasoning models (c.g., GoT-RI and T21-R1), MILR per- forms better and requires no parameter tuning. For fairness, we also compare MILR with test-time reasoning models. Surprisingly, it outperforms ReflectionFlow and PARM (+4.5%) that rely on scal- ing up test-time computation, demonstrating the superiority of our test-time optimization method.
We further evaluate MILR on two additional benchmarks: T2l-CompBench and WISE (see Table 2). Again, it achieves the best performance on both, highlighting the robustness of our model. Specifi- cally, "21CCompBench, MILR improves over the base Janus-Pro-7B by large margin (+0.14)

---

Published as a conference paper at ICLR 2026
Ours
Janus-Pro
T2I-RI
BAGEL
Ours
Janux-Pro
T2I-RI
BAGEL
photo of three bails.
metailic desk lamp and flaffy sweeter.
Greut Willi China when Los Angoles, AM Chinal
flower that symbolizes purity China. [Reefrrving lotus]
Figure 3: Qualitative studies on theree benchmarks. Reasoning cues are highlighted red.
Table 3: Ablations of MILR on GenEval, 121-CompBench, and WISE
GenEval
121-CompBench
WISE
Method
Single Obj.
Obj.
Counting
Color
Pos.
Attr. Binding
Overall
Overall
Avg
Janus: Pro Pho-7B+MILR (ours)
1.00
0.96
0.90
0.98
0.98
0.91
0.95
0.5325
0.63
0.35
w/o Image
1.00
1.00
0.91
0.95
0.95
0.88
0.94
0,5210
0.61
w/o Text
1.00
0.95
0.88
0.1
0.97
0.89
0,5043
0.56
and slightly outperforms T21-R1, strong training-basse reasoning model. On WISE, which em-
phasizes world knowledge understanding, MILR outperforms the base Janus-Pro-7B (+80%) and the second-best model T21-R1 (+16.7%), implying the importance of reasoning in comprehending mowleddgeintensiin instructions (see our qualitative studies of reasoning trajectories in Figure 8).
MILR is capable of geometric, temporal, and cultural reasoning. Next, conduct quali- tative study on Knowledge-intensive WISE. Surprisingly. MILR demonstrates nontrivial ability in geometric, temporal and cultural reasoning (see Figure 3). Take the prompt "The Great Wall of China when it's 3 PM in Los Angeles". MILR correctly infers the time difference from its geo- metric knowledge of China and Los Angeles, and concludes that "The Great Wall at Dawn" of interest. As another example, MILR correctly infers that lotus symbolizes purity in Chinese culture.
Joint image-text reasoning in the unified latent space leads to the best performance. To better understand the contribution of each modality to the strong performance of MILR, we individually ablate the latent optimization of images and text, denoted by "w/o image" and "w/o text", respec- tively (see Table 3). First, we find that both of them exceed the base model (w/o MILR) by large margin (e.g., > 0.21 on WISE), and optimizing both modalities leads to the best performance. Inter- estingly, text-only optimization (w/o image) fares slightly better than image-only optimization (w/o text), and approaches the performance of our best model (+MILR) on all benchmarks, suggesting substantial room for improvement in the language understanding component of MUG-based image generation models.
4.3 ANALYSIS
4.3.1
HYPERPARAMETERS
We analyze three important hyperparameters of MILR: (1) the maximum optimization step T, (2) the portion of text tokens be optimized X, in text-only optimization, and (3) the proportion of image tokens to be optimized A, in image-only optimization. We perform analysis on all three benchmarks and analyses and 3 on held GenBval validation split. For each setting, we run MILR three times with different random seeds and report the average scores.
Scaling up the number of optimization steps improves performance on all benchmarks. MILR admits test-time compute scaling by increasing the number of optimization steps. We il- lustrate and compare three setups: MILR, text-only optimization, and image-only optimization (see Figure 4). On all benchmarks. increasing the number of optimization steps leads to consistent im- provements, with the best performance achieved at step 16, after which model performance platcaus. Moreover, among the three setups, optimizing both images and text (i.e., MILR) yields the best per-

---

Published as a conference paper at ICLR 2026
(a)
GenEval
(b)
121-CompBench
(C)
WISE
0.96
0.34
0..4
0.94
0.52
0.62
Soore
0.50
0.58
0.90
0.40
0.56
0.86
0.44
0.52
10 12 Stegs
10
20
10
14
16
14
Steps
Steps
MILR
w/o Image
w/o Text
Figure 4: Performance across three benchmarks for varying optimization steps.
(a)
(b)
Text Token Ratio Analysis
Image Token Ratio Analysis
0.97
0.96
0.96
0.95
Gen Score
GenEval Score
0.94
0.95
0.93
0.94
0.92
0.93
0.91
0.04 0.06 0.07 0.08 0.09 0.10 Image Token Ratin (Ay)
Text Token Ratio (Ac)
5 steps
10 steps
steps
Figure 5: GenEval scores with varying optimization ratios of text and image tokens.
formance for almost all steps, except for a few early steps on GenEval, suggesting that MILR is well-suited for test-time compute scaling.
Optimizing a moderate amount (e.g., 20%) of text tokens leads the best performance. When varying X from 0.1 1.0, text-only optimization fluctuates between 0.91 and 0.96, and reaches the highest score at X = 0.2 (see Figure 5 (a)). We observe the same trend for different numbers of optimization steps. All of these suggest that MILR relatively robust text optimiza- tion. Moreover, we empirically find that the optimal X 0.2 corresponds to the prompts that elicit more coherent reasoning (see examples in Figure 10).
Optimizing a tiny amount (e.g., 2%) of image tokens gives rise to the best result. Hu et al. (2025) have empirically shown that, autoregressive image generation, early-stage tokens govern the overall image structure, and perturbing the first 20% of image tokens results in significant struc- tural deviations. Therefore, we conservatively adopt a very small A,, varying from 0.01 to 0.1 (see Figure (b)). Surprisingly, optimizing only the first 2% of all image tokens already yields peak performance, while further increasing A tends to degrade it (see examples in Figure 12).
4.4 REWARD MODELS
The reward model is crucial component of MILR, providing learning signals for latent reasoning. Following previous work (Liu et al., 2025a; Tong et al., 2025), we have used the benchmark's 's evalu- ator as the reward model (denoted by was OracleReward), but in real-world scenarios, oracle rewards usually unknown, and it is difficult design domain -spaiine rewards. To show that MILR is effective without the reliance on we test it with a set of fff-hhe-shef reward models on GenEval:
SelfReward uses MUG itself (e.g., Janus-Pro in this work) to evaluate images.
GPT-4o represents frontier critic for assessing image quality (Hurst et al., 2024).
UniftedReward is specifically tuned for unified evaluation of MUG (Wang al., 2025).
8

---

Published as a conference paper at ICLR 2026
1.0
1:5
0.9
0.8
9:34
CenEval
0.6
0.5
8090
Counting
Colors
Posilion
Altri. Binding
Overall
Baseline
BaN MixodReward
SelfReward
GPT4-6
UniftedReward
MinddRwward
Oraclekeward
Figure 6: Performance with different reward models on GenEval.
Errorl: Textual Reasoning Collapse
Error2: Visual Reasoning Collapse
Error3: Reward Hacking
in white glove
Evahueter
positioned beige
distinctly beige glove heige glove beige
However
glove beige...
boloe
photo of four baseball gloves.
a photo four handbags
a photo of bench above spoon
Figure Error Case Study
MixedReward a composite critic for more comprehensive evaluation. It aggregates rewards from specialized models, including GroundingDINO (Liu et al., 2024) (evaluating object detec- tion), GIT (Wang et al., 2022) (judging colors), and @ (assessing aesthetics).
Unsurprisingly, Wer gives rise the best performance across all dimensions (see Figure 6). For oracle crities, all variants surpass the baseline in terms of the overall score. Notably, MILR remains relatively robust different reward models, except for S. which performs poorly Counting (around 0.5). Among non-oracle critics, + performs the best, suggesting that, in the absence of ora- cle rewards, can derive a strong universal reward model by combining specialized critic models. Moreover, MILR+ slightly outperforms the strong Best-of-N+ baseline (+2.4% under compa- rable computation (i.e., N = = 20), once again demonstrating the superiority of our method.
4.4.1 ERROR CASES
Though MILR has shown the best performance across all three benchmarks, observe three major failure modes and illustrate them respectively in Figure 7. Specifically, the failure modes include (1) Textual Reasoning Collapse. The regenerated chain of thoughts degenerates into repetitive phrases and becomes nonsensical for guiding image generation (e.g., repeated "beige glove"). (2) Visual Reasoning Collapse While textual reasoning fine, visual reasoning degrades prefix image tokens that govern the overall structure and subsequently adversely affects the generation of remaining tokens that control fine- -naiire details (e.g.. blurred handles). (3) Reward Hacking. Models exploit shortcuts to achieve high rewards, but the generated image does not align with the given instruction perfectly (e.g.. unmatched position relationship). This implies that the benchmark's evaluator limited in that it is not yet fully capable of spatial reasoning.
5 DISCUSSION
We acknowledge two primary limitations of MILR, First, we have focused on an implementation of MILR that builds upon autoregressive MUG, where both text and image tokens are generated autoregressively. Another strong paradigm of MUG is through diffusion image generation, where image tokens within the same image can attend to each other (Deng et al., 2025). is interesting to see if the effectiveness of MILR transfers. Second, MILR relies on reward model for learning signals, but, in practice, a perfect reward model usually does not exist, and difficult to design a -amain- reward model. In addition, experimental results have revealed that the strongest non-oracle reward model still lags behind the oracle reward. Thus, future work is well-suited for designing reward models that generalize like the unified reward model of Wang et al. (2025).

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
We have proposed MILR, a test-time latent reasoning method for multimodal image generation. MILR employs policy gradient optimization, guided by an image quality metric, to search over latent vector representations of discrete image and text tokens, leading unified multimodal rea- soning framework. We implement MILR with a unified multimodal understanding and generation model that natively supports language reasoning before image synthesis. Across three benchmarks, MILR achieves state-of-the-un results. We further perform an in-depth analysis; we find that jointly reasoning in the latent image-text space is the key its strong performance. Our qualitative studies also highlight MILR's nontrivial capability in temporal and cultural reasoning.
ETHICS STATEMENT
We strictly adhere the ICLR Code of Ethics. Below we elaborate on the ethical considerations relevant to our work and the measures we have put in place.
Safety of content. To reduce potential risks, we confine our text prompts those of the standard benchmarks, avoiding sensitive and offensive input. We manually reviewed the generated images and text used in the work and did not find harmful content.
Privacy and intellectual property. Our work does not involve processing personally identifiable in- formation, as we experiment exclusively on publicly available benchmarks and off -he-ll models, as per their respective licenses.
REPRODUCIBILITY STATEMENT
To ensure reproducibility, we present experimental details, including settings and hyperparameter configurations, in Section 4.1 and release our code at https://gihuub.com/spiigggng
ACKNOWLEDGEMENTS
Yanpeng Zhao acknowledges the support of the National Natural Science Foundation of China (12574467).
REFERENCES
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang. Joyce Lee, Yufei Guo, al. Improving image generation with better captions. Computer Science. https//ddc..oppppai..mm///peeeee
2(3):8, 2023.
Black Forest Labs. Flux. https///iiibbboombbbcccccceecsss 2024.
Miaomiao Cai, Guanjie Wang, Wei Li, Zhijun Tu, Hanting Chen, Shaohui Lin, and Jie Hu. Au- toregressive image generation with vision full-view prompt. arXiv preprint arXiv:2502.1696. 16965, 2025.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. International Conference on Learning Representations (ICLR), 2023.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025. URL
Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.1377 13171, 2024.
10

---

Published as a conference paper at ICLR 2026
Ethan Chem, Zhulin Hu, Steffi Chern, Siqi Kou, Jiadi Su, Yan Ma. Zhijie Deng, and Pengfei Liu. Thinking with generated images, 2025. URL
Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. International Conference on Machine Learning (ICML), 2024.
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang. Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du. Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun. Tian Pei. Tianyu Sun, T. Wang, Wangding Zeng. Wanjia Zhao. Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang. Yanhong Xu, Yao Li, Yao Zhao. Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning, 2025. URL https//arrxv..or//bb///555 /ttp://bb./50..144......
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie. Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 14683, 2025.
Chengqi Duan, Rongyao Fang. Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng, Hong- sheng Li, and Xihui Liu. Got-rl: Unleashing reasoning capability of milm for visual generation with reinforcement learning. arXiv preprint arXiv:250S. 2025.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederie Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference muchine learning, 2024.
Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing. arXiv preprint arXiv:2503.10633 2025.
Jonas Geiping, Sean Michael McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time com- pute with latent reasoning: A recurrent depth approach. In ES-FoMo III: 3rd Workshop on Efficient Systems for Foundation Models, 2025, URL http:://opeereviee.ne//fooa hitprr/www..o/ooommmodooooooooo -id--io6B 7t.
Dhruba Ghosh, Hannanch Hajishirzi, and Ludwig Schmidt. Geneval: An
framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
11

---

Published as a conference paper at ICLR 2026
Ian J Goodfellow, Jean Pouget-Abadis Mehdi Mirza, Bing Xu, David Warde-Farie Fardee-arley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng- Ann Heng. Can we generate images with cot? let's verify and reinforce image generation step by step. Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. arXiv preprint arXiv:2412.06766 2024.
Taihang Hu, Linxuan Li, Kai Wang. Yaxing Wang, Jian Yang, and Ming-Ming Cheng. Anchor token matching: Implicit structure locking for training-free ar image editing, 2025. URL hitps: 104444444
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-ing to-image generation. Advances in Neural Informa- fion Processing Systems, 36: (6:78723-78747 2023.
Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench++:: An enhanced and comprehensive benchmark for compositional -ot-tto-mmae generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025a.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749) 2025b.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703, 2025a.
Jingjing Jiang, Chongjie Si, Jun Luo, Hanwang Zhang, and Chao Ma. Co-reinforcement learning for unified multimodal understanding and generation. arXiv preprint arXiv:2505.17534 2025b.
Diederik P Kingma. Adam: A method for stochastic optimization. International Conference Learning Representations (ICLR), 2014.
Hengli Li, Chenxi Li, Tong Wu, Xuckai Zhu, Yuxuan Wang, Zhaoxin Yu, Eric Hanchen Jiang, Song- Chun Zhu, Zixia Jia, Ying Nian Wu, and Zilong Zheng. Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space, 2025a. URL https///arrv..or/////5555...333
Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Arsh Koneru, Yusuke Kato, Kazuki Kozuka, and Aditya Grover. Reflect-dit: Inference-tine time scaling for text-to-image diffusion transformers via in-context reflection. International Conference on Computer Vision (ICCV), 2025b.
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint Stti:25555.047700 2025a.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang. Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. European conference on computer vision, pp. 38-55. Springer, 2024.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning, International Conference on Computer Vision (ICCV), 2025b.
12

---

Published as a conference paper at ICLR 2026
Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world knowledge-informen semantic evaluation for text-to-image generation. arXiv preprint arXtv:2503.07265. 2025.
OpenAl.
Introducing Burooducing----imggg-eennnnnnn
40 image generation. 2025. Accessed: 2025-07-25.
https://openai.com/mnee)
OpenAI, Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameyy, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghor- bani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong. Eliz- abeth Prochl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclerc, Hadi Salman, Haiming Bao, Hao Sheng, Hart An- drin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O'Connell, Ian Osband, Ignasi Clavera Gilaberte, Iige Akkaya, Ilya Kostrikov, Ilya Sutskever, Irina Kofman, Jakub Pachocki, James Lennon, Jason Wei, Jean Harb, Jerry Twore, Jiacheng Feng. Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joaquin Quiñonner Candela, Joe Palermo, Joel Parish, Johannes Heidecke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga, Julie Wang, Kai Chen, Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Lemberg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang. Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng, Linden Li, Lindsay McCallum, Lindsey Held, Lorenz Kuhn, Lukas Kon- draciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tintor, Mason Meyer. Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yatbaz, Melody Y. Guan, Mengyuan Xu. Mengyuan Yan, Mia Glaese, Mianna Chen, Michael Lampe, Michael Malek, Michele Wang, Michelle Fradin, Mike McClay, Mikhail Pavlov, Miles Wang, Mingxuan Wang, Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowdhury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao, Paul Ashbourne, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes. Raz Gaon, Reah Miyara, Reimar Leike, Renny Hwang, Rhythm Garg, Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer, Samuel Miserendino. Sandhini Agar- wal, Santiago Hernande7, Sasha Baker, Scott McKinney, Scottie Yan, Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang, Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji. Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang. Tay- lor Gordon, Ted Sanders, Tejal Patwardhan, Thibault Sottiaux, Thomas Degry, Thomas Dimson, Tianhao Zheng, Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi, Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong. Vlad Fomenko, Weiyi Zheng, Wenda Zhou, Wes McCabe, Wojeiech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha, Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang, Zheng Shao, and Zhuohan Li. Openai ol system card, 2024. URL httpss/aaxii.orr////2422.....
Jiadong Pan, Zhiyuan Ma, Kaiyan Zhang, Ning Ding, and Bowen Zhou. Self-reflective reinforce- ment learning for diffusion-baar image reasoning generation. arXiv preprint arXiv:2505.22407, 2025a.
Kaihang Pan, Wendong Bu, Yuruo Wu, Yang Wu, Kai Shen, Yunfei Li, Hang Zhao, Juncheng Li, Siliang Tang, and Yucting Zhuang. Focusdiff: Advancing fine-grained text-image alignment for autoregressive visual generation through rl. arXiv preprint arXiv:2506.0m 05501, 2025b.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.063477 2017.
13

---

Published as a conference paper at ICLR 2026
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of of of-tthught into continuous space via self-distillation. arXiv preprint arXiv:2502.21074, 2025.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024.
Chengzhuo Tong, Ziyu Guo, Renrui Zhang, Wenyu Shan, Xinyu Wei, Zhenghao Xing, Hongsheng Li, and Pheng-Ann Heng. Delving into for image generation with cot: A study on dpo vs. grpo. arXiv preprinl 25xv: 2025.
Jianfeng Wang. Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-ted textet transformer for vision and language. Transac- fions on Machine Learning Research, 2022.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409, 18869, 2024.
Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multi- modal understanding and generation. arXiv preprint arXiv:2503.05256, 2025.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. thain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Ronald J Williams. Simple statistical gradient-followngg algorithms for connectionist reinforcement learning. Machine learning, 8(3):229-254) 1992.
Huimin Wu, Xiaojian Ma, Haozhe Zhao Yanpeng Zhao, and Qing Li. Nep: Autoregressive image editing via next editing token prediction. Advances in Neural Information Processing Systems (NeurIPS), 2025a.
Mingrui Wa, Lu Wang, Pu Zhao, Fangkai Yang, Jianjin Zhang, Jianfeng Liu, Yuefeng Zhan, Weihao Han, Hao Sun, Jiayi Ji, Xiaoshuai Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang, and Rongrong Ji. Reprompt: Reasoning-augmmented reprompting for text-to-image generation via reinforcement learning, 2025b. URL https://ariv.orr/bb/2555....7.
Yicheng Xiao, Lin Song, Yukang Chen, Yingmin Luo, Yuxin Chen, Yukang Gan, Wei Huang, Xiu Li, Xiaojuan Qi. and Ying Shan. Mindomni: Unleashing reasoning generation in vision language models with rgpo arXiv preprint arXiv:2505. 2025.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single trans- former to unify multimodal understanding and generation. International Conference on Learning Representations (ICLR), 2024.
Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo visual generation. arXiv preprint arXiv:2505.077 078/8, 2025.
Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:2502. 15589, 2025a.
Yu Zhang, Yunqi Li, Yifan Yang, Rui Wang. Yuqing Yang, Dai Qi, Jianmin Bao, Dongdong Chen, Chong Luo, and Lili Qiu. Reasongen-rl: Cot for autoregressive image generation models through sft and rl, 2025b. URL https///aarrivonraa////22227777
14

---

Published as a conference paper at ICLR 2026
Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Min- jia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-basee fine-grained image editing at scale. Advances in Neural Information Processing Systems, 37:3058-3000 2024.
Le Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed El- hoseiny, and Hongsheng Li. From reflection to perfection: Scaling inference-time optimization for text-to-image diffusion models via reflection tuning. International Conference on Computer Vision (ICCV), 2025.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 OPTIMIZATION ALGORITHMS
By default, MILR jointly optimizes text latent z and image latent 2( in each gradient descent step, which is denoted MILR-Joint. As alternativ, nating fashion (denoted by MILR-Alt), akin to the strategy used coordinate descent. We further propose another variant of MILR that first optimizes ztt) its approximate optimum, followed by optimizing (z its approximate optimum (denoted by MILR-T2V), We illustrate the three algorithms in Algorithm 1. Their performance on GenEval is presented in Table 4. We do not see significant differences among them, so we use MII.R-Joint by default because of its simplicity.
optimize ((() and z(") in alter-
Algorithm MILR
Require: Instruction c, Learning rate v, MUG model p. reward threshold т, text and image fraction de Ag (0, 1]. optimization steps K, Reasoning strategy S (JOINT, ALT, T2V)
2, t, v t p(t, vlc)
> Initial latent vectors
Vj p(-lv)
c))
> Reward Calculation
2 - (2
(2):Ar/e): 112.//
(e)
> Set X and A fraction b Starting step index
+
while k < K and < do
V₁, t LatenttResoninngs (z" ",, k. K, c.
> Select strategy: Algorithm
r c))
k- 1-1
end while return V/
Algorithm 2 LatentReasoning (default: JOINT)
(a) JOINT
(b) ALT
(c) T2V
1: x + k- +1. J(z
1: k mod
1 then
1: if S [K/2] then
2: Vy ~ , 2²
2y 2 (1)
- (( +
2:
2 (c)
1 -
+1
3: return V
3:
Ty ~ p(Trlzi (t) c)
3:
Ty ~ p(Ty
4:
Vy ~ p(V 2 C),c)
4:
Vf ~ (2 Ty,c)
(v)
5: else
5: else
Zk
(e) - (0) -1- (2
6:
Z
(n) (n) +
7:
Vy ~ p(Vrlz
(e) c),
7:
Vi ~ p(V 2 T),
(2
8: end if
8: end if
9: return V
9: return V,,
Table 4: Geneval results with different reasoning strategies.
Algorithms
Single Obj.
Two Ohj.
Counting
Colors
Position
Attr. Binding
Overall
MILR-JOINS
1.00
0.96
0.90
0.98
0.98
0.91
0.95
MILR-ALT
1.00
0.98
0.87
0.95
0.97
0.88
0.94
MILR-T2V
1.00
0.96
0.95
0.95
0.96
0.89
0.95
Table 5: Geneval results under random subset optimization image tokens.
Algorithms
Single Ohj.
Two Ohj.
Counting
Colors
Position
Attr. Binding
Overall
MILR (Image+Prefix)
1.00
0.95
0.88
0.91
0.97
0.89
0.93
MILR (Image=Random)
1.00
0.92
0.84
0.89
0.94
0.86
0.90
16

---

Published as a conference paper at ICLR 2026
A.2
QUALITIVE STUDY
A.2.1 REASONING TRAJECTORIES
MILR admits multi-roum round latent- seeettspace refining test time. In Figure visualize the full trajec- tories of rounds 2, 3, and 4 that finally arrive at the correct generation.
The object commonly udmired during the Mid-Autumn F Festival. [Referring to the moon]
The most iconic haircut that churacterized the women's 's fashion trend of the 1920s. [Referring to the bob]
The image depicts a full moon, surrounded by
Iteration
goiden glow, with aeeu scottered clouds the sky.
Visual imagel 1920 moat's haiistyle was featured! women'ss trend.
The moon the most commoniy admired object during the Mid-Aniume Festival.
Latent V
The 1920s women's fashion trend was characterized distinctive the "bol. The bub was short, sleek hairstyle that popularized celetvities such Louise Brooks and Clora Bow.
Mid a surromnded
Iteration
glow with a few lanterns hanging in the sky.
photo four donuts.
mentioned the Santa Claus]
during Christmas, [Referving
Four donats are displuped background. The donuts of differons colors: one is pinh. one is blue, one is yellow, und one is brown, They have smooth testure and are munnd shape.
rows with a white
The image depicts Christmas free with lurgo, red ornament the top. representing the most mentioned character duriny Christmas. The ornament surronnded by smaller, white ornamene,, symboliziing the character's popularity
Iteration
Latent F
large adorned ornaments Santa ornament glowing representing purounded Sauta ckaracter- Clo istmas.
Vinall visualization
calorfulut, arranged in a square formation, with golden glaze and sprinkle sugar top.
Iteration
hunging
V
Based visualization shows Santa bustling adorned ornaments red ornament brightly, slaped center labeled surrounded Santa churucter- Christmas, must meniioned choracter during Chrisimas.
The roundat in placed on white plate, with Iteration golden glaze a sprishle of powdered sugar.
a photo of yellow carrot.
photo three kites.
Iteration
yollow carnet with smooth, elongared shape and slightly rough textur.
A photo of three kites, each with distincs color and shape, flying the
Latent F
The image deplets a stagle yellow carror with smooth, elongoted shape slightly rough texture. The carrot the only object to image, and there other objects elements presout
their contrasting against the blue backdrap. The kites
sour
Iteration
shuped diamonds, They are positioned with one lite higher than the others, creating
depth and movement
Latent V
A photo three kites, each with distincs color, different colors: one another is bluc, and the third is groen. The background
Image a yellow carrot smooth, elungated shape, single. bright yellow color, and object
Iteration
bockground
there are other objects elements vistble the image.
Latent y
The colorful image depicts three likes flying sky. kites are flying with one kite beng yellow, amother blue, and the third red. The yellow kite top, the blue kite the middlle, and the kite
currot with smooth, elongated shape, yellow color, and slightly rough testure.
Iteration 3
yellow
buttom.
Figure 8: Example reasoning trajectories. Orange indicates prefix optimization, red highlights incor- rect reasoning, and green represents correct reasoning.
A.2.2 MODEL ABLATIONS
We provide a qualitative study accompanying the model ablations done in Section 4.2. Consistent with the quantitative results, single-modality optimization surpasses the base model but underper- forms joint optimization. Clearly, text-only optimization excels at tasks that rely on nontrivial nu- merical and compositional reasoning (e.g.. "a photo of four clocks/bowls/knivess or "three kites"), "), while image-only optimization tends to refine image structures by adjusting the spatial arrangement of objects (e.g., "a photo of a tie above a sink"). ). To illustrate this scenario, consider the prompt "a photo of four clocks" in Figure text-only optimization produces four spatially uncorrelated clocks,
17

---

Published as a conference paper at ICLR 2026
whereas image-only optimization tends to piece together four visually similar clocks the same wall. Interestingly, across all examples, image-only optimization exhibits tendency to zoom in on the objects of interest.
Ours
w/o MILR
w/o Image
w/o Text
Ours
w/o MILR
w/o Image
w/o Text
a photo of four clocks.
photo red dog.
UNT
photo of above sink.
photo pink skateboard and black train.
photo of four bowls.
photo of three kites.
photo of four knifes.
photo of black kite and green bear.
Figure 9: Example generations of MILR (ours), the base model (w/o MILR), text-only (w/o Image) optimization, and image-only (w/o Text) optimization.
A.3 ADDITIONAL ANALYSIS
A.3.1 ANALYSIS OF TEXT AND IMAGE OPTIMIZATION RATIOS
In Section 4.3.1, we studied how the text-token ratio A and the image-token ratio A, affect MILR. We also provide example language and image reasoning trajectories when varying A in Figure 10 and Figure 11, respectively. In Figure 12, we show how image generation evolves when varying A We empirically find that X 0.2 and X 0.02 generally lead better generation.
A.3.2 ANALYSIS OF REWARD MODELS
In Section 4.4, we evaluated MILR with different reward models: SelfReward, s GPT-4o, o UnifiedReward, MixedReward, and war OracleReward (see Table 6, Table and Table 8). We further provide example images generated under each setup in Figure 13, Overall, wa establishes an upper bar of generation quality, and @ among non-oracle reward models, performs best because incorporates multiple specialized critics to provide comprehensive assessment. Though still lags behind wir. in the absence wir, we can compose strong universal reward model from specialized off- shellf image critics.
Table 6: GenEval results with different rewards. The best performance is in bold.
Reward
Single Obj.
Two Obj
Counting
Colors
Position
Autr. Binding
Overall
Baseline
0.98
0.85
0.56
0.89
0.77
0.64
0.78
SelfReward
1.00
0.90
0.50
0.88
0.77
0.65
0.79
GPT-4o
0.98
0.94
0.80
0.88
0.77
0.68
0.84
UnifiedReward
1.00
0.92
0.76
0.89
0.81
0.66
0.84
MixedReward
1.00
0.90
0.83
0.89
0.88
0.75
0.87
OracleReward
1.00
0.96
0.90
0.98
0.98
0.91
0.95
18

---

Published as a conference paper at ICLR 2026
Prompt: a photo of an apple and donut. Initial Text Reasoning: The apple is red and round, with smooth texture. The donut is brown, round, and has a slightly rough texture. They are placed next each other on a white plate.
Latent Optimization:
4₁ 0.1
Visual vibrant depicts orange apple and chocolate donut. <eos"
2, 0.2
Visual vibrant depicts orange with don donut displays red apple and golden brown donut. <eos
Visual vibrunt depicts a orange with don donul displaysin in image hus round with a hole in the center, and bright rod apple with a smooth skin. <eas>
2, 0.3
Visual vibrant depicts a orange with don donut displaysia in image has round with has with while smooth green applc. <<eos>
2,
Visual vihrant depicts orange with don donut displaysin Image has round with has with while smooth green, and appleuts <eos>
4, 0.5
Visual vihrant depicts orange with don donut displays'n image vound with has with while smooth green, and appleuts has golden and with with donut. <eos>
4, 0.6
Visual vihrant depicts orange with don donut displays in image round with has while smooth green, and It appleuts has golden and with with with covered powdered crumb raised <eos>
4, 0.7
Visual vibrant depicts a orange with don donut displays image has round with has with while smooth green, and It appleuts hos golden and with with with covered powdered crumb raised surface with The sit next to each other. <cos>
4 0.8
Visual vibrant depicts orange green, and appleuts has golden and sit next side to one other. <enx>>
donut displays image has round has with while smooth with covered powdered crumb raised surface with The
A, 0.9
Visual vibrant depicts o orange with don donut displays'w in image has round with hus with while smooth green, and It appleuts has golden and with with with covered powdered crumb raised surface with The sil next side to one other. whut plute surface with The apple and donut are placed on, <eas>>
A = 1.0
Figure 10: Case study of the text token optimization ratio A, The underlined text decoded from the optimized latents. <eos>> denotes the end-of-sentenee token.
0.1 2, 0.2 2, 0.3 2=0.4 =.5 = 1,
= 4, 0.9 4, = 1.0
a photo of apple and donut.
a photo three dining tables.
BOTHT GILLE
photo of real kite and blue bow!.
Figure
Generated images with varying token optimization ratio Aę-
Table T21-CompBench results with different reward models. The best performance bold.
Reward
Color
Shape
Texture
Spatial
Non-Spatial
Complex
Overall
Baseline
0.6359
0.3528
0.4936
0.2061
0.3085
0.3559
0.3921
Self Reward
0.7196
0.4620
0.5887
0.2379
0.3055
0.3868
0.4501
GPT-4o
0.7624
0.4701
0.6561
0.2778
0.3102
0.3881
0.4775
Wnwrd
0,8208
0.4609
0,5835
0.3210
0.3044
0.3700
0.4651
MixedReward
0.8009
0.4765
0.6608
0.4077
0.3055
0.3745
0,5043
OracleReward
0.8508
0.5117
0.6949
0.4613
0.3078
0.3684
0.5325
19

---

Published as a conference paper at ICLR 2026
Ay 0.01 A, = 0.02 A, = 0.03
Ag 1F-1
0.04
A,
0.05
4.
0.06
A,
0.07
0.08
A,
0.09
2, 0.1
phoso of apple and
photo of four handhags
photo of is bench above spoon.
Figure 12: Generated images with varying image token optimization ratio Au-
Table 8: WISE results with different reward models. The best performance in bold.
Reward
Cultural
Time
Space
Biology
Physics
Chemistry
Overall
Base
0.30
0.37
0.49
0.36
0..42
0.26
0.35
Self Reward
0,40
0,43
0.53
0.38
0,44
0.23
0.41
GPT-4o
0.53
0.53
0.62
0.52
0.53
0.34
0.52
Wnwrd
0.45
0,54
0.60
0.46
0,52
0.30
0.48
MixedReward
0.48
0.51
0.61
0.44
0..1
0.33
0.49
Metric Reward
0.64
0.65
0.72
0.66
0.71
0.37
0.63
SelfReward
GPT-4o
MixedReward UniffedReward
OracleReward
photo of four frishees.
photo of computer keyboard above snowboord.
photo of a yellow pizza green oven.
photo of dining table above suitcase.
photo of four handbags.
Figure 13: Case study various reward models.
A.3.3 EFFICIENY ANALYSIS
We summarize the computing resources used by different image generation models in Table 9. Com- pared with training-based reasoning approaches such T21-R1 (Jiang al., 2025a), Janus-Pro with
20

---

Published as a conference paper at ICLR 2026
DPO/GRPO (Tong al., 2025), and Flow-GRPO (Liu et al., 2025a), test-time reasoning method achieves the best performance, without relying on curated reasoning data for training. As an exam- ple, Flow-GRPO requires about 2K A800 GPU hours for training but only matches MILR, not to mention its labor cost in curating reasoning data. Moreover, when compared with the strong test-time reasoning method Best-of-N (N=20 T=20 in MILR), MILR surpasses it by 0.04 while requiring less inference time because it employs an early-stop strategy, i.c., it stops once the generated image satisfies evaluation criteria.
Table 9: Efficiency comparisons.
Method
#GPU
GPU
Training Time
Inference Time (s/sapple) 1
Training Data
GenEval Score
T21-RI (liang et al., 2025a)
HSD0
16h
0.79
2025)
Janas-Pro-DDP (Tong et 2025)
A100
~9h
0.83
Flow-GRPO (Liu al., 2025a)
24
A800
~100 h
0.95
Reflect-DiT (Li 2025b)
A6000
24hh
104.1
0.81
Janus-Pro-7BB Best-of-N, N-20)
52.07
MILR (ours)
A100
0
32,54
0.95
A.4 LATENT REGULARIZATION TO MITIGATE REWARD HACKING
As discussed in Section 4.4.1, MILR can suffer from reward hacking, which refers scenario where a MUG model achieves high rewards but exhibits low image generation quality. To mitigate this issue, we adopt a simple lg regularizer
constrain the difference between the latents before and after optimization. Let Zinit denote the initial latents produced by frozen MUG model before optimization, and let J(z²) denote the original objective used at the latents optimization step (see Equation (6)), we define the regularized objective as:
(xe [2*)) - 3 B1|x** Zinit
(7)
where B controls the strength of regularization. Intuitively, this term encourages the optimized latents z to stay close to the initial latents, thus maintaining image generation capability similar to that the frozen MUG model.
In experiments, use small coefficient 0.05, which empirically stabilizes the opti- mization and alleviates semantic collapse, without degrading generation quality. In Figure 14, we illustrate that adding latent regularization helps mitigate reward hacking.
+1
+L2
4
photo of bench above spoon
a photo of dining table above suitcase.
Figure 14: l2 regularization on latents mitigates reward hacking.
A.5 THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used LLMs (c.g., OpenAI ChatGPT) only to polish writing and to create 1TTX algorithms and math equations. LLMs did not contribute to our research idea, model design, experimentation, etc. All contents of the paper have been verified by the authors.
21


Reviews:
[{"review_id": "ZC0RHGfyHh", "reviewer": "Reviewer_bDbb", "summary": "The authors introduce MILR, a novel method for improving multimodal image generation during test time. In this approach, they perform reasoning in latent space jointly over unified image and text. The use REINFORCE method to optimize the intermediate latent vectors, with the reward model that assesses how well the generated image aligns with the input.\nThey show that MILR significantly improves the performance of Janus-Pro models on several benchmarks, achieving state of the art results.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- Their approach of performing joint reasoning in latent space is very interesting and novel, as well as, achieves state of the art performance.\n- The paper is well written, and the experiments are thorough.", "weaknesses": "- The authors did not discuss about latency. Given that their technique involves iterative optimization at test time, there will be significant increase in latency.\n- This method is heavily dependent on the reward function. There is a possibility of reward hacking, as authors also mentioned. One way to resolve it would be to add a regularizer like penalize semantic incoherence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FPj1Aylp4V", "reviewer": "Reviewer_uX5G", "summary": "This paper introduces MILR, a novel method designed to improve text-to-image generation. The core contribution is a test-time optimization technique that reasons jointly over text and image representations within a unified latent space. Instead of refining raw text or image pixels, MILR operates on the intermediate latent vectors of a pre-trained MUG model. It employs a policy gradient method to iteratively update these latent vectors before they are decoded into the final output.", "questions": "Could you elaborate on the choice of REINFORCE as the optimization algorithm and the choice of the pre-trained model?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Novel and elegant proposal for test-time latent reasoning, which enhances powerful pre-trained models without requiring any fine-tuning.  \n\nImpressive empirical results across multiple challenging benchmarks.", "weaknesses": "The term \"reasoning\" is debatable here. The process is more accurately described as a guided latent space search that optimizes a latent vector to maximize a reward, rather than a structured, logical thought process (like reasoning defination in LLM).\n\nThe most significant flaw is that the headline, state-of-the-art results reported in Tables 1 and 2 are achieved using the benchmark's own evaluation toolkit as the reward model. This  setup is not representative of any realistic application. In practice, a perfect reward function is never available.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ytSoAJtnWg", "reviewer": "Reviewer_ncVe", "summary": "The paper introduces MILR, a test-time optimization method for text-to-image generation that performs joint reasoning over text and image in a unified latent space. Concretely, the authors take intermediate hidden states z = [z(t); z(v)] from a unified multimodal understanding-and-generation (MUG) model (Janus-Pro; Chen et al., 2025) and iteratively update a prefix of the text and image token latents using REINFORCE-style policy gradients (Williams, 1992) to maximize a reward measuring text–image compatibility. The approach leaves model weights frozen and only optimizes latents at inference time. The paper reports strong gains on three benchmarks—GenEval (Ghosh et al., 2023), T2I-CompBench (Huang et al., 2023), and WISE (Niu et al., 2025)—and analyzes hyperparameters (prefix lengths λt≈0.2, λv≈0.02; steps up to 16) and ablations (text-only, image-only).\n\n  Empirically, Janus-Pro-7B+MILR achieves 0.95 overall on GenEval (vs. 0.78 base Janus-Pro-7B; larger category jumps include Counting +0.34, Position +0.21, Attribute Binding +0.27), outperforms test-time strategies like PARM and ReflectionFlow (both ~0.91), and approaches training-time RL methods such as Flow-GRPO (0.95). On T2I-CompBench, MILR improves overall from 0.3921 to 0.5325; on WISE, from 0.35 to 0.63 (the paper states +80% over its base and +16.7% over T2I-R1). The authors discuss failure modes (textual/visual reasoning collapse, reward hacking) and note reliance on reward models, sometimes using the benchmark’s evaluator (“oracle” reward).", "questions": "- Compute fairness: For Best-of-N, PARM, and ReflectionFlow, please report per-sample wall-clock time, total forward passes, and early-stop statistics. Are the search budgets strictly matched across methods and benchmarks?\n  - Generality to diffusion MUG: Can you show MILR with a diffusion-based unified generator (e.g., Show-o or a diffusion-tokenizer model) to demonstrate portability beyond Janus-Pro?\n  - Reward robustness: Can you report full results with only non-oracle rewards (e.g., MixedReward) across all three benchmarks, including failure modes and qualitative examples? Any signs of reward hacking under these critics?\n  - Unified vs. separated latent loops: Could you add an ablation that optimizes text and image latents in separate spaces without sharing a unified latent layer, to isolate the benefit of the unified representation more directly?\n  - Sensitivity to prefix choice: Beyond contiguous prefixes, did you try structured subsets (e.g., entropy or attention-based selection for z(v)) that may better target global-structure tokens? You mention random subsets are worse; a principled selection could help.\n  - Out-of-benchmark generalization: Any tests on prompts outside the evaluator’s training (for OracleReward), or user study-aligned metrics (e.g., paired preference) to complement automated scores?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Clear technical core: optimizing a prefix of multimodal token latents z(t), z(v) via policy-gradient updates at test time, without finetuning parameters. The optimization target and the forward path are precisely stated (Eq. (3)–(6)).\n- Strong empirical gains across three widely used evaluations: GenEval (NeurIPS 2023; object-focused alignment), T2I-CompBench (NeurIPS 2023; compositionality), and WISE (Niu et al., 2025; knowledge-intensive prompts). Results are specific and category-level improvements are reported.\n- Solid ablations: text-only vs. image-only vs. joint, token prefix ratios (λt, λv), and step counts; joint optimization consistently performs best and compute scaling is characterized (best around 16 steps).\n- Practicality: method is test-time-only and uses a single A100 in reporting; no curated reasoning data required, contrasting with GRPO/DPO-style training (e.g., Flow-GRPO; Liu et al., 2025a; T2I-R1; Jiang et al., 2025).\n- Thoughtful discussion of failure modes (reasoning collapse, reward hacking) with qualitative evidence; acknowledges reward-model dependence.", "weaknesses": "- Reward reliance / oracle leakage: Many claims hinge on using the benchmark’s evaluator as the reward (“OracleReward”), which risks overfitting/reward hacking and inflating benchmark scores (authors themselves show mis-evaluations in spatial relations). More evidence with non-oracle, public reward models would strengthen claims of generality beyond GenEval’s metric.\n- Base-model coupling: All main results use Janus-Pro. The approach should be validated on a diffusion-based MUG (e.g., Show-o or a diffusion-tokenizer system) to test portability; current evidence is limited to one AR MUG paradigm.\n- Baseline fairness and compute: Best-of-N and ReflectionFlow/PARM comparisons need tighter compute normalization. The paper mentions “comparable compute (N=T=20)” with early stopping, but wall-clock, sample counts, and variance should be reported per benchmark to rule out search budget confounds.\n- Unified-latent novelty vs. prior latent/test-time reasoning: Prior latent-space test-time computation and “latent reasoning” lines (e.g., Geiping et al., 2025; Hao et al., 2024; Shen et al., 2025; Dao & Gu, 2024) are cited but the empirical isolation of “unified cross-modal latent” as the key driver remains partial. The ablation “w/o image” vs. “w/o text” helps, but an ablation that replaces unified latents with modality-specific latents in separate loops would more directly test the unified-space hypothesis.\n- Reward models and robustness: MixedReward and other non-oracle critics improve over baseline but still lag the oracle. Robustness across unseen prompts/domains (outside the three benchmarks) is not demonstrated.\n - Tuning on GenEval validation: λt/λv tuned on a GenEval split (then used elsewhere) may introduce slight bias. Cross-validated tuning or tuning-once on a separate development set would reduce concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g2lmmwgVBy", "reviewer": "Reviewer_Cur9", "summary": "The paper introduces MILR, a test-time reasoning method that enhances image generation by jointly reasoning over image and text in a unified latent vector space. Utilizing a policy gradient method guided by an image quality critic, MILR operates within the Multimodal Understanding and Generation (MUG) framework, optimizing intermediate model outputs at test time without fine-tuning.", "questions": "# Questions:\n1. How does MILR handle cases where the reward model provides inconsistent feedback across iterations?\n\n2. What are the potential impacts of varying λt and λv values on different types of image generation tasks?\n\n3. Can MILR be adapted to work with other MUG frameworks beyond Janus-Pro, and if so, what modifications would be required?\n\n4. How does the method perform on real-time image generation tasks with strict latency constraints?\n\n5. I read the code in the supplementary material. Are there plans to release the code to reproduce this work?\n\n\nOverall, I think this work is interesting. If the author could address my concerns, I am willing to increase my rating to 8.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "# Strengths:\n(+) MILR achieves superior performance across all tested benchmarks, outperforming both training-based and test-time reasoning models.\n\n(+) The method's test-time optimization avoids the need for curated reasoning data or model fine-tuning, making it cost-effective and practical.\n\n(+) Qualitative studies highlight MILR's nontrivial abilities in geometric, temporal, and cultural reasoning, enhancing its versatility.", "weaknesses": "# Weakness:\n\n(-) The reliance on a reward model for optimization may introduce bias, potentially limiting exploration of the generative capacity of MUG.\n\n(-) The method's efficiency and performance depend on empirical hyperparameter settings (e.g., λt = 0.2, λv = 0.02), which may not generalize across all scenarios.\n\n(-) The paper lacks a detailed discussion on computational costs and scalability, which could be a concern for large-scale applications. Besides, this paper claims latent test-time reasoning. I wonder about its cost (GPU Memory, Per Time, Reward Loss across time and step).\n\n# Minor Weakness:\n\n(-) Fig. 2 appears to have an issue, as it suggests the reward model only receives the final image, while the text tokens should also be input to assess compatibility accurately.\n\n(-) The format of data input to the reward model is unclear; if it takes in data format like final text and final image, the computational load seems excessive—why not directly calculate the loss based on the latent vector z?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)