From Broad Exploration to Stable Synthesis: Entropy-Guided Optimization for Autoregressive Image Generation
Authors: Han Song
Yucheng Zhou
Jianbing Shen
Yu Cheng
Venue: ICLR 2026 Poster (2026)
Keywords: Language Models
Autoregressive Image Generation
Chain-of-Thought
Primary area: foundation or frontier models, including LLMs

Abstract: Combining Chain-of-Thought (CoT) with Reinforcement Learning (RL) improves text-to-image (T2I) generation, yet the underlying interaction between CoT's exploration and RL's optimization remains unclear. We present a systematic entropy-based analysis that yields three key insights: (1) CoT expands the generative exploration space, while RL contracts it toward high-reward regions; (2) final reward is strongly negatively correlated with both the mean and variance of image-token entropy, highlighting the need to reduce uncertainty and instability; and (3) the entropy of the textual CoT directly governs downstream image quality, with lower-entropy CoTs leading to better generations. Motivated by these findings, we propose Entropy-Guided Group Relative Policy Optimization (EG-GRPO), a fine-tuning strategy that reallocates optimization budget by uncertainty: low-entropy tokens are excluded from reward-driven updates to preserve stability, while high-entropy tokens receive an entropy bonus that encourages structured exploration without collapse. Experiments on standard T2I benchmarks demonstrate that EG-GRPO achieves state-of-the-art performance.

Published as a conference paper at ICLR 2026
FROM BROAD EXPLORATION TO STABLE SYNTHESIS: ENTROPY-GUIDED OPTIMIZATION FOR AUTOREGRES- SIVE IMAGE GENERATION
Han Song"", Yucheng Zhou²>, Jianbing Shen', Yu Chengi The Chinese University of Hong Kong, Univvrrity of Macau hansongelink cuhk.eue yucheng. zhou@ um. edu. chengyuêcse. cuhk. edu.hk
ABSTRACT
Combining Chain-of-Thougee (CoT) with Reinforcement Learning (RL) improves text-to-image (T21) generation, yet the underlying interaction between CoT's ex- ploration and RL's optimization remains unclear. We present systematic entropy- based analysis that yields three key insights: (1) CoT expands the generative ex- ploration space, while RL contracts it toward high-rewart regions; (2) final re- ward is strongly negatively correlated with both the mean and variance of image- token entropy, highlighting the need to reduce uncertainty and instability; and (3) the entropy of the textual CoT directly governs downstream image quality, with lower-entropy CoTs leading to better generations. Motivated by these findings, we propose Entropy-Guided Group Relative Policy Optimization (EG-GRPO), a fine-tuning strategy that reallocates optimization budget by uncertainty: low- entropy tokens are excluded from reward-driven updates to preserve stability, while high-entropy tokens receive an entropy bonus that encourages structured exploration without collapse. Experiments standard T2I benchmarks demon- strate that EG-GRPO achieves of-tte----th--rrr performance. Our code available at /tttpp:://iittub..mmm http:///www...ooooooneeettttee/e GRPO
INTRODUCTION
Text-to-image (T2I) generation has progressed rapidly with large-scale pretraining and strong au- toregressive and diffusion architectures (Chen et al.) 2023 Labs 2024; Wann et al.) 2024 Zhou ctt al., 2024), yet two core challenges remain: (i) balancing exploration for diversity against ex- ploitation for reward-aligned fidelity, and (ii) ensuring generation stability under repeated sampling. Chain-of- Thouuhh (CoT) prompting promises richer semantic planning (Wei et al.) 2022), while rein- forcement learning (RL) directly optimizes task preference rewards (Jiang al. 2025). However, how CoT's exploratory behavior interacts with RL's optimization in T2I, and how this interaction governs uncertainty and stability, has not been systematically understood.
We analyze entropy dynamics in autoregressive T2I models that combine CoT with RL (via Group Relative Policy Optimization, GRPO (Jiang et al.) 2025) and use Shannon entropy to quantify token-level uncertainty in both modalities: textual COT tokens and image tokens. Three empirical findings emerge. First, CoT expands the exploration space, broadening the entropy distribution of generated outputs, whereas RL contracts this space toward higher-reward regions. Second, final reward exhibits strong negative correlation with both the mean and the standard deviation of image-token entropy, indicating that reducing uncertainty and instability is central to quality. Third, the entropy of the textual CoT directly influences downstream image quality: lower-entropy CoTs yield tighter, higher-reward clusters under stable sampling.
Guided by these findings, we propose Entropy- Guided Group Relative Policy Optimization (EG- GRPO), token-level modification of GRPO that reallocates gradient budget by uncertainty. Low- entropy (high-confidence) tokens are excluded from reward-driven updates, retaining only the KL-
* Equal contribution. "Corresponding Author.

---

Published as a conference paper at ICLR 2026
to-reference term preserve stability and previously acquired knowledge. High-entropy tokens receive an entropy bonus added to the advantage, encouraging structured exploration and accelerat- ing uncertainty reduction where it matters. A batch-level calibration ties the bonus magnitude to the mass saved on low-entropy tokens, keeping update scale close to GRPO, and the bonus vanishes at GRPO equilibrium, preserving the stationary points of the base objective (Wang al. 2025c).
We evaluate EG-GRPO on T21-CompBench (Huang et al. 2023) and WISE (Niu et al. 2025) using Janus-Pro autoregressive backbone in a discrete latent space (Chen al.) 2025) and standard reward pipeline combining human-preference scoring, object grounding, and VQA signals (Wu al. 2023 Liu et al. 2024, Wang et al., (022). EG-GRPO attains state-of the-ahr results, with pronounced gains in compositional generalization (e.g., attribute binding and object relations). Ab- lations that apply entropy guidance to only CoT tokens or only image tokens underperform the full model, confirming the need to control uncertainty in both semantic planning and visual decoding.
The main contributions of our paper can be summarized as follows:
We provide a quantitative account of the CoT-RL interaction through entropy dynamics: CoT expands exploration, RL contracts toward high-reward regions; reward is strongly negatively cor- related with entropy mean and std of ima; and textual CoT entropy governs downstream image quality.
We introduce an entropy-guided, token-level refinement of GRPO that protects confident tokens via KL-only updates and focuses optimization on uncertain tokens via an entropy bonus, with calibrated budget and qquilibiium-vanisiine properties.
On Tee-CompBench and WISE, EG-GRPO achieves state-of-tte-art performance and reduces uncertainty and instability in line with the analysis.
RELATED WORK
2.1 TEXT-TO-IMAGE GENERATION MODEL
Text-to-image generation has been advanced along two major paradigms: autoregressive model- ing and diffusion- based approaches (Team et al.) 2025 Zhou et al., 2025). On the autoregressive side, Parti (Yu et al. 2022) demonstrates that large-scale transformer models can achieve impres- sive compositional generation by predicting image tokens sequentially. Fluid (Fan al., 2024) further explores continuous tokens and generation order, showing improved efficiency and quality. STAR (Ma et al., 2024) introduces scale- wise autoregressive framework that progressively gen- erates images from coarse to fine scales, while JetFormer (Tschannen et al. 2024) directly models raw images and text in unified autoregressive manner without discrete tokenization. More recently, NextStep-1 (Team et al. 2025) scales continuous- autoregressive generation to 14B parame- ters, achieving strong performance in both image synthesis and editing. parallel, diffusion models have become dominant for high-quality synthesis. VQ-Diffusion (Gu et al. 2022) combines vector- quantized representations with diffusion for discrete latent modeling, and ERNIE-ViLG 2.0 (Feng 2023) extends this to harge-scale multilingual text-to-image generation. UPainting (Li et al.) 2022 introduces cross-modal guidance to unify simple and complex scenarios, while RPG (Yang jet al. 2024) enhances controllability via recaptioning, planning, and region-based diffusion. These works illustrate the complementary strengths of autoregressive and diffusion frameworks in advanc- ing the controllability, fidelity, and scalability of text-to-image generation.
et al.
2.2
CHAIN OF THINKING AND REINFORCEMENT LEARNING
Recent works have explored integrating Chain-of-Thought (CoT) reasoning with reinforcement learning (RL) to improve text-to-image generation. Visual-CoG (Li et al.] 2025) introduces stage-aware RL with intermediate rewards across semantic planning, refinement, and evaluation, while ReasonGen-R1 (Zhang et al. 2025b) and T21-R1 (Jiang al. 2025) incorporate rationale- augmented data and bi-level reasoning chains optimized via GRPO. PromptEnhancer (Wang et al.] 20256) further demonstrates that CoT-based prompt rewriting with RL can enhance image quality without modifying the generator, and verification- methods also inject preference alignment during synthesis (Zhang al. 2025a). Beyond explicit Co'T, RL has been applied for alignment
2

---

Published as a conference paper at ICLR 2026
in diffusion models, such comparing DPO and GRPO (Tong et al. 2025 subject-driven pref- erence optimization (Miao et al. 2024). and DPOK fine-tuning with KL regularization (Fan et al.
2023). Related multimodal reasoning approaches, including ImageGen-Col (Liao et al. 2025) and reflective CoT for retrieval (Wu et al. 2024), highlight the broader potential of structured reasoning. Combining CoTT and RI. provides promising avenue for enhancing controllability. interpretability. and human alignment in text-to-image generation. While SimpleAR (Wang et al., 2025a) and Gal- lici & Borde (025) demonstrate the efficacy of standard GRPO for high-fidelity and style-aligned autoregressive generation, our approach introduces an entropy -uuidde mechanism to reallocate the optimization budget at the token level dynamically.
PRELIMINARIES
We begin by formalizing autoregressive text-to-image image generation, introducing Shannon entropy measure generative uncertainty, and reviewing Group Relative Policy Optimization (GRPO) for model refinement.
3.1 AUTOREGRESSIVE GENERATION IN DISCRETE LATENT SPACE
Autoregressive text-to-image models operate in a discrete latent space. An image / is first encoded into tokens (zı, .2..... 21.) lihood given a text prompt factorizes autoregressively:
via pre-trained tokenizer such VQ-VAE, The conditional like-
p(
c) =I0 p(z) 2).,
(1)
i=1
policy me parameterized by 0 models these conditionals. At step i, the policy outputs a categorical distribution Ta(- Zeg; c) over the vocabulary V, from which 2; is sampled. The final image is reconstructed by decoding the full sequence 2.
3.2 ENTROPY AS PREDICTIVE UNCERTAINTY
For each step i, the policy distribution admits Shannon entropy:
H(mo(: Ret,, c)) = Σ Py logPj-
(2)
jeV
where Py denotes the probability of token j. Low entropy reflects confident, often high-fidelity pre- dictions with reduced diversity, whereas high entropy reflects uncertainty, encouraging exploration and diverse generations the cost of possible incoherence.
3.3 GROUP RELATIVE POLICY OPTIMIZATION
Group Relative Policy Optimization (GRPO) refines generative policies using relative rewards with- out requiring a value function. For prompt c, the policy me samples G candidate sequences
1 1 Te(- c), (*) (i)
(i)
)
(3)
Each sequence receives reward ,(0, which is normalized within the group:
#
M (r())
2)3,
- max o.=
(4)
1=1
1=1
Broadcasting (1) to all tokens yields the training objective
T
LCRPO - Σ
Σ
T
Σ
(1)
log (o( | c, + B DRI.( Tret),
(5)
t=1
where 2 0 controls a KL regularizer toward a reference policy. This group- normaliza-
tion yields scale-invarian invariant advantages and stable updates, making GRPO well-suited for aligning autoregressive generators with task-specific rewards.

---

Published as a conference paper at ICLR 2026
4 ANALYSIS OF ENTROPY DYNAMICS IN TEXT-TO-IMAGE GENERATION
To investigate the role of uncertainty in text-to-image generation, we analyze generutive entropy as quantitative indicator of model behavior. Our study focuses on disentangling how CoT and RL fine-tuning affect entropy within the pipeline. By examining their individual and combined effects, clarify how these techniques balance exploration and exploitation, and how the resulting entropy dynamics define the optimization objective for high-quality visual synthesis.
4.1 THE DICHOTOMY OF EXPLORATION AND EXPLOITATION: COT vs. RL
We begin analysis by examining the distinct yet complementary roles
orange cat and a yellow
of Chain-of-Thought (CoT) prompt- ing and reinforcement learning (RL) fine-tuning. For each textual prompt, we generate multiple image candi- dates under three settings: the base- line model (Janus-Pro), the base- line augmented with CoT reason- ing (Jamus-Pro+ Col") and a GRPO- finetuned variant built upon Janus- Pro+CoT (T2I-RI). To investigate their differences, we assess each gen- erated image by jointly measuring its output mean entropy and reward score, and visualize the resulting distribu- tions in two-dimensional space.
T2I-CoT
orange yellow
Model
o
O
expanded exploration space but unstable performance
TZI-COT-RL
onange
yellow
O
promes low loe-quality CiT reduced veriance
Figure 1: Comparison of different texl-to-image generation methods: (a) autoregressive text-to-image generation, (b) CoT, and (c) with CoT and GRPO optimization.
As illustrated in Figure 2 the introduction of CoT significantly broadens the entropy dis- tribution of the generated images. shows that CoT expands the model's exploratory space, enabling
033
0.5
jenue Proedot 12k-1
generate more diverse set of outputs. This expanded space contains both low- and high-reward samples, indicat- ing that CoT itself does not guarantee qual- ity but rather increases the range of gener- ative possibilities. Conversely, the applica- tion of Group Relative Policy Optimization (GRPO), which results in the T21-R1, leads to a notable contraction and leftward shift of the entropy distribution. This demonstrates that the model learns to exploit the vast space un- locked by CoT, converging towards much narrower, lower-entropy region that consis- tently yields higher rewards. This reveals a complementary relationship: CoT serves to expand the exploratory landscape, while RL acts as a refinement mechanism to exploit this landscape and guide the model towards stable, high-quality regions.
([Rward)
-35
Entropy Imag Mean
Frez.
Figure Entropy-reward distributions different methods. CoT (Janu-Pro+CooT) expands the exploratory space with more diverse outputs, while GRPO fine-tuning (T2I-RI) contracts it toward higher-reward regions, yield- ing more stabilized, high-quality generations.
4.2 UPSTREAM INFLUENCE: How COT'S TEXTUAL ENTROPY GOVERNS IMAGE QUALITY
Since Chain-of-Thought serves the entry point of exploration, we further examine how the intrin- sic uncertainty of the textual CoT affects downstream image generation. To isolate this effect, we focus subset of samples where image generation remains stable, defined as those with entropy variance below small threshold of 0.011 across multiple runs. This filtering minimizes confound- ing factors due unstable sampling, allowing us probe the impact of CoT entropy directly.
4

---

Published as a conference paper at ICLR 2026
Prompt: brown bird and pink sheap
3.0
32
2.5
Reward
1.5
2.8
1.0
0.5
Date (Image Endropy 0.0111 Ritled Iie Islope -1.764]
0.22
0.26
0.26
0.26
0.30
0.32
Entropy
Image Entropy Meen
Figure 3: Left: Reward vs. CoT entropy (stable cases, Image Entropy Std < 0.011). Higher CoT entropy correlates with lower image reward. Right: Reward distributions across different CoTs for the prompt. Images from same CoT cluster together, with certain CoTs consistently yielding lower rewards.
As shown in Figure (Left), observe clear negative correlation between the mean entropy of the textual CoT and the average reward of the resulting images. Intuitively, high-entropy CoTss correspond less coherent or more uncertain reasoning traces, which tend degrade visual quality. In contrast, low-entropy CoTs provide more confident and consistent reasoning, ultimately leading higher-reward generations. This analysis grounded on the stable subset, ensuring that the observed trend is not an artifact of sampling instability.
To further dissect this phenomenon, we visualize the reward distributions of images conditioned on different CoTs for the same prompt (Figure 3 Right). Each distinct CoT forms a compact cluster the reward space, highlighting
consistent influence on generation outcomes. Notably, certain Co'T' repeatedly produce clusters with lower average rewards, suggesting that the quality bottleneck determined upstream, at the textual reasoning stage. These findings demonstrate a direct trans- mission of uncertainty from text to image: the entropy of CoT reasoning acts critical upstream factor that governs the attainable quality of visual outputs.
4.3 THE LEARNED OBJECTIVE: MINIMIZING UNCERTAINTY AND INSTABILITY FOR HIGHER REWARDS
1.4
-Fier the (rape 34
3.0
12
2.0
1.5
0.
1.0
Regaiive
0.-
1.0
Fitac Ino loloo: 2.041
02
Fitted line (alppe 1,040
Fhann sats S5t
0.50
2.02
2.44
0.00
0.00
6.00
6.2
0.11 Entropy standdard deviation
6.02
3.03
5.04
0.2
25
0.6
Image Entropy Stal
Image Entropy Mean
Figure 4: Left: Reward entropy std. Higher instability (larger std) consistently lowers reward. Middle: Relation between entropy std (x-axis) instability strengthens the negative correlation. Right: Reward vs. entropy mean under (std > 0.03). Large std implies exploratory generation where RL has not converged; in this regime, reducing mean entropy is especially beneficial.
negative correlation of reward-entropy mean (y-axis). Greater
cases
We next investigate how the fully RL-trained model internalizes entropy control to optimize for re- ward. For each prompt-CoT pair, multiple images are generated, and their entropy statistics aggregated. Specifically, the mean entropy characterizes the global level of uncertainty in the gen- erative process, while the standard deviation (std) captures instability across different runs.
Empirical results reveal a consistent negative correlation between reward and standard deviation of entropy (Figure 4 Left). This indicates that generative instability inherently detrimental: mod-
5

---

Published as a conference paper at ICLR 2026
els that produce more consistent entropy trajectories across samples yield higher-quality outputs. Beyond this first-order observation, we further examine how instability modulates the role of un- certainty. Figure | (Middle) demonstrates that the negative correlation between reward becomes progressively stronger as std increases. In other words, instability amplifies the adverse impact of uncertainty, rendering mean entropy more decisive factor under high-variance conditions.
entropy and
To isolate this effect, we analyze the high-variance regime (std 0.03), where the image token generation remains exploratory and the RL policy has not yet converged. As shown in Figure (Right), in such cases, lowering mean entropy proves particularly effective in improving reward. This finding suggests that, when the model operates in an unstable exploratory state, suppressing overall uncertainty a critical pathway toward higher-quality generations.
These analyses indicate that the RL agent implicitly optimizes a compound objective: simulta- neously minimizing overall uncertainty (mean entropy) and reducing instability across runs (entropy std).
5 ENTROPY-GUIDER GROUP RELATIVE POLICY OPTIMIZATION
Building on Section we introduce a token-level optimization scheme that preserves GRPO's group-relative structure while reallocating updates toward uncertain parts the generation pro- cess. Entropy-Gutin Guided GRPO (EG-GRPO) applies to both textual CoT and image tokens and adds an entropy-based bonus only where uncertainty high.
DESIGN PRINCIPLES FROM ENTROPY ANALYSIS
Section showed that: (i) CoT broadens exploration while RL contracts it toward high-reward gions; (ii) rewards are negatively correlated with both the mean and the variance of token entropies; and (iii) textual (CoT) entropy causally influences downstream image quality. We adopt three prin- ciples:
Focus on uncertainty. Allocate more update mass high-entropy tokens to reduce instability where it matters.
Protect confidence. On the lowest-entropy tokens, set the reward-driven advantage to zero so that only the KL-ttorrferrence acts, preventing drift on confident regions.
3.
Stay reward-driven Retain the GRPO group-relative objective; entropy contributes an additive per-token bonus at high entropy without replacing advantage.
5.2 REVISITING GRPO AND TOKEN BROADCAST
For prompt c, policy my samples G sequences relative advantages (6) GRPO optimizes
{o with rewards (r(0) normalized group-
G
7(
LGRPO
Σ
(0) log (o(o
B (II(TT Teet)-
(6)
G
T
i=1
This broadcasts the same coefficient A to all tokens of sequence i, ignoring per-token uncertainty and potentially wasting gradient budget on already-confided tokens.
5.3 ENTROPY-GUIDEE TOKEN SELECTION
Let H H(mo(- c,02()) be the Shannon entropy token of sequence i, and define the normalized entropy 17(1)) & H(()) /log |V] E 10. For each sequence i and each modality ftext, image) independently compute per-sequence percentiles and define
(i.m) top-50% by ())))) (f,m) bottom-20%
s s(i.m) = remaining.
hi
We rank and threshold entropies textual CoT and image tokens separately, consistent with Section

---

Published as a conference paper at ICLR 2026
Introduce masks M U, E (0,
M 1lt $ Si 1
u = Ule "
so M2 removes reward-driven updates on low-entropy tokens, and U.!) = marks high- entropy tokens to receive bonus.
Budget view. Let Plo Punid, Phi be the fractions of tokens in S( ss(im) s(im) (phi 0.5, Si
Plo 0.2, Pmid = 0.3). GRPO's per-sequence coefficient budget is B -
1 E | =
T
|4 Under EG-GRPO,
(
(
Σ M A0) Asg[[t,"
T
with À 0 and goopggradeent sg|.]. Zeroing low-entropy updates saves roughly PlolA() of mass and reinvests it on high-entropy tokens through the additive bonus {g][
Proposition 1 (Per-batch budget balance). For batch B, choose
A K. Σies | Eres(t.m)
T
with K. E (0, 1].
(8)
EiEB TG Elesfiim (H(
Then Eg B to Appendix sense.
Es Bss[Crrol A detailed derivation and calibration discussion deferred Setting k = yields batch-level budgel neutrality in the calibrated upper-bound
Fixed-point neutrality. Because X scales with E, |A the bonus vanishes at GRPO equilib- rium where group-relative advantages cancel. See Appendix[B.2 [B2 formal proof.
Corollary 5.1 (Preserving GRPO stationary points). If A for all i and X x with 1, then X = and A = 0 for all t; EG-GRPO reduces to the KI. regularizer and preserves the stationary point.
5.4 ENTROPY-BIASED ADVANTAGE
We modify the broadcasted coefficient at token of sequence by
A (
M ( A + Ui()) Asg (1,
where M removes reward-driven updates the lowest-entropy 20% tokens and T,!) adds an entropy bonus on the highest-entropy 50% tokens. The EG-GRPO loss is
1 LEG-GRPO = 2 i=1
A log (o
( c, 0
+ 3 DKI(TO Teer.. (10)
and the reference-policy KL unchanged; low-entropy tokens therefore governed solely by KL when M(()) 0.
Reward-shaping view. Define a token-level pseudo-reward p(() (8)
Tgp M, + {sg U,!),
(1)
where reip induces (66), Then E V, log (o
7(() is an unbiased policy-gradient estimator
for augmented objective whose baseline component is GRPO.
5.5 WHY EG-GRPO REDUCES UNCERTAINTY AND PRESERVES KNOWLEDGE
(A) Targeted entropy reduction. For small steps, the update at token is proportional to
A Va log (o ( 1). On high-entropy tokens, A A + Asg(H.()) strengthens positive
updates and attenuates negative ones, lowering entropy where is largest under softmax parameter- izations.

---

Published as a conference paper at ICLR 2026
(B) Stability confident tokens. When M reward-driven gradients vanish and only KL-tt-rrferrence remains, protecting confident regions from drift and preserving learned knowledge.
(C) Proximity to GRPO equilibrium. With À = X and x = 1, the batch-wise coefficient budget matches GRPO (Appendix [.I and the bonus disappears at equilibrium (Appendix) B.2). reallocat- ing update mass without altering stationary points.
EXPERIMENTS
6.1 EXPERIMENTAL SETTINGS
Training Details. Following (Jiang et al. 2025). we train policy the same 6,786 text prompts drawn from T21-CompBench (Huang et al. 2023), which contain only textual descriptions without paired images. The prompts accompanied by structured object attribute annotations that were automatically extracted using GPT-4o mini in prior work (Jiang et al.] (025). The policy backbone is initialized from lanus-Pro-7B (Chen al.) 2025). Optimization
a learning rate of x 10-6 and KI. coefficient, 33 = 0.01. For the reward pipeline, we combine HPS (Wu et al,, 2023) as the human-preferenee estimator, GroundingDINO (Liu al., 2024) the object detector, and GIT (Wang et al. 2022) as the VQA model. The object-relation module (ORM) is implemented by finetuning LLa VA-One One Vision-7B following the procedure of Gf et (2025)
Benchmark. To assess the effectiveness of our approach, we rely on two established evaluation suites: 121-CompBench (Huang et al. 2023) and WISE (Niu et al. (025). T21-ComBBench provides 6,000 prompts designed to test compositional generalization. The benchmark covers three broad categories: attribute binding, object relations, and complex compositions, which are further split into six sub-categories, such as color/shape/texture bindings, spatial and non-spatial relations, and multi- object compositions. In contrast, WISE focuses knowledge-intensive reasoning. It contains I 1,000 prompts spanning cultural commonsense, the model to infer what specific entity situation should appear the image. For WISE, since the corresponding reasoning instructions from prior work (Jiang et al., 2025) were not released, we reimplemented them ourselves and provide the exact templates in the Appendix [foo transparency. For both benchmarks, we otherwise follow the official evaluation protocols.
reasoning, and natural science, requiring
6.2
MAIN RESULTS
Model
T2I-CompBench
WISE
Color
Shape
Texture
Culture
Spatio-temporal
Science
PixArt-a (Chen et al.
2023
66.90
49.27
64.77
45.00
49.00
46.33
SD-v1.5 (Rombach et al.
2022
37.58
37.13
41.86
34.00
33.50
26,00
FLUX. dev (Labs 2024
74.07
57.18
69.22
48.00
60.00
42.67
Emu3 Wang
75.44
57.06
71.64
34.00
46.50
37.67
Show-o Xie ct al.
2024
56.00
41.00
46.00
28.00
44.00
35.33
Janus-Pro-7B (Chen ct
202.5
63.59
35.28
49.36
30.00
43.00
34.67
T21-R1" Jiang et al.
2025
82.58
58.67
76.94
48.00
55.50
45,00
EG-GRPO
84.11
60.88
77.38
49.00
56.00
46.33
Table 1: Comparison of models across T21- "2l-CompBenchh and WISE. Spatio-temporal is the average of Time and Space; Science the average of Biology, Physics, and Chemistry. "Results are obtained by evaluating the officially released model under the same experimental settings for fair comparison.
As shown in Table T EG-GRPO achieves the strongest results on 121-CompBench, surpassing all baselines Color (84.11) Shape (60.88), and Texture (77.38), with particularly notable gains on Shape binding. On WISE, EG-GRPO improves over T2I-R1 in Culture (49.00 vs. 48.00) and Sci- ence (46.33 vs. 45.00) while maintaining comparable performance in Spatio-tempores (56.00 vs. 55.50). These consistent improvements demonstrate that entropy-gue guided updates effectively hance compositional reasoning and robustness while preserving the stability of knowledge learned by the base model.
8

---

Published as a conference paper at ICLR 2026
Model
T21-CompBench
WISE Spatio-temporal
Color
Shape
Texture
Culture
Science
EG-GRPO
84.11
60.88
77.38
49.00
56.00
46.33
w/ only sem
81.29
55.68
74.10
47.00
56.00
43.67
w/ only tok
79.25
53.73
72.46
45.00
56.00
43.00
w/o All
82.58
58.67
76.94
48.00
55.50
45.00
Table Ablation results of EG-GRPO. We compare the full method with variants that apply entropy guidance only to textual CoT tokens (w/ only sem), only to image tokens (w/ only tok), not at all (w/o All).
6.3 ABLATION STUDY
Table 2 summarizes the effect of applying entropy guidance on different token types. The full EG- GRPO model, which adds entropy bonuses both textual CoT tokens and image tokens, achieves the best overall performance. In contrast, applying entropy guidance to only textual CoT tokens (w/ only sem) only image tokens (w/ only tok) leads degraded performance. This suggests that introducing entropy regulation in single modality can create an imbalance during optimization, which may be more harmful than applying no entropy guidance at all. The baseline without entropy guidance (w/o All) performs worse than the full EG-GRPO model, confirming that the proposed entropy-awan updates are essential for improved compositional generalization.
6.4 ANALYSIS ON DIVERSITY
In this section, we address trade-off between entropy reduction and generative diversity. While entropy is measure of uncertainty.
Step
Diversity (Vendi Score)
100
2,7305
reducing raises the question of whether the model's creative ex-
200
2.7233
pressivity and output diversity gate this, we employ the Vendi Score (Friedman & Dieng 2023). reference-free metric designed quantify diversity in machine learning models. Additional qualitative diversity comparisons are provided in Sec.
negatively impacted. To investi-
400
2.7212
600
2.7151
800
2.7159
Table 3: Evolution of diver- sity during the GRPO train- ing process. As the model converges towards high-reward regions, a slight reduction in diversity is observed, reflect- ing the exploration-exploitae trade-off.
Dynamics of Diversity during RL. The RL naturally contracts the exploration space to focus on high-reward regions. As shown Table our analysis of the GRPO training dynamics confirms this expected behavior: as the model optimizes for reward-aligned fi- delity from Step 100 to 800, the diversity naturally shows a slight decrease.
Preserving Diversity at Similar Quality. Crucially, our method targets instability rather than semantic diversity. To verify this, we compared EG-GRPO against the baseline (T21-R1) on filtered subset of generated samples where both models achieved similar quality scores (defined
as [QQulity]] < 0.1). The quality score an aggregate of BLIP-2 (Li et al. 2023). LAION-
Aesthetics (Beaumont et al., 2022),, and PickScore (Kirstain al. 2023
As shown in Table 4 EG-GRPO maintains Vendi Score (2.593) virtually identical to the baseline (2.592) under the same quality con- straints. This indicates that the Entropy Bonus (Section 5.3) successfully preserves valid ex- ploration pathways while suppressing "bad" uncertainty that leads to instability, rather than collapsing the model into single mode.
Metric
EG-GRPO
T2I-R1
Quality Mean Diversity (Vendi Score)
13.86
13.85
2,593
2.592
Table 4: Comparison of Diversity (Vendi Score) be- tween EG-GRPO (Ours) and T21-R1 (Baseline) on subset of samples with similar quality scores. Our method maintains diversity comparable to the baseline.
6.5 ANALYSIS OF ENTROPY
Figure compares the entropy distributions of EG-GRPO and T2I-R1 for textual CoT tokens (left) and image tokens (right). The entropy measures the conditional uncertainty of token decoding under a given plan or partial image, reflecting generation stability rather than sample-level diversity.

---

Published as a conference paper at ICLR 2026
In Figure 5 EG-GRPO reduces both the mean and variance of entropy, with a markedly stronger effect image tokens. This indicates more confident and stable vi- sual token generation, where high entropy typically corresponds to decoding instability or local artifacts rather than meaningful vari- ation.
9M
Enpayy
Moreover, lower token-level entropy does imply reduced diversity. EG-GRPO is de- signed to suppress bad uncertainty while pre- serving good diversity across samples, as di- versity primarily emerges the sample level rather than the token level. To avoid prema- ture collapse, we introduce an Entropy Bonus (Eq. 99 that explicitly encourages exploration in high-entropy regions. Consistent with this design, the Vendi Score (as mentioned Sec. 6.4) shows that sample diversity remains stable throughout training, with only mild decrease as the model converges to high-reward regions, reflecting a normal exploration-exploiaiion trade-off rather than mode collapse.
tG torouet
12-R20megel
Figure 5: Entropy distributions of EG-GRPO T21- R1: left for textual CoT tokens, right for image tokens.
6.6
CASE STUDY
a fabric hat and a flufly sweater
small brown kitchen with white wood floor.
leather
glass metallic atomizer spray
yellow daisy and purple violet
The cylindrical vase with metang ular openings and cubic flaral arrongement added a pop color to the diamond catree table.
Figure 6: Qualitative case study of our method on diverse prompts. These results are randomly sampled.
As shown in Figure our method consistently produces high-quality generations across wide range of prompts. All examples in this case study are randomly sampled. Our method captures fine-grained attributes such as colors and textures with higher fidelity, preserves coherent spatial layouts in complex scenes, and maintains stability when composing multiple objects. These results confirm that entropy-guided optimization enhances both the accuracy and consistency text-to- image generation.
CONCLUSION
We studied entropy dynamics in text-to-image generation, showing that CoT expands exploration while reinforcement learning contracts it toward stable, high-reward regions. Both the mean and variance of entropy strongly predict image quality, motivating our Entropy-Guided GRPO (EG- GRPO). By protecting low-entropy tokens and focusing updates on high-entropy ones, EG-GRPO balances stability with structured exploration. Experiments on "21-ComBBench and WISE confirm its state-of-the-ant performance and reduced instability, underscoring uncertainty control as a key principle for advancing text-to-image generation.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Romain Beaumont, Christoph Schuhmann, and contributors. Laior-al/hesthhiii-prccccoo A linear estimator on top of clip to predict the aesthetic quality of images. htttps://githm: com/
2022. Commit "main", MIT license.
Junsong Chen, Jincheng Yu, Chongjian Ge. Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.78//1 2025.
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint arXiv:2410. 13863, 2024.
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances Neural Information Processing Systems, 36:79858-79885. 2023.
Zhida Feng, Zhenyu Zhang, Xintong Yu, Yewei Fang, Lanxin Li, Xuyi Chen, Yuxiang Lu, Jiaxi- ang Liu, Weichong Yin, Shikun Feng, et al. Ernie-vilg 2.0: Improving text-to-image diffusion model with mnowledg--enbanee mixture-ot soture-of-denoisiin exeeorsing-experts.. experts. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10135-10145, 2023.
Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. Trans. Mach. Learn. Res., 2023, 2023, URUhttps::/ hptp:ree/e/peeeeeeeeeeee http:/iiwwweeeeoeuneeeeee
1d==970Hb@@yk]
Matteo Gallici and Haitz Sáez de Ocáriz Borde. Fine-tuning next-scale visual autoregressive models with group relative policy optimization. arXiv preprint 1.Xiv:2505.2333: 2025.
Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. Proceedings of the IEEF/CVF conference computer vision and patlern recognition, pp. 10696-10706, 2022.
Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Rui Huang. Haoquan Zhang, Manyuan Zhang, Jiaming Liu, Shanghang Zhang, Peng Gao, et al. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint 2rXiv:2501.13920 2025.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-7874 2023.
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and leken-level cot. arXiv preprint arXiv:2505.00700 2025.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset user preferences for text-to-image generation. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL
Black Forest Labs. Flux. http
2024.

---

Published as a conference paper at ICLR 2026
Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. BLIP-2: bootstrapping language- image pre- tre-training with frozen image encoders and large language models. In Andreas Krause, Emma Brunskill. Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning. ICML. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 19730-19742. PMLR, 2023. WRRLhttts://ppcceeeeet wttpeeeiiiinnm.m.pppmpmmmm mlr prwss/pssssee... wwtppppwweeeeeeeeeeeeee htmm
Wei Li, Xue Xu, Xinyan Xiao, Jiachen Liu, Hu Yang. Guohao Li, Zhanpeng Wang. Zhifan Feng. Qiaoqiao She, Yajuan Lyu, et al. Upainting: Unified text-to-image diffusion generation with cross-modal guidance. arXiv preprint arXiv:2210. 16031, 2022.
Yaqi Li, Peng Chen, Mingyang Han, Bu Pi, Haoxiang Shi, Runzhou Zhao, Yang Yao, Xuan Zhang, and Jun Song. Visual-cog: Stage-aware reinforcement learning with chain of guidance for text- to-image generation. arXiv preprint arXiv:2508. 110008..800 18032, 2025.
Jiaqi Liao. Zhengyuan Yang, Linjie Li. Dianqi Li, Kevin Lin, Yu Cheng, and Lijuan Wang. Imagegen-cot: Enhancing text-to-image in-context learning with chain-of-thought reasoning. arXiv preprint arXiv:2503.19312 2025.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pp. 28-55. Springer, 2024.
Xiaoxiao Ma, Mohan Zhou, Tao Liang. Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: Scale-wise text-to-image generation via auto-regressive representations, arXiv e-prints, pp. arXiv-2406 2024.
Yanting Miao, William Loh, Suraj Kothawade, Pascal Poupart, Abdullah Rashwan, and Yeqing Li. Subject-driven text-to-image generation via preference-based reinforcement learning. Advances in Neural Information Processing Systems, 17:123563-122555 2024.
Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint urX25 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the ence on computer vision and pultern recognition, pp. 10684-10695, 2022.
the IEEE/CVF confer-
NextStep Team, Chunrui Han, Guopeng Li, Jingwei Wu, Quan Sun, Yan Cai, Yuang Peng, Zheng Ge, Deyu Zhou, Haomiao Tang, et al. Nextstep-1: Toward autoregressive image generation with continuous tokens at scale. arXiv preprint 250. 10711, 2025.
Chengzhuo Tong, Ziyu Guo, Renrui Zhang, Wenyu Shan, Xinyu Wei, Zhenghao Xing, Hongsheng Li, and Pheng-Ann Heng. Delving into for image generation with cot: A study on dpo vs. grpo. arXiv preprint arXiv:2505.17017, 2025.
Michael Tschannen, André Susano Pinto, and Alexander Kolesnikov. Jetformer: An autoregressive generative model of raw images and text. arXiv preprint arXiv:2411. 2024.
Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-text transformer for vision and language. arXiv preprint ar Xiv: 2205.14100 14100, 2022.
Junke Wang. Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXiv:2504.1 1455, 2025a.
Linqing Wang, Ximing Xing, Yiji Cheng, Zhiyuan Zhao, Jiale Tao, Qixun Wang, Ruihuang Li, Xin Li, Mingrui Wu, Xinchi Deng, et al. Promptenhancer. A simple approach to enhance text-to- image models via bfin-offttoough prompt rewriting. arXiv preprint MXXiv:55090045:5 2025b.
12

---

Published as a conference paper at ICLR 2026
Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang. Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for Ilm reasoning. arXiv preprint arXiv:2506.011 01939, 2025c.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang. Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409, 188699 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Ren-Di Wu, Yu- Yen Lin, and Huei-Fang Yang. Training-free zero-shot composed image retrieval via weighted modality fusion and similarity. In International Conference on Technologies and Applications of Artificial Intelligence, pp. 77-90. Springer, 2024.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2306.09344. 2023.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Stefano Ermon, and Bin Cui. Mastering text- to-image diffusion: Recaptioning, planning, and generating with multimodal 1lms. In Forty-first International Conference Machine Learning, 2024.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich text-to-image generation. arXiv preprint arXiv:2206.10788 2(3):5, 2022.
Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Ziyu Guo, Haoquan Zhang, Manyuan Zhang, Jiaming Liu, Peng Gao, and Hongsheng Li. Let's verify and reinforce image generation step by step. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 28662- 28672, 2025a.
Yu Zhang, Yunqi Li, Yifan Yang, Rui Wang. Yuqing Yang, Dai Qi, Jianmin Bao, Dongdong Chen, Chong Luo, and Lili Qiu. Reasongen-rl: Cot for autoregressive image generation models through sft and arXiv preprint arXiv:2505.2244 24875, 2025b.
Yucheng Zhou, Jihai Zhang, Guanjie Chen, Jianbing Shen, and Yu Cheng. Less is more: Vision representation compression for efficient video generation with large language models, 2024.
Yucheng Zhou, Jiahao Yuan, and Qianning Wang. Draw all your imagine: A holistic bench- mark and agent framework for complex instruction-based image generation. arXiv preprint arX1v:2505.24787, 2025.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
In this work, we employed GPT-5 to assist with the polishing of English text. The model was used primarily for improving readability, clarity, and fluency of the written content, ensuring that the final manuscript meets high academic standards.
B PROOFS AND DERIVATIONS FOR EG-GRPO
B. DETAILED PROOF OF PROPOSITION I (PER-BATCII BUDGET BALANCE)
Setup and notation. For sequence i, write (1)) - |40| 2 0, s(t)
sign(A {+1), and
h & ( E [0. 1]. With the low/mid/high partitions from Section 5.3
EG-GRPO's per-sequence
budget can be written
BB - Pio) (())
(1-Pio)
Σ
(15(0(()
An al1))
(11)
T(
tES(Um)
Lemma (Symmetric sign averaging).
For any 2 and 2 0,
2 a + bl
+ l-a-++ a + b1) max(a, b).
Proof. If 2 a, then la b) = + and - + bj = a, so the average is b. If < a, then la + b] + b and -a + bl
b. the average is
Taking expectation over (() (assumed approximately symmetric due to group-normalizationn) Lemma yields
Exact decomposition.
E E,00 s (1 (0) + Ah max(a!)) >h(")).
(12)
Plugging into equation
11 and using (z o)+ - max((, a),
E Eg0 B = (1 Plo) (1) + T()
Σ (an
()))
(13)
(ES(Um)
8(0)
Averaging over the batch gives
Es BG = (1 Pio) Es(a + Rss ((() (A)]
(14)
Here 8(0) (A) is nondecreasing in À and equals zero at = 0.
Target equation for exact k-scaaing. If one desires Eg (B = <E (1) then equation 14 is equivalent to
Σ TO Σ Σ (An (2n
-000 o(0) (x Plo) a
Σ
(15)
tES(i.))
iEB
The left-hand side is continuous, nondecreasing in X, hence admits (numerically) unique solution for any E (0, 1].
Closed-form calibration (upper-hound match). For an implementable closed form, use (x
a)+ with A
((0 S T
14

---

Published as a conference paper at ICLR 2026
Substituting this into equation 14 yields the upper bound
Es BB
(1) - (1 - P1o) Es(a + AE [
(16)
Matching the upper bound to x E.E..d (9) leads to the batch-calibrated choice
Eiega (1))
Enes(i.
Eies (A [S
I R. EiEB T:5
THT
K. . EieB Tho Exestim sgl [1100
T
(17)
Eles(i 1/g
which is exactly equation @ This calibration equates "saved" low-entropy budget with "reinvested" high-entropy mass in an upper-bound sense; see Remarks below.
Remarks calibration accuracy.
(i) Define the nonnegative discrepancy
(A) & AH (A)
Σ
min (Ah (1) )
T
rES (ES(i.))
(6) for most high-entropy tokens, and is statistically damped batch aver- aging. (ii) For exact scaliing one may solve equation [5 via a 1D root finder; we keep equation [ for simplicity and stability. (iii) With 1, equation 8 delivers batch-level budget neutrality in the
It vanishes (4)
calibrated upper-bound sense; empirically it closely tracks neutrality while avoiding per-step root solving.
Conclusion. Combining equation 14 equation 16 with the calibration above yields Eg (B
Ess (B as stated in Proposition
B.2 PROOF OF COROLLARY 5.1 (PINED-POINT NEUTRALITY)
Suppose A 0 for all in a batch. Then a = 0, and the numerator of equation 8 vanishes, yielding X for any 6 E (0, 1]. By equation ((()) 0 for all tokens I, the loss equation 10 reduces to the reference-polle policyy KL term. Hence any GRPO stationary point remains stationary under EG-GRPO, proving the corollary.
B.3 OPTIONAL EXACT PER-BATCH SCALING (IMPLEMENTATION NOTE)
If precise k-scaling required, solve the monotone equation equation 15 for À by bisection or Newton's 's method. This guarantees Eg ( = E exactly per batch, at the cost of a ID search.
15

---

Published as a conference paper at ICLR 2026
C INSTRUCTION FOR WISE
WISE Instruction
You asked to write concise text description to guide the generation of an image based on this prompt: "()". Provide a brief, precise visualization of all elements in the prompt. Your description should:
1. Include every object mentioned.
2. Specify visual attributes (color, number, shape, texture) if given.
3. Clarify spatial or relational positioning if specified.
4. Be concise ($50 words) but include the most common features and states inferred from real-world knowledge.
Apply real-world knowledge (cultural, religious, temporal, spatial, biological, physical, or chemical reasoning) and select only the single most relevant aspect that naturally enhances the original prompt infer context (e.g., season, appearance, identity, cultural usage, or natural state) and reflect it in the objects. Use direct, widely accepted interpretations; include cultural or religious meanings only when they are common real-world associations. Avoid abstract purely metaphorical interpretations.
Emphasize the current state of each object individually, as inferred from its environment context. Reason separately for each object, considering temporal, cultural, physical factors, and prioritize states logically implied by the prompt.
7. If multiple objects present, reason from each object's 's inherent physical chemical properties and their interactions with the environment and with all other objects. Ensure that the inferred state, behavior, and interaction of every single object is logically correct and consistent with real-world rules, and clearly describe all differences and interactions relative to each other.
8. Ensure realism and aesthetic quality: all objects and interactions must follow real-world rules and appear visually consistent and appealing.
9. Do not omit objects explicitly mentioned, or add ones not specified or logically inferred.
10. Always preserve and emphasize the original objects and scene as the primary focus.
11. Always output complete natural language description, never an image or symbolic shorthand.
D COMPUTATIONAL EFFICIENCY AND CONVERGENCE ANALYSIS
To address concerns regarding computational overhead the proposed EG-GRPO, specifically the token-level entropy computation, percentile thresholding, and batch-level bonus recalibration, we conducted a rigorous benchmarking comparison of our method against the bascline T2I-R1.
D.1 WALL-CLOCK TIME AND MEMORY OVERHEAD
We measured the training step time and peak GPU memory usage on NVIDIA A100 GPUs under identical experimental settings. As shown in Table EG-GRPO introduces negligible overhead.
Metric
T2I-R1 (Baseline)
EG-GRPO (Ours)
Overhead
Step Time (s)
50.20
50.88
+1.35%
GPU Memory (GB)
30.79
31.17
+0.38 GB
Table 5:
Computational overhead comparison between T21-R1 and EG-GRPO. The overhead intro- duced by our catropy-guides mechanism is marginal in both time and memory.
The minimal increase in wall-clock time (~ 1.35%) suggests that the complexity of entropy opera- tions (O(LVV) for sequence length L and vocabulary size V) is trivial compared to the computational cost of the model's forward and backward passes. Importantly, this overhead ratio is expected to de-
16

---

Published as a conference paper at ICLR 2026
crease further model scale increases (e.g., larger hidden dimensions), the entropy calculation depends only on the output logits and not the model depth or parameter count.
D.2 CONVERGENCE EFFICIENCY
While the per-step ciency. We compared the reward progression of both models over the same number of training steps. As detailed in Table 6 EG-GRPO consistently achieves higher rewards earlier the training process.
marginally higher, EG-GRPO exhibits significantly better sample effi-
Step
100
200
400
600
800
Reward (T21-R1)
2.1105
2.1130
2.1208
2.1339
2.1340
Reward (EG-GRPO)
2.1411
2.1836
2.1968
2.2075
2.2117
Table 6: Convergence comparison: Reward scores different training steps. EG-GRPO achieves higher performance consistently, indicating superior time-to-convergence ratio.
This superior convergence efficiency outweighs the slight computational overhead, making EG- GRPO more practical choice for large-scale training.
E QUALITATIVE DIVERSITY CASES
To qualitatively compare the diversity of T2I-RI and EG-GRPO, we generate 20 samples for each of three identical prompts and concatenate them for side-by-side visualization, as shown in Figure For fair comparison, samples from both models are selected under comparable generation quality scores.
17

---

Published as a conference paper at ICLR 2026
The soft, plush texture of the teddy bear was a comforting companion for the children at bedtime.
The triangular shelf with its crescent curves and circular brackets held items in the pentagonal hallway.
chicken hidden by couch
Figure 7: Comparison of generation diversity for T2I-R1 (left) and EG-GRPO (right).
18


Reviews:
[{"review_id": "APYmtInu5l", "reviewer": "Reviewer_Wsoe", "summary": "The paper investigates the relationship between CoT prompting and RL in autoregressive text-to-image generation models. The analysis shows that CoT broadens exploration, while RL narrows it toward high-reward regions. Both the mean and variance of image-token entropy are negatively correlated with final rewards, emphasizing stability and reduced uncertainty. The authors propose Entropy-Guided Group Relative Policy Optimization (EG-GRPO), which allocates optimization based on token-level entropy. Experiments show the proposed method balanced exploration and stability in RL fine-tuning.", "questions": "1. Have the authors benchmarked the wall-clock or memory overhead of EG-GRPO (with batch bonus calibration) compared to vanilla GRPO or diffusion-based models, especially at scale? \n2. Are there cases where the additional computation outweighs the quality gains?\nCould the authors provide a quantitative assessment of diversity (e.g., via FID/IS for diversity, or human preference rating) to determine whether entropy suppression negatively impacts creative diversity?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The authors provide a thorough analysis of the proposed method to show how CoT and RL are balanced to reduce uncertainty and preserve knowledge, which seems convincing.\n\nPaper is well organized. writing is clear and easy to follow.", "weaknesses": "1. The proposed approach adds nontrivial computational overhead due to token-level entropy computation, percentile thresholding, and per-batch bonus recalibration. However, the paper lacks a quantitative analysis of the resulting scaling, memory, and wall-clock costs, particularly for large-scale models or datasets.\n2. While entropy is treated as a measure of uncertainty, the method primarily focuses on entropy reduction, potentially at the expense of output diversity and creative expressivity. A discussion of this trade-off is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iwsAqbb4R9", "reviewer": "Reviewer_7HWk", "summary": "This paper studies the problem of autoregressive text to image generation, using chain of thought before generation. The authors devise three conclusions about the relationship between entropy and reward, before going on to devise an algorithm EG-GRPO. This algorithm nulls out what is called low entropy tokens to instead focus on high entropy tokens. The authors offer a theoretical explanation for \"reinvesting\" in high entropy tokens. The paper experimentally verifies this on T2I-CompBench and WISE. Where on most tasks it shows superior performance.", "questions": "see weaknesses for questions.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- this paper is well justified and for the most part I agree with the three points flagged to instantiate this algorithm.\n- this paper does a good job justifying the motivation for the algorithm", "weaknesses": "- I am curious about the effect of EG-GRPO on diversity of the reward/image. Indeed, I worry that the by further optimizing high entropy tokens, the diversity of the rewards and images generated is lowered. \nAs seen in fig 5, it seems that average entropy is lower, and thus the model is generating less diverge images. Some analysis and discussion would be preferred.\n- The algorithms performance seems better than the other methods, but I worry this is at the cost of sample diversity.\n- It is unclear to me why even entropy minimization in this case is the correct method? To me, this seems like reducing exploration?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PRMJxgSL79", "reviewer": "Reviewer_6gZ9", "summary": "The paper proposes an approach to balance exploration via CoT vs exploitation using RL for improving the text to image models and ultimately proposes a modified version of GRPO to combine these two insights.", "questions": "- Is Figure 6 representative across multiple prompts or is it a curated set of images? please add a disclaimer if so.\n- In Figure 2 why does T2I-R1 translate the whole distribution rather than reshaping it around the original Janus distribution, the shift seems extreme. The entropy image mean goes down, could we include an appendix showing real images so we can qualitatively compare the diversity?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The logic behind the paper makes a lot of sense, the combination of exploration vs exploitation.\n\nThe objective of the paper is clear and well communicated.", "weaknesses": "Missing references GRPO in autoregressive models: \n\n'Fine-Tuning Next-Scale Visual Autoregressive Models with Group Relative Policy Optimization' \n'Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl.'\n\n\nThe relative improvement with respect to T2I-R1 is small. \n\nExperiments could be more comprehensive:\n- Can you apply the same method to improve aesthetic score like in 'Fine-Tuning Next-Scale Visual Autoregressive Models with Group Relative Policy Optimization' ?\n- Can you try it with a more diverse set of text-to-image models? (I expected to work too, but its good for completeness even if its just on one benchmark to see how different base models improve with your technique)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)