Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
Authors: Xiaoxiao Ma
Feng Zhao
Pengyang Ling
Haibo Qiu
Zhixiang Wei
Hu Yu
Jie Huang
Zhixiong Zeng
Lin Ma
Venue: NeurIPS 2025 poster ()
Keywords: auto-regressive image generation
training-free
image generation
Primary area: deep_learning
TL;DR: improve both generation quality and efficiency of autoregressive image generation by leveraging entropy of token-wise probability distributions

Abstract: In this work, we first revisit the sampling issues in current autoregressive (AR) image generation models and identify that image tokens, unlike text tokens, exhibit lower information density and non-uniform spatial distribution. Accordingly, we present an entropy-informed decoding strategy that facilitates higher autoregressive generation quality with faster synthesis speed. Specifically, the proposed method introduces two main innovations: 1) dynamic temperature control guided by spatial entropy of token distributions, enhancing the balance between content diversity, alignment accuracy, and structural coherence in both mask-based and scale-wise models, without extra computational overhead, and 2) entropy-aware acceptance rules in speculative decoding, achieving near-lossless generation at about 85% of the inference cost of conventional acceleration methods. Extensive experiments across multiple benchmarks using diverse AR image generation models demonstrate the effectiveness and generalizability of our approach in enhancing both generation quality and sampling speed.

Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
Xiaoxiao Ma
Feng Zhao
Pengyang Ling'
Haibo Qiu
Zhixiang Wei!
Hu Yuu Jie Huang Zhixiong Zeng²
Lin Ma²
University of Science and Technology of China Meituan
Meissonic
S0p5-6005.044 SiegbeseS (2.9 Steg-egt (27 Stopse563 (2365)) Stepse (75.0). Stepue (65.94) 80 Steps (6
Figure 1: Top row: Our method generates images with finer details and better structure. Bottom row: Combined with existing acceleration methods, ours reduces inference cost by 15%. (Left two pairs are from LlamaGen [I: right from Lumina-mGPT Inference steps and latency are reported.)
Abstract
In this work, we first revisit the sampling issues in current autoregressive (AR) image generation models and identify that image tokens, unlike text tokens, exhibit lower information density and non-uniform spatial distribution. Accordingly, we present entropy-informed decoding strategy that facilitates higher autoregres- sive generation quality with faster synthesis speed. Specifically, the proposed method introduces two main innovations: 1) dynamic temperature control guided by spatial entropy token distributions, enhancing the balance between content diversity, alignment accuracy, and struetural coherence in both scale-wise models, without extra computational overhead, and 2) entropy- aware acceptance rules in speculative decoding, achieving near-lossless generation about 85% of the inference cost conventional acceleration methods. Exten- sive experiments across multiple benchmarks using diverse AR image genera- tion models demonstrate the effectiveness and generalizability of our approach in enhancing both generation quality and sampling speed. Code is available https:// /ttpppub///gtubbb...... thub. com/ krennnn ernnniion c999/ ARsample
and
Introduction
Autoregressive (AR) modeling, as the mainstream in language generation 3 5 6 has recently
demonstrate strong potential in visual generation (2) [. offering improved scalability and potential for unified vision-language modeling [. In this paradigm, images are first quantized into discrete
Project lead; Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

token sequences TO ITI which are then generated either token- by-token in a raster-scan order 8 12 or in parallel through multi- token generation strategies 73
Unlike diffusion or flow models that regress continuous tokens
AR models learn the probabilistic distribution over discrete vocahulary, and sampling strategies (e.g., top- K top-p [8]
15 16
required to specify token, which is essential and significantly impacts the quality and characteristics of generated content. Within the area of language modeling, strategies have been proposed augment
reasoning capabilities and mitigate hallucinations, including logit shaping [9 20). contrastive
decoding 21 22].
leveraging model-specific features [23], and search-based methods 24 25
which emphasize answer accuracy over generation diversity.
However, clear distinction exists between image and language: images exhibit lower information density and highly non-uniform spatial information distribution, as shown in Fig. 2 (a), making Lanuaag--rriettr methods suboptimal for image generation. This mismatch often leads to trade-
off between diversity in image contents and text consistency. As observed in [2] 26), increasing
randomness (e.g. high top-K) helps enrich visual content but compromises structural stability, leading to artifacts, distorted structures, or chaotic textures. Conversely, reducing randomness stabilizes structure and improves alignment, but often yields flat, oversmoothed details, or simplistic background. How to balance randomness and determinism during sampling is thus critical for high-quality image generation. Unfortunately, existing methods typically rely on uniform sampling approaches like fixed top-K K or top-p, overlooking the inherent spatial information imbalance, which limits their ability achieve high-quality images.
In this work, we aim to leverage the uneven distribution of information in images, and propose sampling method specifically for autoregressive image generation. We observe that entropy of predicted logits effectively reflects information density in image during generation-low entropy corresponds to large homogeneous regions, while high entropy highlights content-rich areas such as foreground objects and complex backgrounds (see Fig. B. An intuitive idea is to encourage higher randomness in low-entropy regions, while applying stricter sampling in high-entropy areas. This helps balance image richness with structural stability, and also allocating fewer inference resources to low-entropy regions, which enables further acceleration with minimal impact on generation quality. Unlike [27], which applies entropy to control sampling randomness in super-resolution to modulate stochasticity, our work leverages entropy to guide autoregressive generation dynamics.
Building on this observation, we propose an entropy-aware sampling strategy that adjusts token distributions dynamically during inference. By computing the entropy of each predicted token distribution, we assign adaptive temperatures-injeciinn more randomness in low-entropy (simple) regions and applying stricter sampling in high-entropy (complex) areas. This improves the balance between image quality, structural stability, and text-image alignment without additional training or inference cost. Moreover, our method generalizes well to a variety of autoregressive frameworks based on discrete token prediction, including mask-based and scale-wise generation. We also extend the entropy-awe aware idea to acceleration: by incorporating entropy-depende decoding, we reduce inference cost to 85% of standard baselines with minimal quality loss. We summarize our contributions follows:
acceptance in speculative
1. Motivated by the observation that image information sparse and unevenly distributed, which can be reflected by the entropy of tokens, we introduce entropy-driven sampling strategy tailored for AR image generation that dynamically adapts sampling behavior based on entropy.
2. In contrast to conventional sampling methods like top-K or top-p, our approach enhances image quality and structural stability without modifying the model or increasing inference cost, and benefits multiple types of AR generation frameworks.
3. We further extend the entropy-aware perspective to speculative decoding, achieving a 15% reduction in inference time while maintaining visual fidelity across multiple benchmarks.
2 Related works
2.1 Autoregressive image generation
Early work (88 generates images directly at pixel level. Later approaches adopt a two-stage pipeline: images first quantized into discrete tokens [10. [1) then generated with Transformers in raster

---

Top-
Top-2
90-
mmage
(8g
3345
0 ko06584 Ch612
18 krd6584, 699-7.5
Temo.
15-
5-
0.00
0.05
0.10
0.15
0.20
Aug. Spectral Sogments
(3) Kel, 6F6c2
(3) Iel, CP6.7.5
CLP
(a) Difference between Text & Image
(b) Qualitative Comparison Actoos Settings
(c) Quancitatvve Evaluation Accoss Settings
Figure 2: (a) Comparison of information density between image and text. Histogram of average frequency-domait embeddings from LlamaGen [1] (image) and Qwen2 [6] (text) show the uneven spatial distribution in images with large amount of low-frequency components. (b) Qualitative results under various configurations. High CFG (Classifier- fidelity, while lower CFG with higher top- K improves fidelity but may reduce text-image consistency. (c) Quantitative evaluation of LlamaGen under different sampling settings.
Guidance) or low top- often harms
Recent efforts scale this paradigm with larger models and stronger conditioning. LeamaGen provides class and text-conditioned baselines; Lumina-mGPT 22 and Anole 35 fine-tune Chameleon [12 for improved text-conditioned generation. Unified frameworks further bridge understanding and generation
order 129
30
32
33
34
9 36 37 in a single Transformer. Mcanwhile, image tokenizers have evolved for better reconstruction 38
39
40
multimodal integration [42 43)
While proven effective, the vanilla autoregressive paradigm suffers from slow and rigid next-token prediction. To improve efficiency, recent studies explore more strategies, including multi-token
prediction via random masking [14 44 45 46), coarse-to-fine modeling ? 13
47
48 49 or
hybrid approaches [5). Nonetheless, pector-quantized models still rely on sampling from token distributions, making generation quality sensitive to the sampling strategy.
2.2 Sampling strategies in autoregressive models
Transformers model the probability distribution over tokens, requiring specific sampling strategies to obtain concrete outputs. Common approaches in language modeling include top-k [52) and top-
p [8] sampling, which truncate the candidate space by rank or cumulative probability. EDT 20
dynamically adjusts temperature based on entropy to balance diversity and precision. Other ap- proaches explore repetition penalties
53
contrastive decoding [22], speculative decoding (4
and search-based techniques [56 25
57 58
to reduce hallucination or speed up inference.
In visual generation, a higher degree of randomness often needed produce more realistic and detailed content. LlamaGen [ and Lumina-mGPT [2] demonstrate that much larger top-k values than those used in language models help avoid over-smoothed and low-detail outputs. Recent methods 226 59 apply speculative [54] or parallel decoding [60] 61 to accelerate image synthesis. PURE (27 designs a top-k strategy based on token entropy and detail levels improve autoregressive super-resolution. However, they overlook the highly uneven spatial information distribution in images during generation, and do not tailor decoding for autoregressive image generation.
Methods
In this section, we first introduce basic of autoregressive image generation in Sec. 3.1 Starting from
the difference between image and text generation, we present our method from entropy- perspective by adjusting token-level randomness during generation (Sec. 3.2 Sec. 3.3 and further extend this view acceleration (Sec. 3.4.

---

BD
233-
Entropy
()) Visualization of Generated Image & Entropy
(b) Froquency Histogram Entropy
(6) Temp. Sensitivity by Entropy Level
Figure (a) Entropy map during generation: complex regions exhibit higher entropy (more dispersed probabilities), while simpler areas show lower entropy. (b) Histogram of entropy distribution on COCO val2017 (from LlamaGen Stage II). (c) Varying temperature by entropy range affects FID and CLIP score: lower-entropy tokens benefit from higher temperatures, and vice versa.
3.1 Preliminaries and motivation
Autoregressive image generation. In typical autoregressive generation process, an image I R xW X3 is quantized into set of discrete tokens (21. Thxw), where each token x; [V], V denotes the size of the VQ-VAE codebook. The image tokens are generated sequentially by a transformer, with the i + 1-th token Ii+1 conditioned on the previously generated tokens. This bw process is modeled as II i=1 p(#,+1 I1::), where I1:; (#1 22 ...., #f,, Iz), and p(r (1::) represents
a categorical distribution over token at position i + 1. In text-conditioned generation tasks, the full image sequence is generated conditioned on prefix of text tokens. At each step, a sampling method such as top-K or top-p is applied to select a token from p(x x1s)). ). And choice of sampling strategy significantly affect the quality of generated image, illustrated in Fig. (b).
Difference between image & text generation. Compared to text, the information density in images is lower and highly non-uniform. For example, images often contain large regions of solid color or visually similar content, while nearby tokens in text are typically distinct. To illustrate the difference, segment the embedding sequences of both images and texts into equal lengths, compute the average frequency spectrum per segment, and visualize the distributions in Fig. 2 (a). The average frequency spectrum distribution of image segments is more dispersed, with large amount of low low-frequency areas; whereas textual segments demonstrate more compact and uniform distributions.
This discrepancy poses challenge for sampling: fixed parameters like top-K or top-p, though effective in language generation, perform suboptimally when simply applying them across all image tokens. They fail to account for spatial variability. resulting in regional artifacts: overly deterministic sampling may lose fine details, producing flat regions, excessive smoothness, or simplistic backgrounds; while excessively random sampling compromises semantic consistency and structural coherence, causing artifacts, distorted limbs, or chaotic textures.
Relationship between entropy & image contents. We demonstrate that the entropy of predicted token distribution serves as an effective indicator of local information density in an image. Specifically, we compute the entropy e of log- ligeelilooood over all V codebook entries at each generation step as:
Σ ph log(pk).
(I)
As shown in Fig. 3 regions with simple content (e.g., solid colors) typically exhibit lower entropy, while more complex foreground (e.g., objects, structures, and textures) areas have higher entropy. Low-entropy regions correspond to peaked distributions over a few tokens, indicating high model confidence. Conversely, high-entropy regions display more uniform distribution, reflecting greater uncertainty in token selection and higher information density. These observations validate entropy as a reliable proxy for measuring information density in images.
3.2 Eatropy-aware dynamic temperature
Building on the observation in Sec. 3.1 we further investigate how regions with different entropy levels affect image quality and the optimal sampling strategy. Under simple experimental setup, we adopt [1] to analyze the entropy distribution of logits during generation, discretize it into intervals,

---

Step
Step
Step
Step
Step
Step
Step
Step
Figure 4:
During generation of mask-based model 44,, large number of early steps (0~550) are allocated to compute tokens in simple regions, while only few later steps (e.g., 50~633 for generating complex content. This often leads to degraded quality in the main visual subjects.
and adjust token temperature within each interval to control sampling randomness. We then examine the relationship between image quality and text alignment via FID and CLIP-Score as indicators, results can be seen in Fig. 3 (c). Our findings are as follows:
1. Across most entropy intervals, adjusting randomness leads to a trade-off between image quality and text alignment, especially for tokens in the entropy range of [2,8].
2. In high-entropy regions (>5), lower randomness helps improve text-image consistency.
3. In regions with extreme low entropy (<2), increasing sampling randomness consistently improves visual quality, while having negligible impact on text alignment.
These findings suggest that token-level sampling should entropy-aware: during inference, tokens with lower entropy should be assigned relatively higher randomress to enhance the quality and visual richness generated image, while high-entropy tokens should be sampled more cautiously to preserve clear structure, details and text alignment.
To better adapt real-world inference, we introduce a dynamic temperature mechanism that adjusts sampling randomness on per-token basis. Specifically, after computing the entropy of predicted distribution at each position, we determine a temperature value with predefined mapping function:
T = Toe & +
where denotes the entropy at current token position, To represents the maximum temperature, 0 sets the lower bound, and controls the decay rate temperature with increasing entropy. See Sec. 4.4 for further analysis and discussion. Subsequently, the resulting temperature T is then applied to rescale the predicted logits as follows:
P
Then, by applying softmar(f;). the differences between logits of different tokens are amplified (when 7*11) or reduced (when T>1), which makes the probability distribution more concentrated or spread out, achieving the dynamic adjustment of sampling based on the region's 's content distribution.
3.3 Adaptation more AR models
Additionally, many recent methods deviate from strict next-token prediction and instead adopt paradigms such as mask-prediction or scale-wise generation. We show that the proposed entropy- based strategy remains effective in these settings. After obtaining multiple token-level logit distribu- tions from the transformer, we directly apply Eq. (2) to them. As shown in Table [ this approach consistently improves performance across standard evaluation metrics. Moreover, for different paradigms, specific designs can be incorporated to further enhance performance:
Mask preedccteiition models. For mask-based models such as 44 45), a full probability distribution over all image tokens is obtained at each forward step. After sampling, a dynamic masking mechanism based on token confidence is applied. We observe that this masking strategy also has a significant impact on the quality of the generated results (see Fig. 4 Specifically, a soft categorical distribution is used to select i tokens from all candidates to be accepted at the current timestep t:
conf = log g.
(4)
5

---

Baseline
bride cutting wedking
A pink cupcake liner
Gold hexagonal glasses book.
dack
orange video game.
Plush
Figure 5: Visual comparison on next-token model. Examples from Lumina-mGPT, proposed method ("Ours") maintains richer content while offering more accurate structure and finer details.
Table 1: Performance sampling strategies on various models. "Baseline" original sampling:
"+Prob./Ours" refers to Sec. "+Massing/4SSall-wii""" are paradigm-specific from Sec 3.3
Method
Config.
FID.
CLIP-Scoret
DPG
HPSv2.11
SDv2.1 15
22.87
26.31
68.09
26.38
PixArt-a 62
33.23
25.70
71.52
30.04
SDXL [63
23.20
26.46
74.21
28.54
SDy3-medium
29.82
26.24
85.85
30.22
Baseline
21.94
25.95
43.51
21.24
LlamaGen
Ours
20.36
25.96
48.63
21.39
Baseline
29.15
26.04
79.68
28.92
Lumina-mGPP
Ours
27.44
26.25
79.77
28.87
Baseline
53.61
25.27
63.83
29.33
Meissonic 14
+Proh.
48.37
25.49
66.19
29.94
+Masking.
48.43
25.54
67.08
30.04
Baseline
35.05
25.43
70.25
28.79
STAR 13
+Prob.
32.75
25.56
70.83
28.93
Scale-wise
32.37
25.61
70.86
29.06
where p is the predicted token probability, T is the dynamic temperature defined in Eq. Gumbel(0, 1) is sampled from standard Gumbel distribution.
2 g
M₁ conf TopK(conf @ M,-1,kk.
(5)
Here, TopK(conf,, k) returns the k-th highest confidence and tokens with lower confidence
masked out. M, and M,-1 represent the accepted token masks at the current and previous timesteps, respectively, while M denotes the clement-wiss negation of MM The number of accepted tokens each timestep is determined by predefined scheduler. This design further encourages randomness in low-entropy regions while enhancing accuracy in high-entropy regions, leading to improved image quality.
generate tokens within each scale simultaneously. find that assigning greater randomness to earlier scales while reducing randomness later scales yields more accurate results without compromising image richness. Specifically, we define a temperature term that decreases the scale increases. For the tokens s-th scale, T. calculated as:
Scale-wise models 7
T, T- 3 (s [S/21)]
(6a)
Ps T.
(6b)
where s denotes the scale index and E 21, S): P. is the logits of tokens at s-th scale, with shape of h, x w, x V. T dynamic temperature defined in Eq. ( controls the decay rate of T, across scales and is set to 0.3 in experiments.

---

+Masking
+Makkngg
histaric behind.
man wearing cowfoy gear sitting on horse.
dog sitting next to mirror- like pail.
Baseline
Prob.
Scale-wilse
Baseline
Prob.
Scale-wise
Two scooters riding down the raadway.
Oii painting of black woman white dress amid stardust and roses.
picture
Figure 6: Visual comparison on mask-base bassd (top) and scale-based model (bottom) from Meissonic
and STAR. Proposed method provides better visual quality in structure and detail.
3.4 Autoregressive acceleration
We further explore the use of entropy to accelerate autoregressive generation. Existing speculative
decoding approaches 26 [5) typically generate multiple candidate tokens via draft model, followed by verification step using a target model. When draft and target share the same model, the process reduces to comparing the confidence scores from two consecutive iterations. Specifically, the
probability at the (j-1)-th step, p(æ " and the j-th step, p(æ = =()r1)) acceptance probability of the token x; is then computed based on these two distributions:
) are compared. The
min (1,
() (x x 18-11
In practice, a token accepted p > т, where is drawn from uniform distribution 2[[0, 1]. which naturally balances randomness required for sampling diversity and accuracy during generation.
To make the process entropy-aware aware, we propose simple modification to the acceptance rule. Since low-entropy regions are more predictable and allow higher randomness, while high-entropy regions require stricter verification, we scale the threshold by an entropy-based factor e/e, where constant. This dynamic adjustment enables more efficient generation based on local uncertainty.
To improve stability, rewrite 0.5 + (r 0.5), treating
0.5 44(-0.5 0.5), where the noise term controls acceptance randomness. We scale this term with decaying factor (1 - A e), where the entropy, making high-entropy tokens more deterministically verified, while low-entropy areas retain near-uniform. Combining both strategies above, the final acceptance rule can be formulated as:
0.5 + - 0.5) - À cc],
(8)
where is entropy at the current token, and À are constants set to and 16, respectively.
7

---

083 (27-300)
Stepp: 01.)
BSB (26, Mea
1909 jan. 2010
1887 (66.20s)
1800 abal
SID
380 (25.22))
[22.50
Soops: 553 (22.950)
1000 (26,500)
Steps: 1401 (88.38s)
Shape: 1345 17a (222
Ours
around field.
soft Brown
side by quili.
midie aged Nack table full bumanan..
holding head.
FAIE bockground
Figure 7: Results on acceleration. We report the inference steps ("Steps") and latency. Our method achieves similar image quality while using only 85% of the baseline's 's ("SJD") inference cost.
Table 2: Quantitative comparison of acceleration on COCO17-val. "Vanilla" refers no acceleration.
Model
Config.
Avg. Latency [s]]
Avg. Steps]
FID
CLIP "Porr"
Vanilla
44.52
1024
53.42
21.47
LlamaGen
SJD [26
26.52
626.9
54.49
21.45
Ours
22.04
535.5
54.60
21.51
Vanilla
169.72
4165
29.15
26.04
Lumina-mGPT
26
84.97
1854.5
30.76
26.09
Ours
72.35
1594.5
30.89
26.10
This dynamic acceptance criterion allocates inference budget more efficiently--eeee more permissive in confident regions and stricter in ambiguous ones, thereby reducing inference time with minimal performance loss. For detailed metrics and comparisons, please refer to Sec. 4.3 and Sec. 4.4
4
Experiments
4.1
Implementation details
Four representative models are selected for comparison: vanilla AR model LlamaGen ① and Lumina- mGPT [2] based on next-token prediction, mask-based model Meissonic [44], and scale-wise model
STAR 13 All models are evaluated under their original inference settings (e.g., CFG=4 and top- K=2000 for Lumina-mGPT, CFG=7.5 for LlamaGen). We use LlamaGen's official Stage-1 model to evaluate the sampling strategy, while Stage-2 used only for acceleration analysis due to its poor performance (FID 53.42, -Loor 21.47), which makes quality differences hard to observe.
evaluate the image quality and prompt-following capability. Moreover, DPG-bench (55 and HPS [66] are adopted to assess the semantic fidelity and perceptual quality of the generated images. All experiments are conducted on A100 GPUs.
FID and CLIP-Score are tested on the MS-COCO 2017 64 validation
4.2 Sampling quality
As shown in Table I and Fig. 6 our dynamic temperature sampling strategy effectively adapts to regions with varying information density in the image, leading to more stable structures and clearer details in the generated outputs. Depending on the inherent sampling mechanism of each model, our method yields varying degrees of improvement across different approaches. In particular, achieves an approximate 4-point gain on DPG for both Meissonic and LlamaGen, along with notable enhancement in visual quality. addition, integrating our approach with the masking- and scale-wise strategies described in Sec. 3.3 can further enhance generation performance. See Table for results with "+Prob" (applying dynamic temperature to logits only), and "+Masking / +Scale-wise" (applying temperature based mask scale).

---

3/
Top-
Top.k
CRG
CPG
1a) Ablition of Parametors an Eq.
(b) Combination Proposed Method with top-c and CPG
Figure 8: (a) Impact of parameters in Sec. 3.2, (b) Combination of our sampling strategy with existing methods (Top-K, K, CFG), Our method reduces the sensitivity of FID and CLIP-Score to these parameters, enhancing image quality and text alignment. Experiments are conducted LlamaGen.
Table 3: Quantitative comparison on acceleration.
"Avg. Lat." " is short for "Avg. Lateney." Experi- ments are conducted Lumina-mGPT.
Avg.
Avg. NFE,
CLIP Score
Config.
FID
Lat.
SJD
Scale
+random
SJD 226
84.97
1854.5
30.76
26.09
Figure Visual comparison for Table "Scale" slightly degrades quality while accelerating, which can be mitigated by +random".
scale
69.38
1523.7
33.86
26.05
+random
72.35
1594.5
30.89
26.12
4.3 Inference acceleration
By integrating with existing vision-based speculative decoding schemes and leveraging entropy to automatically control the acceptance condition, our method saves about 15% inference cost with almost no loss image generation quality compared to the approach in [26], as shown in Table aa and Fig. 7 The entropy-based approach significantly reduces the number of inference steps and latency, while still maintaining comparable image quality the original speculative decoding method.
4.4 Ablation study and discussion
Parameters in Eq.2. The impact of parameters in entropy-aware dynamic temperature is provided in Fig.Bla M()). It is observed that smaller or 0 values lead to higher FID and lower SLore,coree primarily due decreased randomness and overly deterministic sampling. Meanwhile, FID shows trend of initially decreasing and then increasing as increases. This is because governs the proportion of different temperatures. When a is too small, most tokens are assigned very low temperatures, causing the FID to increase. Conversely, when is too large, the image content becomes overly chaotic, resulting in increased FID. CLIP-Score consistently decreases as a increases.
Acceptance rate in Sec. 3.4 We propose to dynamically control the acceptance rate in existing spec- ulative decoding methods based on the entropy of predicted distributions. By adjusting both the scale and randomness of the threshold r, we reduce latency while maintaining quality. As shown in Table ("scale"") reduces inference cost but degrades performance,
and Fig. 9 controlling only scale especially imge quality. In contrast, jointly tuning both scale and randomness ("+randomm) achieves a better orf enabling high-qualit, generation with minimal inference overhead.
Compatibility with different AR models. Our sampling method brings notable performance gains for some models-for instance, DPG in LlamaGen and Meissonic outperforms baseline by over points. In contrast, well-trained models like Lumina-mGPT benefit only marginally. This discrepancy stems from factors such generation paradigm (e.g.. inherent sampling limitations of mask-based methods discussed in Fig. 4). training datasets and iterations (e.g., whether has been thoroughly trained on large-scale data). Nevertheless, these models can still exploit entropy for further acceleration.
Combination with top-K and CFG. We further analyze the performance of our method when combined with top-K K sampling and CFG, shown in Fig. b); results with top-p and temperature are the supplementary. By incorporating proposed method, FID metric becomes less sensitive to sampling parameters, enabling better fidelity while maintaining image-text alignment.
Factors affecting entropy. Unlike text generation with fixed tokenization rules, autoregressive image generation relies on pretrained tokenizers, and the underlying model differences-inccuun including

---

Table 4: Mean and variance ("Var.)) ") of entropy from T on COCOval1
Reso.
CFG
Mean
Var.
256
7.5
4.76
2.75
256
4.0
5.90
2.42
256
2.0
6.87
1.97
Figure 10: For some cases, the semantic-correspondinng fore- ground contents may have smaller entropy.
512
7.5
4.42
3.09
parameter scale, data quality, and training corpus corpus-lead varying entropy distributions and optimal sampling parameters. Empirically, higher CFG and larger resolutions lead to lower average entropy (see Table 4). Further analyses provided in supplementary material.
Discussion of failure cases. Although our entropy- based dynamic sampling strategy brings notable performance improvements, we also observe several failure cases where the relationship between semantic information and the entropy map becomes less consistent (see Fig. TOp. In some cases, regions such as human faces exhibit unexpectedly high entropy, while complex backgrounds receive lower entropy values. Consequently, adjusting temperature based on such entropy patterns may lead to structural distortions and overly smooth details. This ambiguity may potentially limit further performance gains, especially models that have been carefully optimized.
Conclusion
In this work, first point out the need for different sampling strategies in autoregressive image and text generation, given their distinct information distributions. Starting from this perspective, we find entropy effectively represents image information density, offering new possibilities for improving and accelerating image generation. As our method involve parameter adjustments without training, this approach could be further integrated into training fine-tuning frameworks, potentially accelerating training, boosting inference speed, improving stability, and reducing hyperparameter dependence.
6
Acknowledgments
This work was supported by the Anhui Provincial Natural Science Foundation under Grant 2108085UD12. We acknowledge the support of GPU cluster built by MCC Lab of Information Science and Technology Institution, USTC. The AI-driven experiments, simulations and model training were performed on the robotic Al-Scientist platform of Chinese Academy of Sciences.
10

---

References
[1] Peize Sun. Yi Jiang Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation. arXiv preprini urXiv:2406.06523 2024.
[2] Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining. arXiv preprint arXi::2408. 02657, 2024.
[3] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 19711, 2023.
[4] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arxin:3303.08774 2023.
[5] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine tune chat models. arXiv preprint urXiv22307.09288 2023.
[6] Qwen Team et al. Qwen2 technical report. arXiv preprint 2xXi:2407.1067111 2(8). 2024.
[7] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in Neural Information Processing Systems, 37:84839-84885,, 2024.
[8] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma. Xingchao Liu, Zizheng Pan. Wen Liu, Zhenda Xie. Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 12966- 12977, 2025.
[9] Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arxvv:2001. 17811,1111111 2025. 2025.
[10] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11873-12883 2021.
[11] Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quaanized image modeling with improved vegan. arXiv preprint arXiv:21 10.04667, 2021.
[12] Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv:2405.098/1 2024.
arXiv preprint
[13] Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: scale-wise text-to-image generation via -uto-regressive representations. arXiv preprint ar Xiv:224066 10797, 2024.
[14] Huiwen Chang, Han Zhang, Lu Jiang, Cc Liu, and William Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 1 11315-11322 13325, 2022.
[15] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent difffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pottern Recognition, pages 1684-10695, 2022.
[16] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[17] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.0343 03300003206, 2024.
[18] Ari Holtzman. Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration, arXiv preprint 190.::199400971...

---

[19] Chenxia Tang, Jianchun Liu, Hongli Xu, and Liusheng Huang. Top-no:: Not all logits are you need. arXiv preprini 1.X1:12411.07611, 2024.
[20] Shimao Zhang, Yu Bao, and Shujian Huang. Edt:
Improving large language models' generation by entropy-based dynamic temperature sampling. arXiv preprint arXiv:2403. 14541, 2024.
[21] Yixuan Su, Tian Lan, Yan Wang. Dani Yogatama, Lingpeng Kong, and Nigel Collier. contrastive framework for neural text generation. Advances in Neural Information Processing Systems, 35:21548- 21561, 2022.
[22] Yung-Sunt Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03880 2023.
[23] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang. Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and reroosection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 13418-13427, 2024.
[24] Ziqin Luo, Haixia Han, Haokun Zhao, Guochao Jiang, Chengyu Du, Tingyun Li, Jiaqing Liang, Deqing Yang, and Yanghua Xiao. Sed: Self-evaluation decoding enhances large language models for better generation. arXiv preprint arXiv:2405. 16552, 2024.
[25] Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang. Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rstar-math: Small lims can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501. 00551,,0...9........1 2025.
[26] Yao Teng, Han Shi, Xian Liu, Xuefei Ning. Guohao Dai, Yu Wang, Zhenguo Li, and Xihui Liu. Accelerating reto-segressvve text-to- -magg generation with training-free speculative jacobi decoding. In The Thirteenth International Conference on Learning Representations.
[27] Hongyang Wei, Shuaizheng Liu, Chun Yuan, and Lei Zhang. Perceive, understand and restore: Real-worke image super-resolution with autoregressive multimodal generative models. arXiv preprint arXiv:2503. 2025.
[28] Aaron Van den Oord, Nal Kalchhrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelenn decoders. Advances in Neural Information Processing Systems, 29, 2016.
[29] Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in Neural Information Processing Systems, 34: 19835, 2021.
[30] Yuying Ge, Yixiao Ge, Ziyun Zeng, Xintao Wang, and Ying Shan. Planting a seed of vision in large language model. arXiv preprint 230,::2200000041,, 2023.
[31] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, Pmlr, 2021.
pages 8821-8831
[32] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang. Vijay Vasudevan, Alexander Ku, Yinfci Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206. 10789, 2(3):5, 2022.
[33] Wanggui He, Siming Fu, Mushui Liu, Xierui Wang, Wenyi Xiao, Fangxun Shu, Yi Wang. Lei Zhang, Zhelun Yu. Haoyuan Li, et al. Mars: Mixture of auto-regressive models for fine fine-grained text-to-image synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 17123-17131, 2025.
[34] Xinlong Wang. Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, al. Emu3: Next-token prediction all you need. arXiv preprint arXiv:2409. 18869, 2024.
[35] Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large multimodal models for intericaved image-text generation. arXiv preprint arXiv:2407.06133 2024.
[36] Yang Jiao, Haibo Qiu, Zequn Jie, Shaoxiang Chen, Jingjing Chen, Lin Ma, and Yu- Yu-Gang Jiang. Unitoken: Harmonizing multimodal understanding and generation through unified visual encoding. In Proceedings of the Computer Vision and Pottern Recognition Conference, pages 3600-3610, 2025.
12

---

[37] Chuofan Ma. Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A unified tokenizer for visual generation and understanding. arXiv preprint arXiv:2502.20321, 2025.
[38] Qihang Yu, Mark Weber, Xueqing Deng. Xiaohui Shen, Daniel Cremers, and Liang Chich Chen. An image worth 32 tokens for reconstruction and generation. arXiv preprint arXiv:2406.0724 075.50, 2024.
[39] Doyup Lec, Chihcon Kim, Sachoon Kim, Minsu Cho, and Wook- Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11523-11532, 2022.
[40] Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats tiffusion-tukenizzr key to visual generation. arXiv preprint arXiv:23 2023.
[41] Yue Zhao, Yuanjun Xiong, and Philipp Krähenbühl. Image and video tokenization with binary spherical quantization. arXiv preprint urXiv:2406.07548 2024.
[42] Guiwei Zhang, Tianyu Zhang. Mohan Zhou, Yalong Bai, and Biye Li. V2flow: Unifying visual tok- enization and large language model vocabularies for autoregressive image generation. arXiv preprint arXiv:2503.07499) 2025.
[43] Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang. Yiming Gao, Hu Yc, Daniel Du, Zehuan Yuan, and Xinglong Wu. Tokenflow; Unified image tokenizer for multimodal understanding and generation. arXiv preprint arXi::2412.0306) 2024.
[44] Jinbin Bai, Tian Ye, Wei Chow, Eaxin Song, Qing-Guo Chen, Xiangtai Li. Zhen Dong. Lei Zhu, and Shuicheng Yan. Meissonic: Revitalizing masked generative transformers for efficient high-resolution text-to-image synthesis. In The Thirteenth International Conference on Learning Representations, 2024,
[45] Jinheng Xie, Weijia Mao. Zechen Bai, David Junhao Zhang, Weihao Wang. Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
[46] Hu Yu, Biao Gong, Hangjie Yuan, DanDan Zheng, Weilong Chai, Jingdong Chen, Kecheng Zheng, and Feng Zhao. Videomar: Autoregressive video generatio with continuous tokens. arXiv preprint arXiv:2506. 14168, 2025.
[47] Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid autoregressive transformer. arXiv preprint or Xiv:24100 10112, 2024.
[48] Jian Han, Jinlai Liu. Yi Jiang, Bin Yan, Yugi Zhang. Zehuan Yuan. Bingyue Peng. and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-rcsolution image synthesis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 15733-15744, 2025.
[49] Hu Yu, Hao Luo, Hangjie Yuan, Yu Rong, and Feng Zhao. Frequency autoregressive image generation with continuous tokens. arXiv preprint arXiv:2503.05300 2025.
[50] Yefei He, Yuanyu He, Shaoxuan He, Feng Chen, Hong Zhou, Kaipeng Zhang, and Bohan Zhuang. Neighboring autoregressive modeling for efficient visual generation. arXiv preprint 250i::2503.16000 2025.
[51] Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- visual generation. arXiv preprint 14Xx::4411.0077// 2024.
Liang-Chieh Chich Chen. Randomized autoregressive
[52] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8)9, 2019.
[53] Nitish Shirish Keskar, Bryan McCann, Lav R Varshncy, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1999.05858, 2019.
[54] Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274-19286. PMLR, 2023.
[55] Charlie Chen, Sebastian Borgeaud, Geoffrey Irving. Jean- Caan-Bapiiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. arXiv preprint 200.03330,.03118 2023.
13

---

[56] Clara Meister, Tim Vieira, and Ryan Cotterell. If bearn search is the answer, what was the question? arXiv preprini arXiv:2010.02650, 2020.
[57] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan I Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations. 2023.
[58] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint WiXi:2408.003//1 2024.
[59] Doohyuk Jang. Sihwan Park, June Yong Yang. Yeonsung Jung. Jihun Yun, Souvik Kundu, Sung-Yub Yub Kim. and Eunho Yang. Lantern: Accelerating visual autoregressive models with relaxed speculative decoding. In The Thirteenth International Conference on Learning Representations.
[60] Yefei He, Feng Chen, Yuanyu He, Shaoxuan He, Hong Zhou, Kaipeng Zhang. and Bohan Zhuang. Zipar: Accelerating autoregressive image generation through spatial locality. arXiv preprint arXiv:2412.04062 2024.
[61] Yuqing Wang, Shuhuai Ren, Zhijie Lin, Yujin Han, Haoyuan Guo, Zhenheng Yang, Difan Zou, Jiashi Feng, and Xihui Liu. Parallelized autoregressive visual generation. arXiv preprint arXiv:2412.
2024.
[62] Junsong Chen, Jincheng Yu. Chongjian Ge. Lewei Yao, Eaze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo. Huchuan Lu, et al. Pixart-o: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023.
[63] Dustin Podell, Zion English. Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high resouutsntttion image synthesis. In The Twelfth International Conference on Learning Representations, 2023.
[64] Tsung- Yi Lin, Michael Maire, Serge Belongic, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
[65] Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Ilm for enhanced semantic alignment. arXiv preprint 2:3 2024.
[66] Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid henchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint 2306 09341, 2023.
[67] Yufci Wang, Langing Guo, Zhihao Li, Jiaxing Huang, Pichao Wang, Bihan Wen, and Jian Wang. Training- free text-guided image editing with visual autoregressive model. arXiv preprint arXiv:2503 23897, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contri- butions made in the paper and important assumptions and limitations. A No or NA answer this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long as clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). ) The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images taken low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
For each theoretical result, does the paper provide the full set assumptions and a complete (and correct) proof?
Question:
Answer: [NA]
15

---

Justification: NA
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not include experiments. the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might may be
suffice, or if the contribution necessary to either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that appropriate the research performed.
specific model and empirical evaluation,
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reprodueibility, which may depend on the nature of the contribution. For example
1. If the contribution primarily reproduce that algorithm.
algorithm, the paper should make it clear how
2. If the contribution primarily a new model architecture, the paper should describe the architecture clearly and fully.
3. If the contribution is new model (e.g., large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
4. We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may that access to the model is limited some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
16

---

Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https
onPolccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyperparame- ters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: NA
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main
claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar the standard deviation or the standard error the mean.
17

---

It is OK to report 1- 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have a 96% CI, if the hypothesis Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are of range (e.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Answer:
[Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics hatps:///oorrrsscccooool///hlllt
Question:
Answer:
[Yes]
Justification: NA
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), faimeess considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
18

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com/ hommdatasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the paper does not release assets.
19

---

Researchers should communicate the details of the dataset/code/model part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements your country institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage LLMs if an important, original, non- standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https://neur)) http:/iipppipceooooooenen/////// 20LMLLM)2 for what should or should not be described.
20

---

A Introduction
We first provide additional details of method, including parameter settings and further descriptions, in Sec. B Then, we present extended experimental results in Sec. C In Sec. [ we conduct deeper analyses entropy in relation model behavior and image content, along with more visualizations of entropy maps. Sec. didcuusse potential limitations and future directions. Lastly, we include more visual comparisons of the proposed method in Sec.
B Additional details of our method
B.1 Hyperparameter settings in Sec. 3.2
In Sec. 3.2, we propose to dynamically control the sampling temperature based on entropy. However, due to significant differences between base models, is difficult to apply the same parameters across all settings. Therefore, we list the detailed parameters for each model in Table For undertrained models such as LlamaGen stage higher randomness is required low- low-entropy stages to avoid generating large areas of repetitive tokens. In contrast, well-trained models benefit from a smoother temperature schedule.
Table 5: Hyperparameter settings of different models.
To
0
LlamaGen
2.5
3,0
0.6
Lumina-mGPT
2.0
2.5
0.6
Meissonic
2.5
3.0
0.7
STAR
2.5
3.0
0.5
B.2 Detailed description of speculative decoding in images
We accelerate inference based existing speculative decoding schemes (26 in Sec. 3.4, thereby further reducing inference cost without sacrificing output quality. Due to space constraints, we did elaborate the baseline speculative decoding methods in the main text. Here, provide more details.
This method aims to accelerate auto regrereereesive cexteto-image generation by allowing multiple tokens to be generated in parallel without training. Inspired by speculative decoding, SJD introduces probabilistic acceptance criterion that compares the confidence of draft tokens from consecutive iterations. In each iteration j, given draft token ,,("), SJD computes acceptance probability based on the ratio between two conditional probabilities:
Po(a) 1=1111
< min
(9)
(x (1)
x1--1)
where r U(001] 1]. Accepted tokens are fixed, while the others are resampled from a calibrated distribution:
(J+1) max(0, (1(x) ( pf 2b-11)
(10)
Σn ma
This allows high-randomness sampling, crucial for image diversity, while significantly reducing decoding steps. SJD operates in windowed, iterative manner and supports optional spatially- informed token initialization to further improve efficiency.
Additional experimental results
C.1 Generation performance on an additional dataset
Since the COCO2017 dataset used in the main experiments contains only 5,000 images, it may lead to slight estimation bias in the FID computation, as FID becomes more reliable with larger sample sizes. To assess the potential misjudgment of model performance caused by limited image numbers, further evaluated the metrics on larger dataset, COC02014, shown in Table

---

Table 6: Evaluation on the larger COCO2014 dataset (compared COC02017 in the main text). The results demonstrate that the improvements brought by our method remain consistent and significant across datasets.
Model
Method
FID
CLIP-Scoref
Baseline
13.37
0.2561
LlamaGen
Ours
11.59
0.2560
Baseline
44.54
0.2567
Mcissonic
44
Ours+Mask
38,95
0.2590
Baseline
24.86
0.2581
STAR
Ours+Scale
22.09
0.2598
Baseline
18.23
0.2641
Lumina-mGPT
Ours
16.16
0.2659
C.2 Effect of random seeds on model performance
Due to the autoregressive nature of our model, each token is sampled from a probability distribution, making the generated images sensitive to random seeds. Specifically, we observed that metries such as FIDD, CIIP-Score, DPG, and HPS may vary with different random seeds. To further analyze this effect, we randomly selected 10 seeds from the range [0, 1e6), ran the generation model 10 times under these conditions, and computed the mean and standard deviation of the results. As shown in the table below, random seeds have little impact on the performance gain introduced by our method, further confirming the robustness and effectiveness of our approach. See Table 7
Table 7: Mean and standard deviation over 10 random seeds. Our method consistently outperforms the baseline with statistically significant improvements.
LlamaGen
Mcissonic
STAR
Baseline
Ours
Bascline
Ours
Bascline
Ours
FID
21.79 0.16
20.24
0.08
53.31 1 0.33
48.18 0.29
35.48 0.29
33.12
0.49
CLIP
25.95 + 0.02
25.95 + 0.03
25.30 + 0.02
25.62 + 0.03
25.47 = 0.02
25.63 - 0,03
DPGt
43.74
48.87 = 0.32
64.07 = 0.13
06.910011
70.28 0.12
70.40 = 0.11
HPS
21.23 0.03
21.40 0.04
29.33 = 0.03
30.06
0.02
28.70 0.08
29.07 0.11
C.3
Entropy & top-p and temperature
In the main text, we analyze the relationship between our entropy-based sampling strategy and existing sampling parameters such as CFG and top-K. By combining our method with these parameters, observe improved robustness, reducing sensitivity hyperparameter choices and yielding better FID and CLIP- Here, we further examine other sampling parameters top-p and temperature-at which are rarely used autoregressive models due their tendency distort the output distribution and severely degrade either FID or CLIP-Score. Comparative results between our method and the bascline are shown Fig.
Tappp
Tup-
Temp
Tamp:
Figure 11: Combination of our sampling strategy with existing methods (Top-p, temperature). "Temp." is short for temperature.
C.4 Additional comparison with top-K and CFG
In Sec. 4.4 of the main paper, we discussed the differences between our method and existing sampling strategies (Top-K and CFG) using LlamaGen. Here, provide additional comparisons on other models analyze the relationship between entropy-aware temperature and these conventional sampling approaches, results presented in Fig. 12 and Fig 13 Consistent with observations in

---

Sec. 4.4, the proposed strategy mitigates performance fluctuations caused by hyperparameter choices (e.g., CFG and top-K), leading to better balance between fidelity and text-image alignment.
Top
Top
CPG
CRG
Figure 12: Combination of sampling strategy with existing methods (Top-k, CFG) Meissonic.
Figure 13: Combination of our sampling strategy with existing methods (Top-k, CFG) on STAR.
C.5
Additional evaluation of performance regarding temperature
In the main text, we analyzed how adjusting the sampling temperature of tokens in different entropy ranges affects the generation quality for LlamaGen, Here, further extend the study to more models. See Fig. 14
[8.rr]
25.
37-
25.
27-
25.
30-
25.6 37-
25.1
20
25.
36
25.4
25.4
25.4
354
35
34
38
34
25
35
21
24
13
25.
44
54
14
12
25.0
26
08
28
13
25.0
FID
CLIP
Figure 14: Varying temperature by entropy range affects FID and CLIP score: lower-entropy tokens benefit from higher temperatures, and vice Experiments conducted STAR using the COCO2017 validation split by varying temperature across entropy ranges.
D Additional discussion about entropy
D.1 Entropy & generative models
D.1.1 Visualization of entropy & images
Due to space limitations the main text, we did not provide extended entropy visualizations and analysis. Here, include additional entropy maps for LlamaGen and Lumina-mGPT. See Fig. 15 and Fig. I6
D.1.2
models
We provide additional entropy based analysis of the mask model. Since the generation involves multiple timesteps, where a subset of tokens accepted at each step based on previously generated content, compute the entropy of accepted tokens at each timestep and aggregate them into final entropy map. As shown in Fig. 17 applying the proposed entropy-bases temperature leads to a more spatially balanced entropy distribution and enables richer image content while maintaining generation stability.
In addition, we further analyze the average entropy of tokens accepted at each timestep, shown Fig. T8 As discussed in the main text, due to the confidence- token selection strategy, tokens accepted in earlier steps tend to have lower entropy, since they more likely to receive high

---

Figure 15: Entropy visualization of LlamaGen.
Figure 16: Entropy visualization of Lumina-mGPT.
confidence scores. In contrast, tokens accepted in later steps (>60) exhibit significantly higher entropy. Moreover, more tokens are accepted in these later stages, which increases the risk of violating the autoregressive assumption that spatially adjacent tokens should be sampled independently as possible. This may lead to degraded image quality. Therefore, adopting a more conservative sampling strategy for these high-entropy tokens could help improve the overall generation quality.
D.1.3 Scale-wise models
For the scale-wise model, the generation process constructs a complete image by predicting logits maps at multiple scales. Each scale is conditioned on the residuals from the preceding scales, meaning that the sum of the feature maps generated at all scales is passed through the detokenizer to form final output. this generation paradigm, different scales exhibit distinct roles. Specifically, as described in [67, the earlier scales are responsible for generating the main structure of the image, while the later scales refine the result with fine details such as texture. We visualize the entropy maps of each scale during generation, shown in Fig. 19 From scale 8 to scale 12, the model

---

Baseline
Entropy map
Ours
Entropy map
Figure 17: Entropy visualization of Meissonic. Our entropy-basee based temperature leads to a more spatially balanced entropy distribution and enables richer image content.
Mean Entropy
Timestep
Figure 18: Mean entropy of each step from mask mask-prediction model. Values are averaged from 100 generated images.
tends focus corresponding to the primary subject. In contrast, at scale 13 and 14, there is no clear bias between foreground and background, indicating a more uniform attention across the image.
the foreground, with significantly higher entropy observed in the regions
In addition, we compute the average entropy for each scale, as shown in Fig. 20 The later scales exhibit relatively higher entropy, while the earlier scales tend to have lower average entropy (however a decreasing trend observed the final two scales). This further indicates that different scales earry varying amounts of information.
D.2 Entropy & generated contents
In practice, the logits are not simply positively correlated with the complexity of image content. We observe that regions with clear, well-defined content do not always exhibit high entropy; instead, their

---

Figure 19: Entropy visualization of STAR.
Mean Entropy
10
12
Scale Index
Figure 20: Mean entropy of each scale from scale-wise model. Values are averaged from ~ 100 generated images.
entropy typically falls within moderate range (c.g., between and 8). The more deterministic the content, the lower the entropy tends to be. In contrast, regions with entropy lower than 2 or higher than often correspond to simple backgrounds or overly complex, unfaithful details. Especially for regions with entropy above 8, the generated details frequently meaningless. This also explains why adjusting the logits in these low- and high-entropy areas, as discussed in our motivation experiment, does not significantly harm text-image alignment.
D.3
entropy the best indicator for information?
From a theoretical perspective, the entropy of logits reflects the model's confidence in predicting the current token. When the model sufficiently trained when its capacity strong- it may produce low entropy even in semantically important regions. In practice, we observe cases where foreground objects (e.g., faces) yield lower entropy than complex backgrounds. This suggests that the model's s confidence is not solely determined by information density, but also by the number of plausible token candidates in a region. For instance, highly structured areas like faces tend to have unique correct token and thus low uncertainty. despite containing rich semantic information. In contrast, cluttered textures such as grass or foliage may allow for more varied token predictions, resulting in higher entropy.
Based on the above analysis, entropy may need be combined with additional indicators more accurately characterize the information distribution within an image. Specifically, more precise token-wiig Miken-wise handling can be achieved by incorporating the similarity among top-ranked tokens in the logits distribution. For instance, if the entropy is low but the top tokens are not similar, the prediction can be deemed accurate; however, the top tokens are highly similar under low entropy, the randomness at that position may need to be further increased. Conversely, under high-entropy conditions, set of similar top tokens may indicate the existence of genuinely diverse possibilities. Moreover, analyzing the similarity of logits between adjacent tokens could help identify tokens that

---

require more precise predictions for example, if token's probability distribution significantly differs from that of the previous token, may warrant stricter sampling, regardless of its entropy level. We leave these directions for future exploration.
Moreover, since dynamic temperature only adjusts the randomness of the probability distribution (i.e., the variance of the logits) but not the location of its peak, further combining it with CFG may help achieve better performance.
E
Future works & limitations
E.1 Broader impacts
This work is the first to explore the decoding problem in autoregressive visual generation, highlighting the inherent differences between image and text generation. While our approach may not be fully complete and still leaves room for improvement, we hope can inspire future research further investigate this issue and develop decoding strategies tailored specifically for visual generation, ultimately advancing unified multimodal generation.
E,2
Future works
Currently, we propose training- ffren sampling strategy for image generation by adaptively controlling sampling randomness based on the distribution of predicted logits. However, this approach is sensitive to hyperparameters, and due to significant differences across backbone architectures, optimal settings vary across models. Moreover, as simple inference-time method built upon pretrained models, its performance gains may be limited for certain models.
In the future, this strategy could be integrated into the training framework for further performance improvement or acceleration. For example, it may be combined with early-exit mechanisms to allocate computation dynamically across tokens, used to guide training by leveraging entropy focus more on informative regions, thus accelerating convergence.
E.3
Limitations
The proposed method mainly mitigates issues caused by inconsistent token sampling strategies under varying information densities, but it does not enhance the intrinsic generation capability of autoregressive models. The performance gain is model-dependent. If the base model is trained with techniques that promote diverse token distributions, such as noise injection during training, or is well-trained large-scale datasets, the improvement tends be limited. Moreover, for weak base models, such as LlamaGen Stage 2, the method may offer little or no performance gain.
F
Additional visual comparison
Due to space constraints in the main paper, we did not provide additional visualizations. Here, we include further results illustrating the entropy-aware sampling behavior for LlamaGen, Lumina-mGPT. Meissonic, and STAR, well acceleration visualizations for LlamaGen and Lumina-mGii (see
Fig. 21 Fig 25

---

Sacelme
Ours
Baseline
Ours
Figure 21: Visualization of LlamaGen.

---

Baseline
Ours
Baseline
Ours
Baseline
Ours
Figure 22: Visualization of Lumina-mGPT.

---

Baseline
Prob.
Masking
Baseline
Masking
Figure 23: Visualization of Meissonic.
10

---

Baseline
Prob.
Scallewiise
Baseline
+Prob.
-Scale-wise
Figure 24: Visualization of STAR.

---

Steps:
Steps: 596 (34988s)
Steps: 518 (21.186)
Steps: 572 123 (555)
Steps: 552 (22 865)
Stops: $16 (21 60c)
Stops: 578 (24.2%)
Steps:
Steps:
SID
Steps: 1505 (68 97c)
Steps: 1417 80c)
Steps: 1563 (71.679)
Steps: 1649 (74.486)
Stops: 1656 (75 BBs)
Steps: 1609 (73.536)
Figure 25: Visualization of AR acceleration.
12


Reviews:
[{"review_id": "5XqU20ixgr", "reviewer": "Reviewer_yqnx", "summary": "The authors hypothesize that given the lower info density of image tokens and being spatially non uniform, entropy is a good indicator for balancing diversity and accuracy in the generations. Based on this premise, they propose a new decoding approach which, simply put, tries to increase diversity in easier regions of the image, while prioritising accuracy in the detailed regions. The authors present detailed evaluations and ablations to justify the various engineering choices they make in their approach", "questions": "Please see the strengths and weaknesses section. I would be happy to raise my score if my concerns are properly addressed.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThis is a well written paper that methodically identifies the distinction between text and image token distributions, and tries to address them through the lens of entropy. The method is clearly explained and intuitively makes sense. The paper also includes experiments on a variety of methods involving next token generation, masked generation and scale wise generation. The paper also makes a lot of engineering choices, which are well addressed in the ablations. The latency benefits as well as improved metrics through a purely new decoding strategy is a positive step for autoregressive visual generation.\n\nWeaknesses, Questions and Suggestions:\n\n1. Eqn 6b, $p_s = p_s / T_s$ is slightly confusing, i believe the LHS should have a \\tilde perhaps like Eqn 3, which is then carried forward appropriately?\n2. I appreciate the visualization of entropy maps after the proposed methods has been applied in the supplementary, can you also add histograms from similar regions (background and foreground) after your method -- as in Fig. 3a. Do you see a more uniformly distributed histogram now given that for low density regions, diversity has been prioritised?\n3. I understand that easier regions have better diversity using this method, but in the visuals, it also seems that the dense information regions (foreground) also is better in some sense, increasing the overall image quality, even though the accuracy factor has more emphasis here? Can the authors shed some light onto what specifically happens for the foreground part, I would have expected those areas to be almost similar as the base method. \n4. For completeness, can the authors compare their improvements with the current SOTA visual generation methods as well, just as a reference to understand where their improvements in AR generation stands as compared to diffusion/flow methods that currently lead the field.", "was_revised": true, "final_justification": "Based on the provided rebuttal, as well as the contents in the original paper, the authors present a strong entropy driven framework that nicely balances diversity and quality. Given that this idea can be potentially applied in a variety of visual generation settings, I support this paper's acceptance, and will raise my overall score to 5 (Accept).", "rebuttal": "Thanks for your affirmation. I respond to your concerns as follows:\n\n**1. Typo in equation**: Thank you for your pointing out. We agree that Equation 6 (b) may cause confusion, we correct it here as:\n\n$\\text{(Eq. 6a)} \\quad T_s = T \\cdot \\left[1 - \\beta \\cdot \\left(s - \\left\\lfloor S/2 \\right\\rfloor\\right)\\right]$\n\n$\\text{(Eq. 6b)} \\quad \\tilde{p}_s = \\frac{p_s}{T_s}$\n\n**2. Proposed method on histogram**: After applying the proposed entropy-aware sampling method, we observed a more concentrated entropy variance—that is, the distribution of token entropy across the dataset becomes more focused, with fewer extremely high or low-entropy tokens and more tokens falling in the middle range. \n\nWhile we cannot provide visualizations due to policy constraints, similar examples are shown in Figure 6 of the supplementary material. Compared to the baseline, which tends to generate images with flatter backgrounds and less detailed foregrounds (with entropy concentrated in certain regions), our sampling strategy leads to a more balanced spatial distribution of entropy, resulting in more visually diverse and structured outputs. We will add more related analysis in future versions to further support our observations.\n\n**3. What happens in foreground**: Our method not only increases randomness in low-entropy regions to promote diversity, but also reduces it in high-entropy regions to ensure stability. According to Equation 2 in the main text:\n\n$\\text{(Eq. 2)} \\quad T=T_0 e^{-\\frac{\\epsilon}{\\alpha}}+\\theta,$\n\nwhen entropy $\\epsilon$ is large, the temperature $T$ approaches $\\theta$, which is typically less than 1. This lowers randomness and encourages more confident sampling in complex (foreground) areas. Conversely, when entropy is small, $T \\approx T_0 + \\theta$, usually greater than 1, introducing more randomness to help generate diverse content in simpler regions. Thus, for these simpler regions, the model’s stochastic exploration is encouraged to enhance the richness of the generated image content; while for more uncertain regions (those with higher entropy), the model adopts a more conservative sampling strategy to ensure stable generation. Together, this approach improves the consistency between image and text while maintaining generation diversity.\n\n**4. Comparison with SOTA diffusion methods**: Thanks for pointing out, here we provide quantitative results of several diffusion and flow models, including SD v2.1, PixArt-Alpha, SDXL, and SD v3 on FID, CLIP-score (coco validation 2017), DPG and HPS, for comparison, as shown in table below:\n\n| Method                       | FID   | CLIP  | DPG   | HPSv2.1 |\n|-----------------------------|-------|-------|-------|---------|\n| SDv2.1                      | 22.87 | 26.31 | 68.09 | 26.38   |\n| PixArt-alpha                | 33.23 | 25.70 | 71.52 | 30.04   |\n| SDXL (base-1.0)             | 23.20 | 26.46 | 74.21 | 28.54   |\n| SDv3-medium                 | 29.82 | 26.24 | 85.85 | 30.22   |\n| LlamaGen (Baseline)         | 21.94 | 25.95 | 43.51 | 21.24   |\n| LlamaGen (Ours)             | 20.36 | 25.96 | 48.63 | 21.39   |\n| Lumina-mGPT (Baseline)      | 29.15 | 26.04 | 79.68 | 28.92   |\n| Lumina-mGPT (Ours)          | 27.44 | 26.25 | 79.77  | 28.87   |\n| Meissonic (Baseline)        | 53.61 | 25.27 | 63.83 | 29.33   |\n| Meissonic (+Prob.)          | 48.37 | 25.49 | 66.19 | 29.94   |\n| Meissonic (+Masking)        | 48.43 | 25.54 | 67.08 | 30.04   |\n| STAR (Baseline)             | 35.05 | 25.43 | 70.25 | 28.79   |\n| STAR (+Prob.)               | 32.75 | 25.56 | 70.83 | 28.93   |\n| STAR (+Scale-wise)          | 32.37 | 25.61 | 70.86 | 29.06   |"}, {"review_id": "NlBaWA63OP", "reviewer": "Reviewer_oqg3", "summary": "This research tackles the problem of inefficient and suboptimal sampling methods in autoregressive image generation models. The key insight is that image tokens exhibit lower and more unevenly distributed information density across different spatial areas compared to text tokens. To solve this issue, the authors introduce an entropy-based sampling strategy with two main components: first, a dynamic temperature adjustment system that varies the level of sampling randomness according to the spatial entropy patterns in token distributions; and second, an entropy-based acceptance criterion for speculative decoding that aims to speed up inference while preserving generation quality. The researchers test their approach on various autoregressive image generation frameworks, including next-token prediction, masked token generation, and scale-based models. Results demonstrate enhanced performance in both image quality metrics (such as FID, CLIP-Score, DPG, and HPS) and computational efficiency during inference.", "questions": "- Clarification on Entropy Calculation: In Section 3.2, is the entropy always calculated per-token across the entire vocabulary regardless of mask/scale, or is it pooled or smoothed spatially in any way for efficiency or noise reduction? If so, could the authors elaborate on implementation trade-offs and robustness?\n\n- Extension Beyond AR Models: The method is said to be broadly applicable to “AR frameworks based on discrete token prediction,” but could the authors comment (with any preliminary evidence or arguments) on expected utility in hybrid AR-diffusion or continuous generative models, where tokenization or spatial entropy may be ill-defined or noisy?\n\n- Failure Case Analysis: Could the authors provide additional visual or quantitative analysis of cases where the entropy-aware method causes degeneration or perceptual artifacts, especially in ambiguous or extremely high-entropy settings?\n\n- Statistical Robustness: Are the improvements (especially in FID and DPG) significant over multiple runs or seeds? Please report confidence intervals or standard deviations if available. Would such variability affect conclusions?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- **Clear Motivation**: Empirical analysis (Figure 2a) demonstrates spatial non-uniformity in image token distributions, justifying entropy-aware methods\n- **Effective Method**: Dynamic temperature mechanism well-linked to spatial entropy distributions, showing model-agnostic improvements across multiple AR generation models\n- **Comprehensive Evaluation**: Thorough experiments with multiple metrics (FID, CLIP-Score, etc.) and complete ablation studies demonstrating method efficacy\n- **Practical Value**: Entropy-aware speculative decoding reduces inference steps and latency by ~15% without quality loss\n- **Clear Visualizations**: Effective illustrations of method improvements and comparisons with existing techniques\n\n## Weaknesses\n\n- **Limited Novelty**: Significant overlap with prior entropy-based decoding work in text (e.g., EDT), with insufficient differentiation of novel aspects\n- **Generalizability Concerns**: Experiments mainly focused on COCO2017 and few AR models; limited discussion of generalization to different datasets and non-natural image tasks\n- **Missing Failure Analysis**: Insufficient discussion of scenarios where the method might degrade performance, especially in high-entropy edge cases\n- **Weak Theoretical Foundation**: Lacks theoretical justification for entropy-to-temperature mapping, relying primarily on empirical settings\n- **Statistical Significance**: Missing confidence intervals and significance testing for reported improvements\n- **Implementation Details**: Some key implementation aspects under-explained, potentially hindering reproducibility", "was_revised": true, "final_justification": "My concerns have been addressed. I will keep my current score.", "rebuttal": "Thank you for your valuable feedback! Below are our responses to your concerns:\n\n**1. Novelty**: Thank you for your concern regarding novelty. While entropy has been extensively studied in autoregressive text generation (e.g., EDT [1]), we observe that there are key differences between autoregressive image generation and text generation, which make it inappropriate to directly apply these methods to image generation. In fact, EDT proposes reducing the entropy in low-entropy tokens to sharpen the distribution and increasing randomness in high-entropy regions to improve diversity. In contrast, our work focuses on the sampling strategy for autoregressive image generation and proposes increasing randomness in low-entropy regions, which is opposite to EDT’s approach and brings better image generation quality.\n\n**2. Generalizability**: Due to resource limitations, it is challenging for us to systematically evaluate a wide range of scenarios. Besides the COCO dataset, we also report results on DPG and HPS—where DPG assesses the model’s ability to follow complex text prompts, and HPS evaluates the aesthetic quality of generated images. These metrics are commonly used to evaluate multiple aspects of image generation models. We believe they provide meaningful insights, and we plan to conduct more comprehensive evaluations in future work.\n\n**3. Failure analysis**: Due to policy restrictions, we are unable to provide visualizations. In some models, such as Lumina-mGPT, high-entropy regions may actually reflect good diversity, and lowering temperature there can reduce visual quality. Encouraging diversity in low-entropy regions may also occasionally introduce artifacts. We will include further analysis and visual cases in future versions. For a brief discussion of limitations, please refer to Section E.3 in the supplementary material.\n\n**4.Theoretical foundation**: Thank you for pointing this out. Indeed, the current entropy-based temperature adjustment is empirically designed. We believe this originates from the fact that continuous image information is discretized into token indices, and the information distributions of images and text differ significantly. Since generation quality is highly sensitive to these distributions and the learned distribution is often imperfect, we apply further adjustment based on token-level entropy. As we currently lack a suitable theoretical framework to rigorously analyze this phenomenon, we have instead provided empirical evidence in the main text and hope to establish a stronger theoretical foundation in future work.\n\n**5. Statistical significance**: Thank you for pointing this out. We found that generation-related metrics are indeed sensitive to random seeds. To further analyze this, we randomly selected 10 seeds from the range $[0, 1e6]$, ran the generation model 10 times conditioned on these seeds, and computed the mean and standard deviation of the results, as shown in the table below, the rand seeds have little impact on the performance improvement brought by our method, further supports the positive impact of the proposed method on performance.\n\n*LlamaGen:*\n| Metric | Baseline (mean) | Ours (mean) | Baseline (std) | Ours (std) |\n|--------|------------------|-------------|-----------------|------------|\n| FID    | 21.79            | 20.24       | 0.161           | 0.086      |\n| CLIP   | 25.95            | 25.95       | 0.022           | 0.036      |\n| DPG    | 43.74            | 48.87       | 0.348           | 0.326      |\n| HPS    | 21.23            | 21.40       | 0.031           | 0.042      |\n\n*Meissonic:*\n| Metric | Baseline (mean) | Ours (mean) | Baseline (std) | Ours (std) |\n|--------|------------------|-------------|-----------------|------------|\n| FID    | 53.31            | 48.18       | 0.326           | 0.285      |\n| CLIP   | 25.30            | 25.62       | 0.015           | 0.025      |\n\n*STAR:*\n| Metric | Baseline (mean) | Ours (mean) | Baseline (std) | Ours (std) |\n|--------|------------------|-------------|-----------------|------------|\n| FID    | 35.48            | 33.12       | 0.291           | 0.494      |\n| CLIP   | 25.47            | 25.63       | 0.024           | 0.027      |\n\nNote that due to time constraints, we were unable to complete this analysis for all models on all metrics, but we will include the full results in a future version.\n\n**6. Clarification on entropy calculation \\& Implementation details**: The entropy described in Section 3.2 is computed over all spatial tokens—specifically, we calculate the entropy of the probability distribution used when generating each token, without applying any denoising or spatial smoothing strategies. And our work will be open source later.\n\n**7. Hybrid AR and continuous models**: This is a interesting question. The concept of entropy is specific to discrete token-based generation, where each token has an associated probability. This issue does not naturally arise in continuous generation frameworks.\n\nOf course, we believe there can be some alternative ideas in continuous visual generation. For example, the sampling process in discrete models is somewhat analogous to the noise injection process in continuous methods. In multi-step SDE-based models, it may be possible to apply different noise levels to different regions of the image based on content complexity, which could potentially improve generation quality. However, since entropy is not defined in continuous space, other indicators like variance may be needed to guide such adjustments.\n\nFor hybrid discrete-continuous models, it depends on how the two parts are combined. Our method is designed to balance diversity and stability in discrete generation. If the discrete part is responsible for generating fine details (as in models like HART [2]), our approach may still be helpful. Conversely, if the discrete AR component is only used to generate high-level (like BLIP3-o [3]) or abstract information, our method may not be effective in that case. The effect of noise in the latent space on entropy remains unclear and would require further investigation.\n\n**Reference**\n\n[1] EDT: Improving Large Language Models’ Generation by Entropy-based Dynamic Temperature Sampling\n\n[2] HART: Efficient Visual Generation with Hybrid Autoregressive Transformer\n\n[3] BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset"}, {"review_id": "PcoJNNcQy7", "reviewer": "Reviewer_M2Vd", "summary": "The paper proposes a new approach to improve the inference of autoregressive (AR) image generation models by leveraging entropy-based sampling strategies. The authors demonstrate that image tokens exhibit low information density and non-uniform spatial distribution. To address this, they introduce an entropy-informed decoding strategy including (a) a entropy-based sampling temperature for better quality of generated images, and (b) an entropy-aware acceptance rules in speculative sampling for better acceleration. The prpopsed method is validated on COCO2017val benchmark and various frameworks.", "questions": "Please refer to weakness.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strength\n\n- The proposed method demonstrates consistent performance and improved inference speed across multiple generation models, including standard AR, mask-based, and VAR models.\n\n- The paper provides thorough quantitative and qualitative analyses that illustrate the entropy characteristics of visual generation models that use discrete tokens and demonstrate the effectiveness of the proposed method.\n\n- This paper is well-written and easy to follow.\n\n# Weakness\n\n- The paper lacks a theoretical analysis of whether the proposed speculative sampling strategy (Section 3.4) preserves the original token distribution or quantifies the error bound of potential quality loss.\n\n- This paper lacks the evaluation on the COCO2014val benchmark which contains 30k images to provide precise FID values.\n\n- The paper does not investigate whether the proposed entropy-aware temperature could benefit NLP models.", "was_revised": true, "final_justification": "Thank you for your rebuttal. This rebuttal has solved most of my problems. Experiments have proven the proposed method works well across multiple baselines. Considering it lacks a theoretical proof, I'll hold a Borderline accept score.", "rebuttal": "Thank you for your recognition and correction. Here are my answers to your concerns:\n\n**1. Token distribution of speculative strategy**: Thanks for pointing out. Our proposed speculative decoding modification does not fully preserve the original model’s token distribution. In fact, we found that appropriately adjusting the token probability distribution of the autoregressive generation model—such as with the entropy-aware temperature method described in our approach—helps produce higher-quality images.  Specifically, the scale term in Equation 8 in mainpaper encourages the acceptance of low-entropy tokens by increasing randomness in their distribution, while the random term sharpens the sampling distribution for high-entropy tokens by suppressing excessive randomness.\n\n**2. COCO2014val dataset evaluation**: Thank you for your comment. Here, we provide FID and CLIPScore results on the COCO2014val dataset, shown in the table below:\n| Model          | Method           | FID   | CLIP    |\n|----------------|------------------|-------|---------|\n| Llamagen       | Baseline         | 13.37 | 0.2561  |\n| Llamagen             | Ours          | 11.59 | 0.2560  |\n| Meissonic      | Baseline         | 44.54 | 0.2567  |\n| Meissonic             | Ours+Mask     | 38.95 | 0.2590  |\n| STAR           | Baseline         | 24.86 | 0.2581  |\n| STAR             | Ours+Scale    | 22.09 | 0.2598  |\n| Lumina-mGPT*    | Baseline         | 20.78 | 0.2641  |\n| Lumina-mGPT* | Ours          | 18.84 | 0.2660  |\n\nNote that * means result from a subset with around 14,000 images.\n\nDue to time constraints, the Lumina-mGPT model has not completed the standard 30k-image inference; we report its metrics on a subset (~14,000 images) of images and commit to including the full COCO-2014 results in a future version.\n\nAdditionally, since the Meissonic model’s generated images differ significantly in COCO style, we provide its performance on the MJHQ-30k dataset: our proposed method reduces the FID on MJHQ from 20.50 to 17.96 and improves CLIP-score from 27.33 to 27.61.\n\n**3. Entropy and NLP models**: Thank you for your insightful comment. While entropy has been extensively studied in the context of NLP, especially in works such as EDT [1] and SED [2], our study focuses on the entropy distribution of tokens and the corresponding sampling strategies in autoregressive visual generation—a perspective that, to the best of our knowledge, has not been thoroughly explored in prior work. For instance, EDT recommends reducing the entropy of low-entropy tokens to sharpen distributions and increasing randomness in high-entropy regions to promote diversity, while SED emphasizes that high-entropy tokens are more likely to lead to confusion and require careful handling. Moreover, several recent GRPO strategies also incorporate entropy-aware mechanisms. We believe that our visual generation setting poses distinct challenges that motivate a different design choice, as discussed in our work.\n\n**Reference:**\n\n[1] EDT: Improving Large Language Models’ Generation by Entropy-based Dynamic Temperature Sampling\n\n[2] SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation"}, {"review_id": "1JOOCke6Pa", "reviewer": "Reviewer_Y8n9", "summary": "The authors propose to measure entropy and use it to set temperature dynamically to improve speed and quality of autoregressive image generation model. They did some initial experiment with CLIP score and FID vs temperature for different entropies and observed that \n\n1. In high-entropy regions, lower randomness helps improve text-image consistency.\n2. In regions with extreme low entropy, increasing sampling randomness consistently improves visual quality, while having negligible impact on text alignment.\n\nBased on these observations, they propose to set temperature dynamically according to entropy to improve the sampling process. Further, the authors also demonstrate the effectiveness of the method with mask-prediction models and scale-wise models. Finally, they also use entropy to improve speculative decoding and therefore accelerate the inference speed.\nExperiments show promising results and validated the method.", "questions": "1. Could you explain more why \"regions with simple content (e.g., solid colors) typically exhibit lower entropy while more complex foreground (e.g., objects, structures, and textures) areas have higher entropy\" the entropy here sounds similar to frequency, is there any connection?\n2. For figure 3(c), could you add more details of how the experiment is performed? Which dataset is used and which model is used to generate the distribution?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n1. The idea is novel. \n2. The paper is well written and is easy to follow\n3. In addition to the main results, authors have done various ablation studies to validate the proposed method\n\n\nWeakness:\n\n1. Need to provide more details about the initial finding for Figure 3(c). Ideally use different model and dataset to make sure the conclusion is solid.\n2. \"Mask-prediction models\" section needs some improvement. I did not get why \"This design further encourages randomness in low-entropy regions while enhancing accuracy in high-entropy regions\"\n3. The 15% speed up doesn't seem to be a large improvement for acceleration.", "was_revised": true, "final_justification": "My concerns are addressed.", "rebuttal": "We sincerely appreciate your time and effort in reviewing our paper. Below are our point-by-point responses:\n\n**1. Details \\& More experiments about Figure 3 (c)**:\n*Details:* Figure 3(c) is based on the LlamaGen-stage1 model and computed on the full COCO2017-val dataset at a resolution of 256. The CFG scale is set to 7.5, with no top-k or top-p sampling applied. Except for the temperature, all other settings follow the official GitHub repository’s parameters.\n\nFor temperature perturbation, as described in the main text, our temperature perturbation strategy involves computing the entropy of each token during generation. Tokens are categorized into five predefined entropy intervals ([0,2), [2,4), [4,6), [6,8), [8,∞)), and for each interval, we apply a specific temperature to adjust the token distribution. The overall image quality is then assessed on the dataset.\n\n*More experiments:* Additionally, we have added the evaluation metrics for STAR model on COCO2017 (due to limited time, we will provide evaluation on more models and more datasets in the next version), as shown in the table below:\n\n| Entropy Range | Temperature |   FID  |  CLIP |\n|---------------|-------------|--------|-------|\n| Baseline      | all 1.0     |  35.05 | 25.43 |\n| Ours          | dynamic     |  32.75 | 25.56 |\n| [0, 2)        | 2.5         |  33.36 | 25.43 |\n|               | 2.0         |  33.98 | 25.40 |\n|               | 1.5         |  35.01 | 25.42 |\n|               | 1.2         |  35.05 | 25.43 |\n|               | 0.8         |  35.16 | 25.44 |\n|               | 0.5         |  35.19 | 25.41 |\n| [2, 4)        | 2.0         |  33.06 | 25.40 |\n|               | 1.5         |  33.69 | 25.45 |\n|               | 1.2         |  34.16 | 25.46 |\n|               | 0.8         |  35.54 | 25.42 |\n|               | 0.5         |  35.99 | 25.44 |\n| [4, 6)        | 2.0         |  37.63 | 25.14 |\n|               | 1.5         |  36.15 | 25.29 |\n|               | 1.2         |  35.25 | 25.37 |\n|               | 0.8         |  34.87 | 25.48 |\n|               | 0.5         |  36.15 | 25.48 |\n| [6, 8)        | 1.5         |  35.76 | 25.30 |\n|               | 1.2         |  35.29 | 25.36 |\n|               | 0.8         |  34.76 | 25.49 |\n|               | 0.5         |  35.38 | 25.59 |\n| [8, inf)      | 1.5         |  35.19 | 25.42 |\n|               | 1.2         |  35.18 | 25.41 |\n|               | 0.8         |  35.19 | 25.42 |\n|               | 0.5         |  35.20 | 25.42 |\n\n\n**2. Why masking order helps**: In mask-based generation models, each step selects which random tokens to keep based on a predefined rule—typically by adding noise to the confidence scores and keeping the top-k tokens (see Equations 4 and 5 in mainpaper). This can be viewed as a confidence-based top-k sampling strategy, where higher-confidence tokens at a given timestep are more likely to be selected, and the noise level controls the randomness.\n\nFrom our observations, due to the mask scheduler’s design, low-entropy tokens (which tend to have higher confidence) are selected earlier in generation, while many high-entropy tokens are accepted together in later stages (as shown in Figure 4). This can hurt image quality. By adjusting the noise level based on entropy—using lower noise in high-entropy regions and more randomness in low-entropy ones—we encourage more cautious token selection where needed and allow more exploration in simpler areas, which leads to better image quality.\n\n**3. Speedup Ratio**: Our proposed acceleration strategy reduces inference cost by an additional ~15\\% on top of existing acceleration method [1], achieving around 2.3× speedup compared to the original model. Although the current performance gain is modest, this approach highlights the potential of exploiting image non-uniformity for faster generation. We are actively investigating ways to further improve its effectiveness.\n\n**Reference:**\n\n[1] Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding"}]

Author rebuttal:

Meta-review: This paper presents an entropy-informed decoding strategy to enhance autoregressive (AR) image generation, addressing lower information density and non-uniform spatial distribution of image tokens. The approach introduces dynamic temperature control and entropy-aware speculative decoding, improving quality and speed. Reviewers praised its clear motivation, model-agnostic effectiveness across AR frameworks, comprehensive experiments, and practical latency reductions (~15% inference cost). Concerns included limited theoretical foundation for entropy-temperature mapping, generalizability beyond tested datasets/models, missing failure analysis, and modest speedup magnitude. However, the authors addressed key questions in their rebuttal, clarifying implementation details and validating robustness.
Despite minor limitations, the work makes valuable contributions to AR image generation by leveraging entropy to balance diversity and accuracy, with strong empirical support. Its practical improvements and broad applicability justify recognition. The AC concurs this is a worthwhile advancement and recommends acceptance.
Decision: Accept (poster)