D-AR: Diffusion via Autoregressive Models
Authors: Ziteng Gao
Mike Zheng Shou
Venue: ICLR 2026 Poster (2026)
Keywords: visual generation
diffusion models
autoregressive models
flow matching
Primary area: generative models

Abstract: This paper introduces Diffusion via Autoregressive (D-AR) models, a new paradigm recasting the pixel diffusion process as a vanilla autoregressive procedure in the standard next-token-prediction fashion. We start by designing the tokenizer that converts an image into the sequence of discrete tokens, where tokens in different positions can be decoded into different diffusion denoising steps in the pixel space. Thanks to the diffusion property, these tokens naturally follow a coarse-to-fine order, which directly lends itself to autoregressive modeling. Then, we apply standard next-token prediction to these tokens, without modifying any underlying designs (either causal masks or training/inference strategies), and such sequential autoregressive token generation directly mirrors the diffusion procedure in image space. That is, once the autoregressive model generates an increment of tokens, we can directly decode these tokens into the corresponding diffusion denoising step on pixels in a streaming manner. Our pipeline naturally reveals several intriguing properties, for example, it supports consistent previews when generating only a subset of tokens and enables zero-shot layout-controlled synthesis. On the standard ImageNet benchmark, our method achieves 2.09 and 2.00 FID using a 775M and 1.4B Llama backbone with 256 discrete tokens. We hope our work can inspire future research on unified autoregressive architectures of visual synthesis, especially with large language models.

Published as a conference paper at ICLR 2026
D-AR: DIFFUSION VIA AUTOREGRESSIVE MODELS
Ziteng Gao
Mike Zheng Shou Show Lab, National University of Singapore mike. zheng. gma atmmall.co.
Show Lab, National University of Singapore pzt@outlook.com com
DIFFUSION inttal nole
D-AR can produce rough images with partial AR tokens gonerated
Seneinng desclling step $2?
Thaa densking stop
OCT
000
000
000
12 the tokaee
25% takans
100% tokann
cord
Pleer diffctio
autoregressive transformer
nonvertiooa All with Isg. Uamu)
Class Sokie
AR
standard next token prediction with visual tokens
STa la lovenee
Figure 1: Diffusion via autoregressive modeling (D-AR) framework for visual generation. As the autoregressive transformer generates tokens, D-AR can simultancously perform corresponding dif- fusion steps via token conditioning and jump-estimate target samples rough previews effortlessly.
ABSTRACT
This paper introduces Diffusion via Autoregressive (D-AR) models, a new paradigm recasting the pixel diffusion process vanilla autoregressive pro- cedure in the standard next-token prediction tokenizer that converts image into the sequence of discrete tokens, where to- kens in different positions can be decoded into different diffusion denoising steps in the pixel space. Thanks to the diffusion property, these tokens naturally fol- low a coarse-to-fine order, which directly lends itself to autoregressive modeling. Then, we apply standard next-token prediction these tokens, without modify- ing any underlying designs (either causal masks or training/inference strategies), and such sequential autoregressive token generation directly mirrors diffusion procedure in image space. That is, once the autoregressive model generates an increment of tokens, we can directly decode these tokens into corresponding diffusion denoising step pixels in streaming manner. Our pipeline naturally reveals several intriguing properties, for example, it supports consistent previews when generating only subset of tokens and enables zero-shot layout-controlled synthesis. On the standard ImageNet benchmark, our method achieves 2.09 and 2.00 FID using 775M and 1.4B Llama backbone with 256 discrete tokens. We hope our work can inspire future research on unified autoregressive architectures of visual synthesis, especially with large language models. Code and models are available at
next-tokee--reeditton fashion. We start by designing the
INTRODUCTION
Autoregressive models, exemplified by large language models (LLMs) (Touvron al., 2023 GPT- 4- Team 2024 Jlama-3-Teal 2024), now underpin modern NLP, delivering otaee-ofttheann art results
X Corresponding Author.

---

Published as a conference paper at ICLR 2026
9
Figure 2: Uncurated generated samples from D-AR-XL with 256 x 256 resolutions (CFG=4.0).
with simple next-token prediction objective. With widespread adoption of AR models, this simple next token prediction paradigm has established as the de facto standard in modern LLM systems and fostered software ecosystem for optimizing such training and inference pipelines (Shoeybi et al.,
2020
Kwon et al., 2023] Zheng et al. 2024). The remarkable success of autoregressive models in
language has also inspired exploration into visual generation tasks (Esser al. 2021 Sun al.
2024 Lee et al. 2022). with the broader goal of building unified frameworks of both vision and language (Team 2024 Zhou et al. [2225] Xie al., [025 Ma et al.) (2024)) However, unlike text, where sequential structure is naturally defined, images lack an inherently linear ordering, posing challenges for adapting such paradigm to vision modeling. Recent studies explore different visual orderings in autoregressive modeling (Tian et al., (2024. Pang et al. 2024 Yu et al.) 2024a Ren et al.)
2025| Li et al. 2025). However, these approaches typically require significant modifications to the core mechanisms, often deviating from the standard next token prediction objective.
At the same time, modern vision generation pipelines are most dominated by diffusion paradigms (Song et al. [202I] Ho et al. 2020; Lipman al., 2023). exemplified by several com- mercial systems (Ramesh et al. 2021 Labs 2023] Podell et al. 2024). The diffusion pipelines excel modeling continuous image signals: starting from random noise, they iteratively refine in- put through denoising to produce high-quality images. However, diffusion sampling requires many dense sequential dense denoising steps and such architectures pose challenges for seamless integra- tion with LLMs and limit their potential in unified multi-modal systems.
In this paper, we aim bridge the diffusion process and autoregressive modeling for visual gen- eration, leveraging strengths from both paradigms. Importantly, we maintain a strict adherence to the standard next-token prediction paradigm and make no changes the underlying autoregressive mechanism to "simulate" the diffusion process on images. To achieve this, we present the sequential diffussion tokenizer to reinterpret the diffusion process on raw image pixels to-fine discrete tokens. In this formulation, early tokens represent conditions in early diffusion steps from pure noise, whilst later tokens capture progressive steps over less noised inputs, leading to a naturally linearized decomposition of visual sequence. We design the diffusion model in the pro- posed tokenizer to be light and fast, i.e., with around 185M parameters and diffusion steps without extra VAEs, and achieve 1.52 rFID on ImageNet (Deng et al. 2009] with total budget of 256 dis- crete tokens. With this design, we can perform the diffusion process on image pixels via predicting next token in token sequence with the autoregressive mechanism unchanged. Therefore, name
sequence of coarse-
2

---

Published as a conference paper at ICLR 2026
this framework as D-AR (Dffusion via Autoregressive) models. D-AR excels the ImageNet class-conditioned generation benchmark. With the plain LLaMA backbone (Touvron et al. 2023) backbones, 775M and 1.4B D-AR models achieve the leading 2.09 and 2.00 gFID with total of 256 tokens in the standard nex--token-prediction AR regime. We hope our work can inspire future research integrated multi- modal L.M architectures with native visual generation capabilities.
2 RELATED WORK
2.1 DIFFUSION AND AUTOREGRESSIVE MODELS
Diffusion models and autoregressive models are currently two main streams of modern generative
modeling. Diffusion models (Ho et al. 2020 Song et al. 2021 Lipman et al. [023] Liu et al.,
2023), exemplified by several commercial text-to-image models (Labs] [023] Ramesh et al. [202]), excel in generating high-quality visual content by iteratively denoising sample from iniiial noise. Though powerful in generating visually pleasing images, the diffusion process typically operates in dense manner and requires significant sampling steps, which can be computationally expensive. Recent success in language modeling using autoregressive paradigm, especially large
language models (Llama-3-Team) 2024; GPT-4-Team 2024 Team 2025 Bai al., 2023), has inspired researchers to explore the potential of this paradigm in visual generation tasks due to its scalability and mature training and inference infrastructures. However, this adaptation raises several challenges, since images are not inherently discrete and linear structures like text. To this end, researchers use vector quantized autoencoders to quantize images into discrete latent codes (van den Oord et al., 2017, Esser et al. [201]) and use raster-scan order to model the image sequence (Sun ict al., 2024 Team 2024, Wang et al., 2024). Researchers have also found that image sequence ordering can be defined in various ways (Tian et al., 2024| Pang et al. 2024 Ren et al., 2025 Yu ctt al. 2024a), and the next-token prediction paradigm should be adapted to suit vision modeling accordingly.
Though sorts of visual autoregressive models have been proposed, the dominant role of diffusion models in visual generation tasks remains almost unchanged due to their outperforming capabilitiess at visual continuous signals. In this paper, we seek to bridge diffusion models and autoregressive models for visual generation and leverage the advantage of both sides, following previous efforts in this research line (i et al. 2024 Gu et al. 2025aa Chen et al. 2024 Deng et al. 2024| Wu et al.) 2024| Pan et al.) 2025b Ge et al. 2024 Zhou al. 2025 Gu al. 2025b). But different from these work, we strictly adhere to the standard next-token vext-ooke--rediccioo autoregressive paradigm with discrete inputs and outputs, and design diffusion in the tokenizer decoder in a sequential manner to tackle with visual continuous data.
2.2 VISUAL TOKENIZATION WITH DIFFUSION MODELS
How to encode images into sequences of discrete tokens and then effectively reconstruct pixels from them is key design for visual generation in autoregressive models. Due to the vector quantization and downsampling operations, visual tokenization methods inevitably suffer from the loss of infor- mation and lead to suboptimal reconstruction quality, which researchers have put intensive efforts into improving (van den Oord al., 2017 Esser et al.) 2021 Yu al.) 2022 Lee al. (022).
Concurrently, research direction recently emerges on leveraging diffusion models to decode visual
tokens back into image pixels (OpenAl 2023 Zhao et al. 2024 Tang et al. 2024 Sargent et al.
2025 Tang et al. 2024). Specifically, these methods typically see discrete tokens as conditions in the diffusion process. By doing so, they offload visual ambiguity and fine details the diffusion
model and significantly improves the visual fidelity (Zhao et al., [2024] Chen et al. 2025 Sargent
et al. 2025). Further work on this line argues that discrete tokens should focus on structural seman-
tics of images and extract such semantics with flexible sequence length (Wen et al. 2025
Bachmann
ct al. 2025) by large latent diffusion models together with VAE (Kingma & Welling 2014
Rombach
al. 2022
To our best knowledge, our method is the first to propose the tokenizer to interpret the full diffusion process into the autoregressive sequential generation using the diffusion tokenizer. Our method individually developed from related work, DDT-LLama (Pan et al. 2025a) and Selftok (Wang al. 2025), which also uses diffusion decoder sequentialize tokens but in reversed order or
3

---

Published as a conference paper at ICLR 2026
visual tokenization
visual de-tokenization
input
pixel diffusion transformer
query
condition
difisiion timesiap 1 =-x use i-th group of tokons according c(t)
encoder layers
discrete token
reading out sequencidly
vector quantize
telected group of tokkess
Figure 3: Sequential diffusion tokenizer structure. When training the tokenizer, the pixel dif- fusion transformer in the tokenizer decoder calculates the velocity loss with the selected group of tokens, c(I), conditioning tokens.
a recursive way. Also, Pan et al. (025a) and Wang et al. (2025) cannot represent diffusion steps as sequential AR generation process and therefore cannot decode with partial tokens generated by autoregressive models, marking a key distinction from our method and underlying motivation.
3 METHODS
A critical challenge in visual autoregressive modeling (Sun et al. 2024 Tian et al. 2024) is, for long time, how tokenize a 2D image into sequence of discrete tokens since images inherently 1D linear structures like text. Though several works defined the ordering of image pix- els (Pang et 2024, Tian al.] 2024 Yu et al., [024a] Ren al.) 2025), they either introduce spatial inductive bias or require tailored autoregressive designs for vision, posing challenges on unified autoregressive framework.
We propose a systematic solution to address this with Diffusion via Autoregressive models (D- AR), which recasts the image diffusion process as a fully autoregressive model in the standard next-token-preddee
manner. The high-level idea is to perform the diffusion process pixels via autoregressive modeling. To start, we design a sequential difflusion tokenizer that tokenizes images into sequences of ID discrete tokens, which can be sequentially decoded as diffusion steps from the first to the end token. We apply standard next-token prediction on these tokens using a Llama decoder-only autoregressive backbones (Touvrrn et al. 2023). without modifying any AR architecture (either causal masks or training/inffernce designs) to generate images.
3.1 SEQUENTIAL DIFFUSION TOKENIZER
The sequential diffusion tokenizer is designed tokenize images into ID lincarized discrete tokens in the ordering of progressive diffusion steps. The overall tokenizer structure is shown in Figure 3 akin to conventional visual tokenizers, which encodes images into latents, quantize them into dis- crete ones, and then decode them back into diffusion over pixels in auto-encoding manner.
ID encoding. Similar to ID tokenization approach YYu et al. 2024b). the sequential diffusion tokenizer first encodes the image into ID sequence of discrete tokens using a transformer:
21, Z2, z = (U, [qi ......, qNl)),
(1)
where is the input image, typically patchified as set of patch tokens, & is the transformer en- coder /Vaswani et al. 2017), QUANT(-) is the vector quantizer (van den Oord et al. 2017), and [qi] are learnable query tokens, where N the total number of queries. In this step, we not impose specific ordering on the resulting 1D token sequence, which we will further focus on below.
Sequential diffusion decoding. We propose the sequential diffusion decoder to decode ID quan- tized token sequence into consecutive diffusion steps image pixels. The diffusion decoder is diffusion transformer (Peebles Xie 2023), which takes tokens different positions in the se- quence conditions in different diffusion steps. Here, flow matching loss with velocity prediction,
4

---

Published as a conference paper at ICLR 2026
simplified variant of diffusion families (Liu et al. 2023 Lipman et al. 2023
Ma et al.
2024a), is
used to train the diffusion decoder. The loss is defined as:
fm = Ex [Ive Dru(x,, 1, c(t))
(2)
where the flow interpolant is defined as:
X
|x| - (1 (1-100 -1x00
Vz = -dx//l/ = XX X0,
(3)
Xg ~ (0, x1 =1, I, t [0. 1].
With this notation, Xp at timestep = 0 represents pure noise and XI = I at = represents the real data sample. During inference, samples can be generated by solving ordinary differential equation (ODE) from t = 0 to t = when the condition schedule c(t) is given.
The condition schedule c(t) is set of quantized tokens z; used as conditions in the diffusion decoder at timestep f. To enable the sequential decoding property, we design the condition schedule c(t) to start from the first token ž1 and reach the last token n as the flow matching timestep progresses from 0 to 1. In preliminary experiments, we find that multiple z; for a specified timestep is crucial for good performance. We thus first group consecutive tokens z; into K groups, {gi, ..,).. each group g, with N/ K tokens. The condition schedule is then defined as:
c(t)= = g(t-K]
=== - (1/3) * ()); 1));
(5)
where t' is the shifted timestep and |3 is a control parameter. When 3 1, time ranges are evenly split regarding the condition group gi. The higher 33 values lead to denser tokens as conditions over early diffusion steps, which we find empirically beneficial for reconstruction quality.
Discussion. One can view the ID sequence of tokens as the "proxy" of the underlying diffusion procedure on pixels controlled by conditioning tokens c(t). With sequential diffusion decoding. decode increments of AR tokens into consecutive diffusion sampling steps on pixels in the streaming way when reading out tokens sequentially. This token order is naturally linearized by the diffusion process, where early tokens represent conditions needed in early diffusion steps (l - 0) over noisy inputs, often low-frequency spatial layout. Later tokens describe the information needed in later steps + 1) over less noisy inputs, typically localized details or structures (Rissanen al.) 2023). This coarse-to-fine token ordering is well-suited for autoregressive modeling, as shown in experimental section. Also, by the diffusion decoder, the tokenizer decoder can delegate ambiguous details to diffusion and thus focus on semantics (Hudson et al. 2024
3.2 AUTOREGRESSIVE MODELING
Once we have the linearized sequence of discrete tokens by our proposed tokenizer, we can apply standard autoregressive next token prediction to model the image generation process:
N
(2)
po (z, [Z) Zi......
(6)
where 0 the AR model parameters and one can use simple cross entropy loss to optimize parame- ters. In this paper, resort to the decoder-only transformer architecture (Touvron al., 2023
Sun
et al. 2024) for autoregressive modeling.
Vanilla vision autoregressive modeling. General autoregressive modeling assumes linear order- ing of data elements, which is hard to define in images. By using tokens produced by the sequential diffusion tokenizer, D-AR keeps the same discrete inputs and outputs, attention masks/kernels, loss functions, and inference logistics as standard AR models
3.3 DIFFUSION VIA AUTOREGRESSIVE MODELS
The presented framework, diffusion via autoregressive models, simply consists of the sequential diffusion visual tokenizer and the Llama decoder-only transformer on discrete token sequences. Note here that the sequential diffusion tokenizer directly operates on raw pixels and do not require extra VAEs (Kingma & Welling 2014 Rombach al. 2022
5

---

Published as a conference paper at ICLR 2026
Markovian diffusion procedure via vanilla autoregressive models. As the name implies, quential generation in the D-AR framework directly corresponds to diffusion procedure on image pixels via the bridge of token conditioning. When are generating sequence of tokens, we can perform the diffusion sampling on pixels simultaneously whenever we have condition tokens needed at diffusion timestep i ready, i.e., c(t). Since the diffusion is only controlled by autoregressive mod- els via condition tokens, we do not break the Markovian convention of diffusion models, different from conceptually related work (Gu et al., 2025a). Therefore, D-AR can leverage advantages of both diffusion and autoregressive worlds:
1. KV cache-friendly inference: as the D-AR framework uses autoregressive decoder-only transformers on token sequences, natively supports KV eache-friendly fast inference;
2. Streaming pixel decoding and consistent previews at no extra costs. We can perform diffusion steps pixels instantly whenever we have needed tokens ready in a streaming manner. Also, since the diffusion decoder is directly operating on pixels, we can use the diffusion property to jump-estimate the target and generate consistent previews effortlessly;
3. Zero-shot controlled synthesis. As the token sequence is linearized by diffusion, we can simply condition several prefix tokens to control the visual generation without finetuning.
IMPLEMENTATIONS
Sequential diffusion tokenizer architecture. For the encoder in diffusion tokenizer, we mainly follow the design of 11) tokenizer (Yu et al. 2024b) to use the transformer encoder layers jointly processing image patches and learnable query tokens. We apply a causal mask to query tokens to enforce the basic causality
queries but allow both query tokens and image tokens to attend to arbitrary image tokens. As default, we set the number of queries = 256, input patch size p 16, the dimension of transformer d 768, and the transformer layer L 8. Following (Sun) et al., 2024), we use the vanilla vector quantization with Ez-normalized codebook entries, configured with codebook size n. 16384 and dimension d,
We expect better performance with more advanced quantization approaches (Mentzer et al. 2024 Yu et al. 2023) but leave for future work.
We design the diffusion decoder as the diffusion transformer architecture (Peebles & Xie 2023 Ma iet al. 2024a) but on raw pixel patches, which integrates zero-initialized adaptive layer normalization al., 2018). To condition the diffusion decoder with condition tokens c(t), we use the cross attention layer on patch tokens to attend to condition tokens and take attention output as the input of the AdaLN, together added by the time t embedding. The diffusion transformer decoder is configured moderately with La = 12 layers, d, = 768 hidden dimension, and patch size p. = 8, resulting in total parameter of 185M.
(AdaLN)
We add causal decoder transformer layers on encoded tokens z, after the vector quantization and before diffusion decoding, to produce z' for more nonlinearity. We configure it as the same as the transformer encoder. Note that these decoder transformer layers with causal masks do not break the causality of the token sequence. The total parameter of the sequential diffusion tokenizer is 300M.
Training sequential diffusion tokenizer. Training diffusion models on raw pixels with few in- ference steps is challenging task (Hoogeboom al. 2023 2024), even with the strong image encoded conditions (Zhao et al.] (024)). To enable few-step inference and speed up the convergence, use the perceptual matching loss based on LPIPS (Zhang et al., 2018 Zhao et al. (024) and
representation alignment (REPA) loss (Yu et al. 2024c) together with flow matching (2) and vector quantization loss to train the sequential diffusion tokenizer:
Lokenizer = 4m + lvg + XI + X2
where we assign A 0.5 and X2 = 0.5. We do not use adversarial matching loss (Zhao et al. 2024] in our training since observe the instability and over-saturation issue.
In a training forward pass, we first encode an image into a quantized token sequence and use trans- former decoder layers compute z'. Then randomly sample a flow matching timestep t E [0, 1], determine which group gi of z should be used as conditions for diffusion decoder according to the condition schedule c(l), and compute the final loss Eyokenizer-

---

Published as a conference paper at ICLR 2026
Sampling with sequential diffusion tokenizer. Given the token sequence, either encoded from images or generated from autoregressive modeling, we can perform the flow matching sampling by reading out tokens in the sequential order based on the condition schedule c(t). For simplicity and efficiency, we design the default sampling schedule to use each condition group exactly once, that is, bind the number of sampling steps to the number of condition groups K and use timeshifted schedule in the reversed form of (5), following (Esser et al. 2024):
i/K
(i/K)+ + i1-/ i= 0, 1,........ 1-
This sampling schedule results in denser early sampling steps when B and default set = 2 and K = 8 for sampling efficiency, resulting in each conditioning group with N/K 32 tokens. Again, for efficiency, we do not use classifier-free guidance (CFG) (Ho & Salimans 2022) in diffusion sampling decoding steps.
AR models. Our AR model architecture is exactly the same as Llama decoder-only transformers, which are with RMSNorm (Zhang & Sennrich (019) and SwiGLU [ Note that since tokens by sequential diffusion tokenizer are inherently one-dimensional, we apply the original ID RoPE (Su et al. 2024), rather than 2D RoPE, in attention layers positional embedding. The class conditions, e.g.. image labels, are injected as a single prefix token following (Sun et al. (224). We do not use AdaLN in our AR models. Classifier-free guidance logits is used during AR inference. We mainly experiment three variants of D-AR models, D-AR- (L. XI., XXL., with 343M, 775M, image, D-AR
and 1.4B parameters respectively, also following (Sun al. 2024 To generate models first produce sequence of tokens conditioned on the given label in the standard token-by- token manner with KV cache enabled. In pace with sequential generation, we can decode tokens generated into diffusion sampling steps on pixels either concurrently or offline.
5 EXPERIMENTS
Experimental Setup. We conduct D-AR experiments on the ImageNet 256 class-conditional generation benchmark (Deng et al. 2009). The sequential diffusion tokenizer is trained on the Ima- geNet training set with batch size of I024, Adam optimizer (Kingma & Ba 2015) of learning rate 2* 10 and a total of 210K iterations till convergence, together with an exponential moving aver- age with a 0.999 decay rate. The training procedure took around 5 days on 16 A100 GPUs to finish. We follow the training recipe (Pang et al. (024) to train D-AR autoregressive models with a batch size of 1024 for 300 epochs. We use Adam optimizer (Loshchilov & Hutter 2017) with learning (0.9, 0.95) and weight decay 0.05. The learning rate is decayed to 1 10- linearly within the last 50 epochs, following (Pang al. 2024). The performance of D-AR evaluated in terms of FID (Heusel et al.) 2017), Inception Score (Salimans et al. 2016). preci- sion and recall scores, following the standard ADM evaluation pipeline (Dhariwal & Nichol 202I), For the reconstruction performance of the sequential diffusion tokenizer, we mainly investigate the reconstruction FID (rFID) the ImageNet validation 50K set.
rate 4 * 10-4, 4, (B1, 52) =
5.1 RESULTS
Tokenizer results. We investigate the key component of our D-AR framework, i.c., the sequential diffusion tokenizer. [able we compare our sequential diffusion tokenizer with the conventional LlamaGen tokenizer, which has the same budget of 256 tokens and the same vector quantization configuration, as strong basclines. Despite having more parameters (300M versus 72M), which is
tokenizer
#tokens codebook size rFID.
RQ-VAE VAE (Lee et al. 2022
256
16384
3.20
Table 1: Reconstruction results on Ima-
Titok-S Yu et al. 2024h)
128
4096
1.71
geNet validation 50K samples with 256 dis-
LlamaGen Sun et al. 2024
256
4096
3.02
crete tokens. We also finetune our sequen-
llamaGen Sun et al. 2024
256
16384
2.19
tial diffusion tokenizer with smaller code-
ours
256
4096
1.84
book size, 4096, and compare with Llama-
ours
256
16384
1.58
Gen tokenizer counterpart.
7

---

Published as a conference paper at ICLR 2026
Table 2: Different sampling configurations on our sequential diffusion tokenizer, Adams 2nd refers the two step Adams Bashfooth Bashffrtssh & Adams (188), while oth- ers use Fuler.
steps
4
8
Adams 2nd
12
16
#F
2.35
1.58
1.52
1.73
1.93
mainly due the pixel diffusion decoder, our sequential diffusion tokenizer achieves better recon- struction fidelity and is more endurable to smaller codebook size.
We also study different sampling configurations of the proposed sequential diffusion tokenizer in 2 where we vary the sampling steps and flow matching ODE solver. We use Adams-Bashlowe solver for flow matching with steps as it provides clearer samples without increasing numbers of function evaluations (NFEs) on the diffusion decoder.
System-level comparison. To compare with state-of-the-art methods, we experiment with D-AR models on the ImageNet 256 x 256 class-conditional generation benchmark. Following common practice (Li et al. 2024; Pang al. 2024), the linear CFG schedule is used D-AR (1.1-8.0 for D-AR-L, 1.1- 10.0 for D-AR-XL. and 1.1->11. 1.0 L. for (-AR-XXL) In Table 3 D-AR models achieve the leading level performance their parameter count regions. Among vanilla AR models in the strict next-token-per cext-token-rredictioon manner, D-AR-XL achieves 2.09 FID with 775M parameters, outperforming LlamaGen-XXe and even competing with IBQ-XXL 2.1B. D-AR-XXL achieves a state-of-the-art FID of 2.00 ImageNet for vanilla AR models with 1.4B parameters.
Recent attempts to incorporate diffusion into autoregressive models, such as CausalFusion, DART- FM, and MAR, have also shown highly competitive results. However, they require significant mod- ifications in the autoregressive framework to tackle continuous-valued inputs and outputs of images. In contrast, D-AR maintains the vanilla autoregressive mechanism with favored performance. DDT- Llama (Pan et al. 2025a) reported its 6.1 FID on ImageNet 256 but without mentioning parameter counts (2B 8B), therefore do not compare it in the table.
Table 3: System-level comparison on class-conditional generation over 50K samples on 256 x 256 ImageNet benchmark. Note that #params in the table only counts in AR model parameters and our tokenizer with 300M parameters. MAR is difficult to categorize into mask-based or tailored AR methods.
type diffusion
method
#params
FID
IS1
Prect
Reef
DiT-XL (Peebles & Xie
2023
675M
2.27
278.2
0.83
0.57
SiT-XL Ma 2024a)
675M
2.06
270.3
0.82
0.59
mask-based
MaskGIT ( Chang et al.
2022
227M
6.18
182.1
0.80
0.51
TYTok-S- 12 Yu ct 2024b
287M
1.97
281.8
MAR-L
al.
2024
479M
1.78
296.0
0.60
MAR-H [Li ct al.
2024
943M
1.55
303.7
0.81
0.62
tailored AR
VAR-d24 (Tian al.
2024
1.0B
2.09
312.9
0.82
0.59
VAR-d30 TTin et al. 2024
2.0B
1.92
323.1
0.82
0.59
RAR-I. Yu ct
22124a
461M
1.70
299.5
0.81
0.60
RAR-XL
et al.
955M
1.50
306.9
0.80
0.62
RandAR-L. Pang ct al.
2014
343M
2.55
288.82
0.81
0.58
RandAR- (Pan
2024
775M
2.22
314.21
0.80
0.60
RandAR-XXI. (Pang al.
2024
1.4B
2.15
321.97
0.79
0.62
DART-FM (uu et al. 2025a
820M
3.82
263.8
CausalFusion-XI. (Deng
2024
676M
1.77
282.3
0.82
0.61
vanilla AR
LlamaGen-Li (Sun et al. 22044
343M
3.07
256.06
0.83
0.52
LlamaGen-XI. (Sun
2024
775M
2.62
244.08
0.80
0.57
LlamaGen-XXit (Sun et all
2024
1.4B
2.34
253.90
0.80
0.59
IBQ-XL (Shi al. 2024)
1.1B
2.14
278.99
0.83
0.56
IBQ-XXL (Sh
2024
2.1B
2.05
286.73
0.83
0.57
stronger LlamaGen-L. (Pang et al.
2024
343M
2.20
274.26
0.80
0.59
stronger LlamaGen-XX XLL (Pang et
2024
775M
2.16
282.71
0.80
0.61
D-AR-L (ours)
343M
2.44
262.97
0.78
0.61
D-AR-XL (ours)
775M
2,09
298 498.42
0.79
0.62
D-AR-XXL (ours)
1.4B
2.00
300.56
0.79
0.63
8

---

Published as a conference paper at ICLR 2026
Figure 4: Consistent previews as generation trajectories for every increment of 32 tokens (a group). Note that these previews can be generated in a streaming manner with AR tokens partially generated.
reference
'piant panda"
"polden retrievel"
"otter"
16 profic tokens 8 prefii tokens
32. prufix tokens 12 prefic tohans
Figure 5: Zero-shot layout-controlled synthesis with different prefix tokens and varying labels.
Coarse-to-fine diffusion order for AR modeling. The order linearized by our proposed sequen- tial diffusion decoder is naturally coarse-to-fine tine,,, which we argue is good for AR modeling. Here, we ablate this point by feeding the reversed token sequence into D-AR-L L modeling by the sequen- tial diffusion tokenizer, which is coresponding fine-to-coars. visual AR process, with the strict fair experiment setting. We name this reversed D-AR-XL. and the generated sequence is then reversed again and decoded by the tokenizer decoder to get final image pixels. We searched multiple CFG schedules and the best result by reversed D-AR-XL. XL is 4.17, which lags much behind the coarse-to- fine D-AR-L 2.44. More results can be found the appendix. This comparison indicates that the diffusion induced coarse-to-fir order is the key good visual autoregressive modeling, which is in line with Tian et al. (2024).
Consistent previews and generation trajectories. As discussed in Section 3.1 the sequential diffusion tokenizer can generate consistent previews of generated images when partial tokens are generated, inherited from the diffusion property to jump-estimate the target &1 = (1 t)v, + x; for every sampling timestep t. As our diffusion model is on raw pixels, this operation takes almost no extra cost. visualize these previews Figure 4 which are consistent with final samples. These previews can also be interpreted as generation trajectories of our autoregressive model and inherently follow a coarse-to-fine progression (Rissanen al. 2023).
Zero-shot layout-controlled synthesis. We also investigate the zero-shot layout-controlled syn- thesis with D-AR, where several prefix tokens are given and fixed, in Fig Thanks to the linearized structure by the diffusion decoder, we can generate plausible images with reference layouts con- ditioned on reference prefix tokens and given labels, without specific finetuning. As more prefix tokens are provided, layout control becomes stronger, while label-relevant information increasingly concentrates on fine- fine-grained details such as fur textures. We include more ablation studies and qualitative results in the appendix.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
In this paper, we present Diffusion via Autoregressive
incoptton distance on Inayeteer
vanilla generative models
models (D-AR), a framework bridge pixel diffusion and autoregressive modeling for visual generation. With linearized sequence of discrete tokens by the pre-
sented sequential diffusion tokenizer, we can perform
vanilla autoregressive process in the standard next token prediction fashion. Thus, the AR sequence generation process in the D-AR framework directly mirrors consec- utive diffusion denoising steps on pixels. Experiments the standard ImageNet benchmark shows that D-AR can generate high-quality images a vanilla autoregressive model, together with several properties from both autore- gressive and diffusion worlds.
22-
D-AR(ours)
Parame
Figure 6: Vanilla AR comparison for ImageNet generation.
ACKNOWLEDGMENTS
This project is supported by the National Research Foundation, Singapore under its NRFF Award MONNRFF13-2021-006 We would like to thank Zhan Tong, Tong He, and Shuai Wang for helpful discussions and comments.
REFERENCES
Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. abs/1607.06450, 2016.
arXiv,
Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oguzhan Fatih Kar, Elmira Amir- loo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan. Flextok: Resampling images into Id token sequences of flexible length. arXiv, abs/2502.13367, 2025.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang. Shijie Wang. Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report, 2023. URL https::/aaxiis o abs 2209.. 16609
Francis Bashforth and John Couch Adams. An Attempt Test the Theories of Capillary Action by Comparing the Theoretical and Measured Forms of Drops of Fluid. With an Explanation of the Method Integration Employed Constructing the Tables Which Give the Theoretical Forms of Such Drops. Cambridge University Press, Cambridge, 1883.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer. In CVPR, pp. 11305-1 1 1315. IEEE, 2022.
Boyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. In NeurIPS, 2024.
Yinbo Chen, Rohit Girdhar, Xiaolong Wang, Sai Saketh Rambhatla, and Ishan Misra. Diffusion autoencoders scalable image tokenizers. arXiv, abs/2501. 185593, 2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Shi Guang, and Haoqi Fan. Causal diffusion transformers for generative modeling. arXiv, abs/2412. 12095, 12095, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. doi: 10. 2..109/CVPR...2095526....
10

---

Published as a conference paper at ICLR 2026
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans image synthesis. In NeurIPS, pp. 8780-8790 8794, 2021.
Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid linear units for neural network
function approximation in reinforcement learning. Neural Nenworks, 107:3- 11733-1,,30 2018.
Patrick Esser, Robin Rombach, and Björn Ommer. Tamingg transformers for high-resolution image synthesis. In CVPR, pp. 12873-12883. Computer Vision Foundation IEEE, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In ICML. OpenReview.net, 2024.
Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song. Chen Li, Xiaohan Ding, and Ying Shan. SEED-X: multimodal models with unified multi cuulti-raauulrityy comprehension and generation. arXiv, abs/2404. 14396, 2024.
GPT-4-Team. Gpt-4 technical report, 2024.
Jiatao Gu, Yuyang Wang, Yizhe Zhang, Qihang Zhang, Dinghuai Zhang, Navdeep Jaitly, Joshua M Susskind, and Shuangfei Zhai. Denoising autoregressive transformers for scalable text-to-image generation. The Thirteenth International Conference on Learning Representations, 2025a.
Yuchao Gu, Weijia Mao, and Mike Zheng Shou. Long-context autoregressive video modeling with next-frame prediction. CoRR, abs/2503.19325. 2025b.
Alex Henry, Prudhvi Raj Dachapally, Shubham Shantaram Pawar, and Yuxuan Chen. Query-key normalization for transformers. In EMNLP (Findings), volume EMNLP 2020 of Findings of Association for Computational Linguistics, 2020.
ACL, pp.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NIPS, pp- 6626-6637, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv, abs/2207.125998 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. In /CML, volume 202 of Proceedings of Machine Learning Research, pp- 13213-13232. PMLR, 2023.
Emiel Hoogeboom, Thomas Mensink, Jonathan Heek, Kay Lamerigts, Ruiqi Gao, and Tim Sal- imans. Simpler diffusion (sid2): 1.5 FID on imagenet512 with pixel-space diffusion. arXiv, abs/2410. 19324, 2024.
Drew A Hudson, Daniel Zoran, Mateusz Malinowski, Andrew K Lampinen, Andrew Jaegle, James L. McClelland, Loic Matthey, Felix Hill, and Alexander Lerchner. Soda: Bottleneck diffusion mod- els for representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 23115-23127, 2024.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In SOSP, pp. 611-626.1666666 ACM, 2023.
Black Forest Labs. Flux. http
http://tthh..ccoooccooooooea forest -lassttlaabb/aae fflu/llxx//.. 2023.
Doyup Lee, Chiheon Kim, Sachoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In CVPR, pp. 11513-11522 IEEE, 2022.

---

Published as a conference paper at ICLR 2026
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image
generation without vector quantization. In NeurIPS, 2024.
Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Jiuxiang Gu, Bhiksha Raj, and Zhe Lin. Imagefolder: Autoregressive image generation with folded tokens. In ICLR, 2025.
Yaron Lipman, Ricky Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In ICLR. OpenReview.net, 2023.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. In ICLR. OpenReview.net, 2023.
Llama-3- Teamm. The Ilama 3 herd of models. 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi Eric Landen-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-basee generative models with scalable interpolant transformers. In ECCV (77), volume 15135 of Lecture Notes in Computer Science, pp. 23 23-40. Springer, 2024a.
Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, Liang Zhao, Yisong Wang, Jiaying Liu, and Chong Ruan. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal under- standing and generation. arXiv, abs/2411.07977, 2024b.
Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantiza- tion: VQ-VAE made simple. In /CLR. OpenReview.net, 2024.
OpenAl. consistencydecoder. httpss//hhhaaa.coo/ooo//////ssss 2023.
Kaihang Pan, Wang Lin, Zhongqi Yue, Tenglong Ao, Liyu Jia, Wei Zhao, Juncheng Li, Siliang Tang, and Hanwang Zhang. Generative multimodal pretraining with discrete diffusion timestep tokens. arXiv, abs/2504. 14666, 2025a.
Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Jue Ji Hou, and Saining Xie. Transfer between modalities with metaqueries. arXiv, abs/2504.06256, 2025b.
Ziqi Pang. Tianyuan Zhang, Fujun Luan, Yunze Man, Hao Tan, Kai Zhang, William T. Freeman, and Yu-Xiong Wang. Randar: Decoder-only autoregressive visual generation random orders. arXiv,, abs/2412.01827,, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, ppV.pp.... 4172- 4182. IEEE, 2023.
Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron C. Courville, Film: Visual reasoning with a general conditioning layer. AAAI, pp. 3942-3951 -9511. AAAI Press, 2018.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. In ICLR. OpenReview.net, 2024.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, volume 139 of Proceedings of Machine Learning Research, pp. 8821-8831. PMLR, 2021.
Sucheng Ren, Qihang Yu, Ju He, Xiaohui Shen, Alan I.. Yuille, and Liang-Chieh Chen. Beyond next-token: Next-x prediction for autoregressive visual generation. arXiv, abs/2502.20388, 2025.
Severi Rissanen, Markus Heinonen, and Amo Solin. Generative modelling with inverse heat dissi- pation. In ICLR. OpenReview.net, 2023.
12

---

Published as a conference paper at ICLR 2026
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pp. 10674-10685. IEEE, 2022.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Kyle Sargent, Kyle Hsu, Justin Johnson, Li Fei-Fei, and Jiajun Wu. Flow the mode: Mode-seeking diffusion autoencoders for tfateoo-thh--n image tokenization. arXiv, abs/2503. 1056, 2025.
Noam Shazeer. GLU variants improve transformer. arXiv, abs/2002.05202, 2020.
Fengyuan Shi, Zhuoyan Luo, Yixiao Ge, Yujiu Yang. Ying Shan, and Limin Wang. Scalable image tokenization with index backpropagation quantization. arXiv, 2024.
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-Im: Training multi-billion parameter language models using model par- allelism, 2020. htRt:s/::
org htrgg/bb/19990 08053
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR. OpenReview.net, 2021.
Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Au- toregressive model beats diffusion: Llama for scalable image generation. arXiv, abs/2406.06525, 2024.
Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. HART: efficient visual generation with hybrid autoregressive transformer. arXiv, abs/2410.10812, 2024.
Chameleon Team. Chameleon:
Mixed-modal early-fusion foundation models.
arXiv,
abs/2405.09818. 2024.
Gemini Team. Gemini: A family of highly capable multimodal models, 2025.
URL https:
//arxivv.orr/aas/21222.1
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. In NeurIPS, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation
language models, 2023. htth:t/::/// htappp:://aaxvvv.// wwg.iv.orr///oo http///bs//2......// 221//3332111111 13971
Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learn- ing. In NIPS, pp. 6306-6315, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pp.
2017.
Bohan Wang, Zhongqi Yue, Fengda Zhang, Shuo Chen, Li'an Bi, Junzhe Zhang, Xue Song, Ken- nard Yanting Chan, Jiachun Pan, Weijia Wu, Mingze Zhou, Wang Lin, Kaihang Pan, Saining Zhang, Liyu Jia, Wentao Hu, Wei Zhao, and Hanwang Zhang. Selftok: Discrete visual tokens of autoregression, by diffusion, and for reasoning. CoRR, abs/2505.072 077388 2025.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li. Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tiejun Huang, and Zhongyuan Wang. Emu3: Next-token prediction is all you need. CoRR, abs/2409. 1 18869, 2024.
13

---

Published as a conference paper at ICLR 2026
Xin Wen, Bingchen Zhao, Ismail Elezi, Jiankang Deng, and Xiaojuan Qi. "principal components" enable A new language of images. arXiv, abs/2503.082 03500300685, 2025.
Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-lo-any multi- modal Ilm. In Forty-first International Conference on Machine Learning, 2024.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. ICLR. OpenReview.net, 2025.
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved VQ-
GAN. In ICLR. OpenReview.net, 2022.
Lijun Yu, José Lezama. Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, al. Language model beats diffusion- tokenizer is key to visual generation. arXiv preprint arXiv:2310005737 2023.
Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang-Chieh Chen. Randomized autoregres- sive visual generation. arXiv, abs/2411.00% 2024a.
Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems, 37:1 187408990-1,,66,, 2024b.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv,
2024c.
Biao Zhang and Rico Sennrich. Root mean square layer normalization. In NeurlPS, pp. 12360- 12371, 2019.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In CVPR, pp. 586-595. Computer Vision Foundation IEEE Computer Society, 2018.
Long Zhao, Sanghyun Woo, Ziyu Wan, Yandong Li, Han Zhang, Boqing Gong, Hartwig Adam, Xuhui Jia, and Ting Liu. c-vae: Denoising as visual decoding. arXiv, abs/2410.04081, 2024.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng. Sglang: Efficient exccution of structured language model programs. NeurIPS, 2024.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-moda model. In ICLR. OpenReview.net, 2025.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.I LLM USAGE
We use LLM aid or polish writing, more specifically, review and revise typos and grammar in this submission. We confirm that we fully reviewed the L.M--genrrted revisions and that the final revised text authentically reflects our original expression and ideas.
A.2 DETAILED ARCHITECTURE OF SEQUENTIAL DIFFUSION TOKENIZERS
Vector guantization. We follow LLamaGen (Sun al. 2024 set up the vanilla vector quanti- zation (van den Oord et al. 2017) as well as its loss: εvo Isgifl - =112 + f sg[=|lle where
sgi-] is the stop gradient operator and 3 0.25. We do not impose the entropy loss on codebook
learning.
Transformer architecture. In our sequential diffusion tokenizer, we adopt the transformer archi- tecture with vanilla LayerNorm (Ba et al.] (2016) and SiLU activation function (Elfwing al. (0018). We also apply QK normalization (Henry et al. 2020) in attention computation for training stability. For tokens with explicit spatial locations, e.g., those patchified from images in the tokenizer encoder or in diffusion transformer, we apply the 2D RoPE (Su et al. 2024) in attention to encode spatial relations. For those who do not have 2D inherent locations, i.c., ID query tokens the transformer encoder and decoder, we simply disable rotation in RoPE by using the identity matrix.
A.3 DETAILED EVALUATION OF D-AR MODELS
Table 4: D-AR with different CFG schedules.
The value 1.0 indicates disabling CFG.
model D-AR-L
CFG schedule
FID
ISt
Prect
Recall
1.0
7.43
117.60
0.71
0.63
1.5
3.50
245.22
0.83
0.54
1.75
4.70
291.76
0.86
0.50
1.1-+8.0
2.44
262.97
0.78
0.61
D-AR-XL
1.0
5.11
145.78
0.73
0.64
1.5
3.39
276.37
0.84
0.55
1.1+110.0
2.09
298.42
0.78
0.62
CFG schedules. the main paper, we present the performance of D-AR-L and D-AR-XL with linear CFG schedule, following RandAR (Pang et al., 2024). Note that previous work also explore customized CFG schedule for better performance, as common practice on ImageNet (Li et al. 2024 Pang et al. 2024 Yu et al., 2024a). We report D-AR models results with different CFG strategies Table The models here are exactly the models the main paper in Table We do not use top-p, top-k, and temperature in our sampling in the main paper and appendix.
Table 5: D-AR-L Bit-LLjju--esttmatiinn results with partial AR tokens.
#AR tokens
64
128
192
256
#diffusion steps
2
4
8
FID4
7.38
3.94
2.93
2.44
IS
165.25
227.74
257.08
262.97
Prect
0.74
0.78
0.80
0.78
Recall 0.48
0.54
0.57
0.61
Partial AR tokens results. In the main paper, have visualized the diffusion target sample estimation with partial AR tokens generated. We here report quantitative results by D-AR-L in Table 3
15

---

Published as a conference paper at ICLR 2026
A.4 TOKENIZER ABLATIONS
Due to the limited computation resource, we design lightweight version of our proposed sequential diffusion tokenizer with 113M parameters (we change the dimension in the transformer to 512 and the depth of diffusion transformer to 8) and train for 50K iterations with 256 batch size. This ablation A100s.
training typically takes about hours complete
Table 6: Effects on tokenizer training.
2
rFID
39.75
28.65
27.10
codebook utilization
97.8
99.4
99.8
Ablations on / in conditioning schedules. The control parameter, 8. in the conditioning sched- ule, also acts as timeshift parameter in the diffuuson procedure in our sequential diffusion tokenizer. Since we are operating on pixels, we set to as default. Here we investigate different /3 on tokenizer training in Table 6 We see that there is large gap between = and 3 2 in construction FID as well as in coodebook utilization, while 3 and / = matches closer. These empirical results show that early diffusion need denser steps well AR tokens conditioning on diffusion in the pixel space.
K-1
K-B
K
rFID
gFID4
AR 7-
K-16
8.37
34.85
9.66
34.54
8
14.18
35.69
16
22.58
41.20
Figure 7: Effects of the group K tokenizer and AR training. For fair comparison, for K < variants, we use diffusion sampling steps to decode images. rFID refers to sequential diffusion tokenizer reconstruction FID and gFID refers to D-AR-B generation FID 100K iterations.
The numbers of conditioning group K. In this ablation, we stretch the sequential diffusion tok- enizer training to longer 100K iterations. The number of conditioning group K decides how many tokens feed into pixel diffusion model per diffusion step, N/K. We investigate the effects of K in the Table 7 The sequential diffusion tokenizer with single group K = with multiple sampling steps degrades into conventional tokenizers with diffusion decoder (Sargent et al.) 2025 Chen al., 2025). which denoises an image with full token sequence on every timestep. This K = setup does not yield linearized ordering of visual tokens and lacks the sequential nature central our approach.
For reconstruction FID here, is reasonable and expected for small group number variants to per- form better, since the number of conditioning tokens per denoising step become more K de- creases, therefore reducing bottleneck. In the other side, linearized order most strongly, but came out with the worst reconstruction FID.
16 enforces the diffusion-induced
We also train D-AR-B with 111M parameters for 100K iterations with batch size of 1024 with tokens by these tokenizers. In the training loss curve in Figure 7 we can find that the large group number facilitates AR training, which we believe the stronger coarse-to-fine order more well- suited for autoregressive modeling. Interestingly, although the reconstruction FID with K = 8 falls behind = 4 and variants, the generation FID achieved by D-AR-B models remains comparable. This suggests that more strongly linearized token sequences (higher K) can be better for autoregres- sive generation modeling even if they degrade reconstruction. For this reason, we adopt K = 8 as default: it offers moderate trade-off between reconstruction fidelity and a linearized structure
16

---

Published as a conference paper at ICLR 2026
that benefits AR modeling, and as demonstrated in main experiments, our K sequential
diffusion tokenizer can eventually achieve competitive reconstruction performance.
Table 7: reversed D-AR-L with different CFG schedules.
The value 1.0 indicates disabling CFG.
model
CFG schedule
FIDJ
1st
Prect
Recallf
D-AR-L
1.0
7.43
117.60
0.71
0.63
1.5
3.50
245.22
0.83
0.54
1.75
4.70
291.76
0.86
0.50
1.1-+8.0
2.44
262.97
0.78
0.61
reversed D-AR-L
1.0
11.22
96.23
0.68
0.62
1.5
4.17
238.05
0.84
0.50
1.75
5.83
292.94
0.88
0.44
1.1-88.0
9.79
417.78
0.90
0.40
8.0-+1.1
21.15
320.15
0.85
0.15
A.5 REVERSED D-AR
We here include more results by the reversed D-AR-L with different CFG configurations in Table Note that the reversed D-AR-L training strictly follows the normal D-AR-L training setting, except for input token ordering. For the same CFG setting, the reversed D-AR-L falls behind the normal D-AR-L by a large margin.
A.6 MORE VISUALIZATIONS
Tokenizer reconstruction results. We also present reconstruction samples from our sequential diffusion tokenizer (rFID 1.52) in Fig 9 As observed, fine details not strictly reconstructed, which is mainly attributed to the inherent stochastic and denoising nature of the diffusion process. Since primary objective is to model image generation rather than achieve exact pixel-level construction, this trade-off is acceptable and consistent with our diffusion tokenizer design.
Generation trajectories. We show more generation trajectories well as previews in Fig 10 Our D-AR models follow coarse-to-fine generation with consistent previews with final targets.
Zero-shot layout-controlled synthesis. As discussed in the main paper, we can simply condi- tion prefix tokens generate layout-following images in zero-shot manner. We here show more zero-shot layout-controlled generated samples by fixing different numbers of prefix tokens and varying labels in Fig
17

---

Published as a conference paper at ICLR 2026
Figure 8: Uncurated generated samples by D-AR-XL with random labels and CFG=4.0.
18

---

Published as a conference paper at ICLR 2026
Figure 9: Reconstruction results with samples from the ImageNet validation set. Each pair of rows shows:
first row input; second row reconstruction.
19

---

Published as a conference paper at ICLR 2026
Figure Generation trajectory and previews at each diffusion sampling step by D-AR-L.
"solden
16 prefii tokeess 8 prefis, tokerss
12 prefix
32 prefiix
Figure 11: Zero-shot layout-controlled synthesis.
20


Reviews:
[{"review_id": "lUu2aYptUX", "reviewer": "Reviewer_XBux", "summary": "The paper introduces D-AR (Diffusion via Autoregressive models), a new framework that reinterprets the image diffusion process as a standard autoregressive next-token prediction task by using a specially designed sequential diffusion tokenizer that maps coarse-to-fine diffusion steps into a linear sequence of discrete tokens. This approach enables high-quality image generation with vanilla autoregressive transformers (e.g., Llama backbones), achieving state-of-the-art FID scores (2.00 with 1.4B parameters) on ImageNet while supporting streaming previews and zero-shot layout control—without modifying core AR mechanisms.", "questions": "see above", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) Its sequential diffusion tokenizer naturally imposes a coarse-to-fine token ordering aligned with diffusion denoising steps, which is highly suitable for autoregressive modeling. \n(2)  The framework supports streaming, consistent previews during generation at no extra cost by leveraging diffusion’s ability to jump-estimate final images from partial token sequences. \n(3) It enables zero-shot layout-controlled synthesis by fixing prefix tokens, all without finetuning. \n(4) D-AR benefits from the efficiency of AR inference (e.g., KV caching) and operates directly on raw pixels without VAEs, simplifying the pipeline while maintaining compatibility with existing LLM infrastructure.", "weaknesses": "(1) I find D-AR to be a very interesting framework that combines two well-established paradigms. However, the motivation for introducing this new hybrid paradigm is unclear—current diffusion-based or purely autoregressive (AR) approaches already perform very well in image generation. Why is this fused paradigm necessary? What specific problem does it solve?\n(2) D-AR cannot be easily integrated with state-of-the-art step distillation methods, such as one-step image generation. If step distillation is applied to reduce the number of sampling steps to just 1 or 2, what is the purpose or benefit of the autoregressive component in D-AR?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Tdq6KVG6tC", "reviewer": "Reviewer_MQZQ", "summary": "The authors propose D-AR, which reformulates the pixel diffusion process as a vanilla autoregressive procedure. They introduce a diffusion tokenizer that organizes discrete image tokens in a coarse-to-fine order. A conventional next-token prediction framework is then applied to sequentially generate these tokens, which are subsequently grouped and fed into the diffusion detokenizer as conditions to decode images.", "questions": "Please refer to the Weakness section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The idea to project diffusion process into a vanilla autoregressive procedure is interesting.\n2. The paper is well-written and easy to follow.", "weaknesses": "1. Although the idea is interesting, I am not entirely convinced that this approach truly demonstrates a synergistic effect between autoregressive modeling and diffusion. The autoregressive component still relies on discrete tokens, which are then used as conditions for the diffusion model. Consequently, the information loss inherent in the discrete tokens is propagated into the diffusion process. This is evidenced by the higher rFID of the diffusion tokenizer compared to its continuous VAE counterpart.\n2. The results in Table 3 are not a fair comparison with other methods in terms of parameter count. The diffusion tokenizer itself contains an additional 300M parameters, whereas the tokenizers in other approaches (whether continuous or discrete) are much smaller. For instance, D-AR-XL actually has around 1.1B parameters, yet achieves an FID of 2.09, which is worse than the approximately 700M SiT-XL diffusion model with an FID of 2.06. This raises concerns about the advantage of D-AR compared with a pure diffusion model under the same parameter budget.\n3. The authors claim that an intriguing property of their model is the ability to preview the diffusion generation process. However, this is not unique to D-AR, but rather a general characteristic of diffusion models in general.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YSn4iOrPfE", "reviewer": "Reviewer_q2wL", "summary": "This paper introduces D-AR, a framework bridging pixel-level diffusion and autoregressive modeling for image generation. The core idea is to transform 2D images into 1D discrete token sequences using a sequential diffusion tokenizer, where tokens are ordered in a coarse-to-fine manner corresponding to diffusion steps. A standard decoder-only Transformer is then used to perform autoregressive next-token prediction on these tokens. Sequential token generation directly mirrors diffusion denoising steps on pixels, enabling KV cache-friendly inference, streaming pixel decoding with consistent previews, and zero-shot layout-controlled synthesis.", "questions": "Have you compared the differences in reasoning speed or generation throughput between D-AR and other  AR methods ?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Novel framework: Integrates diffusion and autoregressive modeling, providing a unified approach that preserves the strengths of both paradigms.\n2. Sequential diffusion tokenizer: Generates coarse-to-fine token sequences corresponding to diffusion steps, naturally suited for AR modeling.\n3. Preserves vanilla AR architecture: Works with standard decoder-only Transformers without modifications to causal masks, attention, or training schemes. Supports streaming pixel decoding, consistent previews, and zero-shot layout-controlled synthesis.", "weaknesses": "1. Dataset and task limitation: Experiments are only on ImageNet 256×256 class-conditional generation. High-resolution images, other datasets (COCO, LSUN, FFHQ), or tasks (image repair, image edit) are not tested.\n2. Model complexity and inference speed: The sequential diffusion tokenizer adds 300M parameters, and AR generation requires predicting 256 tokens sequentially. Although KV caching and token grouping mitigate some overhead, more tokens still increase generation latency compared to smaller token setups like Titok-S (128 tokens), which may limit real-time applications. It would be interesting to know whether the authors have tried using Titok-S or a smaller token budget within the D-AR framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bGASpUmm43", "reviewer": "Reviewer_iSt8", "summary": "This paper introduces \"D-AR\" (Diffusion via Autoregressive), a novel and elegant framework that reframes the image diffusion process as a standard autoregressive (AR) next-token prediction task.\n\nThe core of this work lies in a \"Sequential Diffusion Tokenizer.\" Unlike traditional tokenizers that linearize an image spatially (e.g., raster-scan), this tokenizer encodes an image into a 1D sequence of **discrete tokens** (e.g., 256 tokens) that are ordered according to the **diffusion process itself** (coarse-to-fine).\n\nThe generation process is as follows:\n\n1. A standard AR Transformer (e.g., Llama) uses standard next-token prediction to generate this discrete token sequence.\n\n2. The tokenizer's decoder (a pixel-space Diffusion Transformer) uses these AR-generated tokens as the condition for its own denoising steps.\n\n3. Critically, groups of tokens in the sequence correspond to specific diffusion steps. For example, the first 1-32 AR-generated tokens are used to condition step 1 of the diffusion; tokens 33-64 are added to condition step 2, and so on.\n\nThe authors claim SOTA results (2.00 FID) in the \"vanilla AR models\" category on the ImageNet 256x256 benchmark. The framework also naturally supports \"streaming previews\" (e.g., generating a 25% rough image after 25% of tokens are generated) and \"zero-shot layout control\" by fixing the initial (coarse-grained) prefix tokens.", "questions": "To strengthen the paper, it is highly recommended that the authors address the following key questions with experiments:\n\n**Question 1: Can a direct inference latency benchmark (e.g., sec/image) be provided?**\n\nThe paper claims \"KV cache-friendly\" inference but seems to have an inherent dual-latency challenge (N AR steps plus K Diffusion steps). How does the true wall-clock inference speed of D-AR (N=256, K=8) compare to (1) a standard 8-step DiT and (2) a standard VQ-GAN AR model (N=256)?\n\n**Question 2: How is this coarse-to-fine tokenization paradigm expected to scale to high-resolution (1024x1024) T2I tasks?**\n\nThe model was only tested on ImageNet 256. Does a 256-token sequence have sufficient bandwidth to capture both the semantics of a complex text prompt and the fine details of a 1024x1024 image? Or would this require a much longer token sequence (e.g., N=1024), further exacerbating the AR latency bottleneck?\n\n**Question 3: The ablation on K (number of groups) in Figure 7 seems counter-intuitive. Can this tradeoff be clarified?**\n\nFigure 7 shows that K=16 (strongest linearization) gives the best AR training loss, yet the worst rFID and gFID. This seems to contradict the argument that a stronger coarse-to-fine order is better for AR modeling. Conversely, K=1 (no linearization) has poor AR loss but better rFID. Please clarify this critical tradeoff.\n\n**Question 4: What is the distribution of \"work\" between the 1.4B AR model and the 300M Tokenizer?**\n\nThe 185M Diffusion decoder in the tokenizer is a powerful generator on its own. How much does the final image quality depend on the AR model's accuracy? For example, what is the FID if ground-truth tokens are provided for the first 50% of the sequence (z_1...z_128) and the AR model only generates the last 50% (z_129...z_256)?\n\nFurthermore: If the AR model only generates the first 50% of tokens (z_1...z_128), but the DiT decoder uses only these tokens as the condition for the entire K=8 diffusion process (i.e., no new tokens are provided for t>0.5) while t progresses normally, can the model still produce an acceptable image? This would help clarify how much critical information is in the last 50% of tokens.\n\n**Question 5: Is the \"streaming preview\" advantage overstated compared to standard diffusion?**\n\nThe paper claims its preview is a key advantage, but this is debatable. Standard diffusion models (pixel or latent) also produce coarse outlines from noise in early steps (e.g., step 4 of 20). The unique strength seems to be the link between \"AR sequence progress\" and \"image clarity,\" rather than the preview capability itself.\n\n**Question 6: Does fine-tuning this way waste the pre-trained LLM's capabilities?**\n\nA selling point is the use of a standard Llama architecture. However, the model is fully fine-tuned in Stage 2 to predict visual tokens. To what extent does this process \"destroy\" the LLM's original, powerful language understanding and reasoning abilities? If the linguistic capabilities are lost, what is the true advantage of using a pre-trained Llama over a Transformer trained from scratch? Would a unified multimodal model (e.g., training on a mix of text and image tokens) be a superior path?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Conceptual Elegance and Novelty**: The main strength is conceptual. Reframing the entire multi-step diffusion process into a single standard AR sequence prediction problem is a highly elegant and novel \"unification\" of the two dominant generative paradigms (AR and Diffusion).\n\n**Coarse-to-Fine Tokenization**: This is a key insight: linearizing an image based on \"diffusion steps\" (temporal, coarse-to-fine) rather than \"space\" (raster-scan). The ablation study in Section 5.1 (2.44 FID for forward order vs. 4.17 FID for reverse order) provides strong evidence for this design choice.\n\n**SOTA Performance in \"Vanilla AR\" Category**: The paper achieves excellent results (2.00 FID with a 1.4B model, Table 3) within its defined category (\"vanilla AR models\"). It clearly and significantly outperforms the previous SOTA in this category, LlamaGen (2.34 FID).\n\n**Zero-Shot Controllability**: The demonstration of zero-shot layout control by simply fixing prefix tokens (which correspond to early, coarse diffusion steps, Fig 5) is a powerful emergent property that validates the design's self-consistency.", "weaknesses": "**Potentially Misleading \"SOTA\" Classification**: The paper's headline result (2.00 FID) is only SOTA when compared to \"vanilla AR models.\" As Table 3 shows, this performance still lags significantly behind \"tailored AR models\" (e.g., RAR-XL @ 1.50 FID), \"mask-based models\" (e.g., MAR-H @ 1.55 FID), and hybrid models (e.g., CausalFusion-XL @ 1.77 FID). Claims of \"leading performance\" should be more precise.\n\n**Hidden Tokenizer Complexity**: The framing of the AR model as \"vanilla\" is debatable, as it outsources enormous complexity to the tokenizer, which is far from a simple VQ-GAN. The tokenizer is a 300M parameter model containing an encoder, a VQ, and a 185M parameter pixel-space Diffusion Transformer (DiT) decoder. This tokenizer is, in itself, a powerful generative model.\n\n**Unaddressed Latency & Scalability Concerns**: The framework presents a dual-latency bottleneck (N-step AR generation + K-step Diffusion decoding) and was only tested on low-resolution ImageNet. Furthermore, the reliance on pixel-space operations (to enable previews) suggests a fundamental scalability challenge for high-resolution or T2I tasks, which is not discussed.\n\n**Limited Evaluation Scope**: The entire paper is benchmarked only on ImageNet 256x256 class-conditional generation. This is a very constrained dataset that fails to test the model's ability to handle complex, open-ended text prompts (T2I) or scale to high resolutions (1024x1024), which are standard for SOTA diffusion models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)