From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation
Authors: Cheng Cheng
Lin Song
Di An
Yicheng Xiao
Xuchong Zhang
Hongbin Sun
Ying Shan
Venue: ICLR 2026 Poster (2026)
Keywords: Auto-Regressive Image Generation
Discrete Diffusion
Primary area: generative models

Abstract: Autoregressive (AR) models have emerged as a powerful framework for image generation, yet they remain bound by a fundamental limitation: once a prediction is made, it cannot be revised. Each step marches forward in a strict left-to-right sequence, causing small errors to accumulate and compromise the final image. In this work, we reimagine this process with TensorAR, a decoder-only AR model that shifts from predicting discrete tokens to predicting overlapping tensor windows. This simple change transforms image synthesis into a process of next-tensor prediction, enabling the model to refine earlier outputs while preserving the causal structure that defines autoregression. To guard against information leakage during training, we introduce a discrete tensor noising mechanism inspired by discrete diffusion theory, which injects categorical noise into input tensors. TensorAR is designed to be plug-and-play: unlike masked AR methods, it requires no architectural modifications, and unlike autoregressive diffusion, it preserves the familiar AR training paradigm. We evaluate TensorAR across both class-to-image and text-to-image tasks, showing consistent gains in generation quality and instruction-following ability, while achieving a superior balance between quality and latency. In doing so, TensorAR offers a new path forward for autoregressive generation---one where predictions are not just produced, but continually refined.

Published as a conference paper at ICLR 2026
FROM PREDICTION TO PERFECTION: INTRODUCING REFINEMENT TO AUTOREGRESSIVE IMAGE GENERATION
Cheng Chengl.4," Lin Song*.* Di An²
Yicheng Xiao
Xuchong Zhangl. Hongbin Sun? Ying Shan
1 State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics,
Xi' an Jiaotong University
2 Johns Hopkins University
Tsinghua University
ARC Lab, Tencent PCG cheng201 60stu. xjtu.edu.co
ABSTRACT
Autoregressive (AR) models have emerged powerful framework for image generation, yet they remain bound by a fundamental limitation: once a prediction is made, it cannot be revised. Each step marches forward in strict left-to-right sequence, causing small errors to accumulate and compromise the final image. In this work, we reimagine this process with TensorAR, decoder-only AR model that shifts from predicting discrete tokens to predicting overlapping tensors, which are essentially several adjacent discrete image tokens. This simple change trans- forms image synthesis into process of next-tensor prediction, enabling the model refine earlier outputs while preserving the causal structure that defines autore- gression. To guard against information leakage during training, we introduce a discrete tensor noising mechanism inspired by discrete diffusion theory, which injects categorical noise into input tensors. TensorAR is designed to be plug-and- play: unlike masked AR methods, requires no architectural modifications, and unlike autoregressive diffusion, it preserves the familiar AR training paradigm. We evaluate TensorAR across both class-to-image and text-to-image tasks, show- ing consistent gains in generation quality and instruction-followe achieving a superior balance between quality and latency. In doing so, TensorAR offers new path forward for autoregressive generation-one where predictions are not just produced, but continually refined.
ability, while
INTRODUCTION
Building on the exceptional success of autoregressive (AR) models natural language processing. attributable to their scalability, flexibility, and capacity to capture complex sequential dependencies, researchers have extended AR approaches to conditional image generation and to unified under-
standing and generation frameworks (Pang et al. 20244 Yu et al. 2024 Sun et al. 2024 Luo al.
2024 Yu al. 2023 Tian et al. 2024 Li et al. 2024a] Esser et al. 2021 Lee et al. 2022 Xiao et al., 2025b a). At their core, AR models rely simple yet effective self-supervised objective: predict- ing the next token in a sequence. Compared with other generation paradigms (e.g., flow-matching
models). AR models enable structured, step-by-step synthesis and offer advantages in controllability and multimodal integration (Wu et al. 2024
Team 2024).
For image generation tasks, standard AR models (Pang al. 2024 Yu et al. 2024 Sun al.) 2024)
typically serialize images by treating each image patch as a discrete token and modeling dependen- cies in predefined order (e.g., raster scan). This paradigm forces prediction in counter-intuitive sequence order that disrupts spatial continuity; early tokens are often blurry, which can degrade
"quual contribution.
I "oorrsponniig Author: zhangxc0:29@xjun.cmo

---

Published as a conference paper at ICLR 2026
overall quality. To improve AR generation quality, variety of approaches have been proposed,
including combining AR with continuous diffusion Gu et al. 2024; Deng et al. 2024). model-
ing per-token probability distributions (Li et al. 2024a Fan et 2024). and exploring alternative
generation paradigms (Tian et al., 2024 Ren et al. 2025). For example, MAR (Li et al. 2024a)
models per-token probability distributions via diffusion procedure, enabling AR models to operate in continuous space and eliminating the need for discrete tokenizers. DART (Gu et al. (2024) uni- fies autoregression and diffusion within a non-Markovian framework, iteratively denoising image patches across spatial and spectral dimensions using an AR model with standard language-model architecture. VAR (Tian al., 2024) adopts next-scale prediction framework that emulates hu- man sketching through coarse-to coars-tto-fing fine,, 2D-parallel generation. Despite strong results, these methods typically require additional VQ-VAE training or modification training objective (from classifi- cation to regression), which increases computational and memory costs and may hinder multimodal integration. Parallel to these existing works, motivated by the coarse-to- principle that underpins diffusion and flow-mater matching models, ask: Can existing standard AR models be enabled to refine their own predictions without modifying their architecture or training recipe?
discrete toker
contirupus lateett
Vanilla AR
Masked AR
Masked AR
Vanilla AR
(discrate Tokens)
Training Paradign
Classficction
Classifiestion / Regnession
Reynession
Classification
Attention Type
Couscl
Bidrectiond
Biclnectiond
Cousal
Refine
Not Supported
Net Supperted
Supported
Supported
Model Type
Stowdard
Mosted AR.
Conticauus
Divenette
c.e. LlamaGiaN:
e.e., Masktill;
Diffistion e.g. DARR:
Liffiuionn TensorAR;
Figure 1: Comparison with different AR-based methods. (a) Vanilla AR models that directly per- form next-token-prediction (b) Masked AR models that predict masked tokens given clean tokens; (c) Integration with diffusion models that utilize the continuous output latent of AR models as the condition to an additional diffusion generation head; (d) The proposed TensorAR that does not mod- ify the base architecture and classification-bar training paradigm.
In this paper, we introduce TensorAR, a coarse- to-fine autoregressive image generation frame- work that reframes the conventional next- token prediction paradigm as "next-tensor- prediction". The core idea behind TensorAR is simple. Unlike standard AR models that gen- erate one token at time, TensorAR predicts a tensor, i.e., group of consecutive tokens, at each step, which is the origin of the name, ie., TensorAR. Because adjacent tensors over- lap, later predictions can revise earlier ones, enabling iterative refinement of image content similar diffusion models. For clarity, we pro- vide a visual comparison in Figure [ Unlike masked AR models, TensorAR does not require architectural modifications, and unlike autore- gressive diffusion models, does not alter the training paradigm.
6800
0.8
TemsocAR
0.54
LammaGEN
4.8
Janus-Pro-78
4.3
0.52
GenEval 100000
0.82
3.8
Consisient
improvement
0.79
2.8
0.74
233
1.K
0.76
200
400
600
800
1000
1200
1400
1600
Model (M)
Figure 2: Model size-FID curves on TensorAR across different tasks. TensorAR achieves sistent improvements on both class-to-image and text-to-image generation tasks. Best view in color.
However, training TensorAR is nontrivial. A naive strategy would mimic standard AR train- ing by feeding a sequence of ground-truth ten- sors and supervising the prediction of next-step tensors. Nevertheless, because tensors are generated in sliding-wind window fashion, some tokens in the predicted tensor already appear in the input tensors, causing information leakage, where the model can minimize loss by copying overlapping tokens rather than learning meaningful causal dependencies. To address this, we introduce a discrete ten- noising mechanism based discrete diffusion theory, which injects categorical noise into input
2

---

Published as a conference paper at ICLR 2026
tensors during training. By modulating noise levels token-wise within each tensor, stimulate an internal progressive denoising process TensorAR. In addition, we incorporate two lightweight modules, ie., an imput encoder and output decoder, to interface with tensor-based inputs and out- puts. Both modules use the residual design to better leverage pretrained models and promote faster, more stable convergence. Together, these components make TensorAR plug-and-play extension that integrates with existing AR models with minimal changes to the base architecture, improving practical flexibility relative to training from scratch. We evaluate TensorAR on representative AR models for conditiont (e.g., LlamaGen (Sun et al. (024 and text-conditional (e.g., Janus- Pro-7B) image generation across multiple model sizes. We conduct extensive experiments across range of base models and model sizes and comprehensive ablation studies, consistent performance gains on both tasks (Figure 2 confirm the effectiveness of the refinement mechanism and show better trade-off between quality and latency.
2 TENSORAR
In this section, we first revisit the details about autoregressive modeling and discrete diffusion in 2.1 and then provide detailed explanations of our proposed method in 2.2
Xirs
New Modile
Lincar
Bade Meddie
Qust
Tinage Tolon
Output Decaden
Pod Teken
h
Boso Model
(1) Ouipue
Trout Encoder
Q.+
Erboddingg
x
26
(3) TetsorAR inference porodigne
(6) Inguz Bncede:
Sketch orQ, and Qoax
Figure 3: (a) Overview of our proposed TensorAR framework during inference time with the window size and the sequence length T: (b) Output decoder that wraps original linear output layer with residual design; (c) Input encoder that wraps the original embedding layer with residual design; (d) Sketch Qin and Qouts which can be implemented by query transformers. The newly introduced modules are colored in orange and the base modules are in purple.
2.1 PRELIMINARIES
In the following paragraph, we denote sequence of discrete tokens; denotes one discrete token; x denotes the one-hot version of x;; x² denotes the noisy token of x.
2.1.1 AUTOREGRESSIVE IMAGE GENERATION
Given sequence of discrete tokens x [x1 F2, Frl of length T and its condition c, where z; (0, 1 1.... C' C' 1) is an integer from vocabulary of size C, an autoregressive model Ço are trained to model the probability distribution of each variable s, based on on its precedents (2,,22, ..., 21-1]:
(x(x;c) c) 11 (o(xe(21,... c), where may be either class labels or textual prompts, and Ço
the token distribution predictor with model parameterized by 0.
3

---

Published as a conference paper at ICLR 2026
To apply autoregressive modeling to 2D images, images are first tokenized into several discrete tokens via a pre-defined order, where each discrete token corresponds to an image patch. Given Pdaaa the distribution of discrete image data, the training objective of autoregressive models is to minimize the negative log-likelihood loss, which is formulated as:
T
[()) Exyr Pra
log Ca xc)
(I)
2.1.2 DISCRETE DIFFUSION
Discrete diffusion models (Sohl-Dickstein al. 2015 Hoogeboom et al. 2021:
Austin et al. 2021
are a class latent variable models characterized by a forward noising process and a learned reverse denoising process. The forward process ( = II ((æ, (x;-1) corrupts the original data zy into a sequence of increasingly noisy latent variables #1:7- The backward process learns grad- ually denoise the latent variables of the data distribution as pe (To.r) = p(xr)) II Po (0
According to existing studies (Zengg et al. 2023), by defining both the forward and backward dis- tribution as categorical distribution, i.c., g x; = Cat(z;;; Q,æ;-1), where Cap( is categorical distribution over the one-hot vector with probabilities given by the vector p and Q the time-dependent transition matrix, the forward process posterior g( zg) and the op- timization objectives can be calculated analytically, which is simply as weighted cross-entropy
loss.
(6) Exg Paan: tw-y(f), ar ar-an(a [..(]] wt log (o 2x. f)
where Pdata is the true data distribution, & is the noise timestep calculated by the scheduling function 7-), We the weighting coefficient.
2.2
TENSORAR
2.2.1 OVERALL FRAMEWORK
TensorAR serves as a plug-and-ple module compatible with existing transformer- based autore- gressive models. Unlike standard AR models that operate sequences of tokens, TensorAR operates on sequences of tensors. To this end, TensorAR rearranges the sequence of tokens [r, into the sequence of overlapping tensors X₂ [x1,,, X2,6 XT,K), where Xi,k = [r) Ti+1. -i-k-1] [] is single tensor with k being its the window size. It is worth noting that an additional padding token 7 is added in the last few tensors of X2, as shown in Figure 3 During training. we ignore the loss on these padding tokens, while during inference, these padding tokens do contribute to the final results. By reformulating the original Markov process over token sequence into Markov process over tensor sequence, TensorAR adopts the next-tensor generation paradigm, which can expressed as:
T
(x c)
II
pa (x( X LMiie) Xi = [2 Ti4h I++k-I]
(3)
2.2.2 REFINEMENT MECHANISM
The major advantage of TensorAR is its ability to refine previously generated tokens, a capability that standard autoregressive models lack. Consider a predicted tensor X, within this tensor, the first token is the most refined, having undergone & refinement steps, whereas the last token æi has been produced only once. Consequently, the corresponding image patch is expected to exhibit finer-grained details as the number of refinement steps increases. Intuitively, TensorAR decodes image patches iteratively in a coarse-to-fine manner, whereas standard AR methods generate each patch in single pass. This paradigm enables TensorAR to more effectively exploit future context to refine earlier content, resulting in higher generation quality.
As shown in Figure (d), to accommodate tensor-based inputs and outputs, TensorAR introduces an input encoder Mene and an output decoder Mase that wrap the original embedding and linear output layers, respectively. The input encoder compresses several token embeddings into single hidden
4

---

Published as a conference paper at ICLR 2026
state, while the output decoder reconstructs several consecutive tokens from single hidden state. Specifically, compression and decompression are performed by two additional modules, Qin and Qont, respectively. These modules share similar architecture and can be implemented with query transformers, which contain an attention module with several cross-attention layers and one output MI.P module. Moreover, to better leverage pretrained models and to facilitate stable convergence during early training, we incorporate a residual mechanism into both Mene and Maec-
2.2.3 NOISE MECHANISM
As shown in Figure 3 (a), considering the overlapping tokens during training, directly applying autoregressive models to tensor sequences encounters the information leakage problem, as some tokens in the predicted tensor already appear the input tensor. This causes the model to collapse into simply replicating the overlapping tokens, rather than learning meaningful dependencies.
To address this issue, inspired by discrete diffusion theory, we propose the discrete tensor noising scheme, which adds noise to the input tensors during training. Let us begin with simple case with tensor (I,, ..... x++)-1) where the superscript represents noisy tokens. During training time, the ideal output will be a tensor of clean tokens (I:+11- ..., Tzt+s). Therefore, for the overlapping tokens, TensorAR serves the denoiser that reconstructs clean tokens from noisy ones. We provide details about the noise mechanism in the following paragraph.
Given tensor Xt, ["t,... ... It+k-1] and the vocabulary size V, we define the discrete diffusion process to each token except the first one using a categorical distribution that has a (j) probability of resampling category uniformly:
((#4,]124)5) j) Cat(ai+,/(1 -()))=+++ + B()/V), E [2...., k 1].
(4)
where z is the noisy token and Cat represents the categorical distribution. Besides, the noise weight B(j) is monotonically increased from 0 to 1 within each tensor, i.e., for [2...., k 1].
We design a series of scheduling functions B()) as shown in Table [I to control how the input and noise tokens are fused. These noise scheduling functions include linear, sine, square root, and exponential forms. By modulating the noise intensity across different tokens within a tensor, simulate progressive denoising process in autore- gressive model training, akin to that in diffusion models. Furthermore, as shown in Figure 3 it is worth noting that we utilize an additional padding token TA, and we ignore the loss calculation at the position of the padding token.
Table I: Noise scheduling functions.
Function
Expression
Linear Sine
8()) jík B()) sin s (Tj/2k)
Square root
B() = v /
Exponential
B()) = j
By combining Equation T and Equation 2 the overall training objective of TensorAR can be formu- lated as follows:
(())=
Egg prpaa poe ( (311))) my (og(po(sa))
(5)
Due to the page limit, we provide the pseudo-code of TensorAR during training in the appendix.
2.3 RELATION TO OTHER IMAGE GENERATION PARADIGMS
Compared with diffusion models, TensorAR models and trains on image patches in an autoregres- sive manner, naturally aligning with the discrete sequence modeling paradigm and causal masking used by multimodal large language models. This design enables seamless integration with standard Transformer backbones. Besides, unlike classical diffusion methods that update the entire image at every step, TensorAR updates only the local region covered by the sliding window, preserving iter- ative refinement while enabling online generation and better scalability. Moreover, unlike standard autoregressive models that generate each patch only once, TensorAR can iteratively refine previously generated patches while producing subsequent content, improving both efficiency and overall visual quality and consistency. In particular. when k = TensorAR reduces to a standard autoregressive model; when l equals the total number of image patches T, TensorAR becomes equivalent to a dis- crete variant of a diffusion process (with different generation order, i.e, left-to-right in TensorAR
5

---

Published as a conference paper at ICLR 2026
and random in standard discrete diffusion). During decoding, TensorAR can simultaneously attend to conditions and forthcoming visual information to enforce consistency on earlier content and to complete fine details. Besides, considering the slow inference speed of AR models, especially for large context length, several distillation methods (Liu et al.) 2024a| 2025) have been proposed to accelerate the decoding process of AR models with acceptable performance degradation. It will be interesting and promising to integrate these distillation methods and TensorAR to achieve further flexibility in the trade-off between sample quality and sampling speed.
In summary, TensorAR bridges autoregressive and diffusion paradigms, offering a flexible refine- ment mechanism and controllable compute-quee quality trade-off: & provides minimal-latency autoregressive decoding, & T approximates discrete diffusion- multi-step denoising pro- cess, and intermediate settings 1 < T balance efficiency and quality by exploiting future information to iteratively improve previously generated content.
EXPERIMENTS
3.1 EVALUATION ON CLASS-TO-IMAGE GENERATION TASK
We use Fréchet Inception Distance (FID) (Heusel et al. 2017) as our primary metric; we also report Inception Score (IS) (Salimans et al. 2016), Precision and Recall (Kynkäänniemí et al. 2019).
Table 2: Model comparisons on class-conditional ImageNet 256 x 256 benchmark. Metrics are Fréchet inception distance (FID), inception score (IS), precision, and recall. "4" or "1" indicate lower or higher values are better.
Type
Model
#Para.
FID
ISt
Precisiont
Recall
MAGVIT-v2 Yu al.
20133
307M
1.78
319.4
Mask AR
MaskBit Weber ul.
2024
305M
1.52
328.6
MAR Li 2024a
943M
1.55
303.7
DART Gu et al. 2024
812M
3.98
256.8
RQTran. (Lec el al. 2022
3.8B
3,80
323.7
ViT-VQGAAN-e Yu al.
2021
1.7B
3.04
227.4
Casual AR
SAR-XL (Lu
2024b
893M
2.76
273.8
0.84
0.55
RandAR- [Pung 2024
1.4B
2.15
322.0
0.79
0.62
VAR Tian al.
2.0B
1.73
350.2
0.82
0.60
Open Meen-MAGVIT2 (Luo et al.
2024
Mnen-MAGVIT2-B (256 256)
343M
3.08
258.3
0.85
0.51
+TensorAR
352M (+2.7%)
2.91
260.2
0.86
0.50
Open-MAGVIT2-LL (256 x 256)
804M
2.51
271.7
0.84
0.54
+TensorAR
820M (+2.0%)
2.35
273.4
0.84
0.53
LlamaGEN (un
2024
LlamaGEN-B B (256 x 256)
111M
5.46
193.6
0.83
0.45
TensorAR
+ensorAR
116M (+4.6%)
4.71
225.8
0.85
0.45
LlamaGHN--L (256 x 256)
343M
3.80
248.3
0.83
0.52
+TensorAR
352M (+2.7%)
2.78
254.8
0.82
0.56
LlamaGEN-L (384 384)
343M
3.07
256.1
0.83
0.52
+TenorAR
352M (+2.7%)
2.52
258.9
0.83
0.55
LlamaGEN (384 x 384)
775M
2.62
244.1
0.80
0.57
TensorAR
789M (+1.9%)
2.29
260.4
0.81
0.59
JamaaGEN -XL, (384 x 384)
2.34
253.9
0.81
0.60
TensorAR
1432M (+1.5%)
2.03
267.7
0.82
0.61
3.1.1 QUANTITATIVE COMPARISON
We evaluate TensorAR on two representative autoregressive (AR) generators Open- MAGVIT2 (Luo et al.) 2024) and LlamaGEN (Sun et al.) 2024 across multiple model scales. Table [ compares our approach with current of methods. Unless otherwise noted, the window size to 4, single-layer Q.n. and Q modules, and adopt

---

Published as a conference paper at ICLR 2026
exponential scheduling function. TensorAR consistently brings substantial gains over the under- lying AR baselines while adding only a small number of parameters. For example, augmenting LlamaGEN-B with TensorAR reduces Fréchet Inception Distance (FID) by 0.71 points. Even a 1. 1-B--prraettr model, TensorAR achieves a 0.31-point reduction FID, narrowing the gap to leading diffusion--ased models. Moreover, because the auxiliary modules (Qin and Qout) are kept fixed across backbones and scales, the relative parameter overhead decreases with model size, i.c., it is approximately inversely proportional to the backbone's 's overall computational cost.
3.1.2 TRAINING FID CURVE
we plot the training FID curves for TensorAR alongside those from standard fine- tuning of ClamaGEN-B and LlamaGEN-L. Fine-tuning for the same number of steps used with TensorAR yields no improvement in FID, confirming that TensorAR's gains stem from its design rather than from additional training.
In Figure 5
4.5
IomaGENN
3.5
4.6
LlmaaGen-R
2.5
LiamaGENNLL
Steady
Thnugleut
tenoy diecr sco0
TensocAR
improvement
1.5
3.6
FID improve
26
2.5
3.5
20000
40000
50000
80000
100000
(D)
Training Steps
Figure 4: Throughput/IID trade-off. Tenso- rAR consistently improves generation qual- ity with negligible decreases in throughput.
Figure 5: Training FID curves. TensorAR shows steady training dynamics based on two different backbones.
3.1.3 TIROUGHHPUT-FI CURVE
Figure further compares the sampling throughput of TensorAR and LlamaGEN across multiple model sizes. Throughput is measured as the number of samples generated per second (including AR generation and VQ decoding) on a single A100 GPU, using float32 precision and batch size of 128. Although TensorAR R incurs modest additional latency, it delivers substantial FID improvements, yielding superior efficiency-quee quality trade-off.
3.1.4 IMAGE QUALITY COMPARISON IN THE CLASS-TO-IMAL GENERATION TASK
We present qualitative comparison of images generated by LIamaGEN-XXI and TensorAR across four categories. Relative to the base LlamaGEN-XXx -XLL, TensorAR produces higher-quality images with richer semantic detail. Additional TensorAR samples included in the appendix, further demonstrating its ability to generate diverse outputs.
3.1.5 VISUAL COMPARISON IN THE TEXT-TO-IMAGE GENERATION TASK
We present qualitative comparison of images generated by LlamaGEN and TensorAR in the text- to-image generation task. Compared with the base LlamaGEN, TensorAR generates higher-quality images and exhibits more stable instruction-followingg.
3.2 EVALUATION ON TEXT-TO-IMAGE GENERATION TASK
We evaluate TensorAR's text-to-image generation on GenEval GGoss et al., 2023) and DPG- Bench (Hu et al. 2024), two benchmarks designed to assess instruction following and compositional

---

Published as a conference paper at ICLR 2026
LhmiCEN-XXI
TeturAR
GEN-XIXXXXX
TenserAR
cooppee
Figure 6: Image generation results comparison. TensorAR can generate high-quality images without loss of diversity. Best viewed in zoom.
LlaraGEN
TensorAR
LlanaGEN
TensorAR
photo of a sand
zibra right
rouch
parking neter
wom pates fleor, They are adjacent to tall, seathered wenden sarrel that spright.
dolphin through chicken with brown and white reathers stants on the nearby sandy shore.
Figure 7: Visual Comparison between LlamaGEN-B and TensorAR in the text-to-image generation task. The two prompts in the first row selected from the GenEval benchmark, and the other two are selected from the DPG-Bench benchmark. Benefiting from the effectiveness of the proposed TensorAR framework and high-quality data from the BLIP3o dataset, TensorAR can generate more vivid and cntructtionfollowiing images compared to its baseline counterpart.
alignment. Following the official protocols and metrics, we compare TensorAR with published re- sults for taae-off-the-at image generation models, summarized in Table and Table 4 Across both benchmarks, TensorAR delivers consistent gains over its base backbones and remains competitive with state-of-the-art flow-based generators. These findings indicate that integrating TensorAR into existing models enhances instruction-followe capability while maintaining strong overall perfor- mance. Additional qualitative comparisons of image quality between TensorAR and Janus-Pro-7B are provided in the appendix.
Evaluation of text-to- toxt-to-mmage generation ability on GenEval benchmark. Applying TensorAR brings consistent improvements for different base models.
Table 3:
Model
Single Obj.
Two Obj.
Counting
Colors
Position
Color Attri.
Overall*
Emu3-GGe Gen WWang et al.
2024
0.98
0.71
0.34
0.81
0.17
0.21
0.54
DALL-E BBaker et
2123
0.96
0.87
0.47
0.83
0.43
0.45
0.67
SD3-Mediumt EsSee
0.33
0.74
SEBD-X Ge él al. 2024
0.97
0.58
0.26
0.80
0.19
0.14
0.49
Show-o
Xie ct al 2024
0.95
0.52
0.49
0.82
0.11
0.28
0.53
D-DiT
et 2025
0.76
0.32
TensorAR
LlamaGen Sun et al.
2024
0.71
0.34
0.21
0.58
0.07
0.04
0.32
AensorAK
Junus- Junus-Pro-7B Pro-7B Chen
2025
0.99
0.89
0.59
0.90
0.79
0.66
0.80
TensorAR
0.99
0.93
0.53
0.92
0.85
0.79
0.83
8

---

Published as a conference paper at ICLR 2026
Table 4: Evaluation of text-to-image generation ability on DPG-Bench benchmark. Applying Ten-
sorAR brings consistent improvements for different base models.
Model
Global
Entity
Attribute
Relation
Other
Overall
Emu3-Gen WWagg et al.
2024
85.21
86.68
86.84
90.22
83.15
80.60
DALL-E (Betker et al. 2023
90.97
89.61
88.39
90.58
89.83
83.50
SD3-Medium FEsser et 2024
87.90
91.01
88.83
80.70
88.68
84.08
Hunyuan-DiT [Ta cl al. 2024b
84.59
80.59
88.01
74.36
86.41
78.87
PixArt- (Chen
86.89
82.89
88.94
86.59
87.68
80.54
TensorAR
LlamaGen (Sun al. 2024
78.72
58.63
68.22
76.63
44.00
43.13
+ TensorAR
84.50
81.92
81.65
90.68
74.80
73.33
Janus-Pro-7B Chen et al. 2025
86.90
88.90
89.40
89.32
89.48
84.19
TensorAR
86.39
90.67
90.66
91.35
84.52
85.57
3.3
ABLATION STUDIES
3.3.1
DIFFERENT NOISE SCHEDULING FUNCTIONS
As discussed above, the noise scheduling func- tion controls the noise level assigned to each position within a tensor. We evaluate four schedules: linear, sine, square root, and expo- nential, whose definitions and hyperparameters summarized in Table 5 We set the base model of all the following ablation studies as LlamaGEN-B in the class-to-image generation task. Across settings, all four schedules yield substantial gains over the base configuration, indicating that TensorAR is robust to the spe-
Table 5: Different noise scheduler functions.
Model
FID
IS
Precision
Recall
Baseline
5.46
193.6
0.83
0.45
Linear Sine
4.79
218.8
0.85
0.44
4.75
221.3
0.84
0.45
Square root Exponential
4.84
214.9
0.83
0.43
4.71
225.8
0.85
0.45
cific choice of schedule. Among them, the exponential schedule achieves the lowest Fréchet In- ception Distance (FID), making it strong default in practice. Overall, these results suggest that the scheduling function is an important factor in TensorAR's performance, with the exponential schedule offering the best efficiency -uualit trade-off.
Table 6: Ablation studies on the design of TensorAR.
(a) Different window size
Depth of Q+. and Qout.
Model
FID
IS
Precision
Recall
Model
FID
Precision
Recall
Latency
Baseline
5.46
193.6
0.83
0.45
Baseline
5.46
0.83
0.45
0.11
k=2
4.78
221.3
0.45
d=1
4.71
0.85
0.45
0.12
k=4
4.71
225.8
0.85
0.45
d=2
4.79
0.85
0.46
0.14
k=8
4.68
226.7
0.85
0.46
d=4
4.90
0.82
0.43
0.15
3.3.2 DIFFERENT WINDOW SIZES
Increasing the window size allows TensorAR to revisit and improve each image token over more steps, which should enhance overall quality. To assess this effect, we vary the window size k e (2 and summarize the results in Table 6al We observe monotonic reduction in Fréchet Inception Distance (FID) as increases, indicating that additional refinement passes are consistently beneficial. Even k 2 2-which provides only single refinement pass per Miken-TeensorAR significantly outperforms the baseline, underscoring the effectiveness of explicit refinement. These findings validate the refinement mechanism entails more sampling steps and thus higher inference cost, practitioners can select & to balance quality and latency, with moderate values offering favorable trade-off.
key contributor to performance. Because larger

---

Published as a conference paper at ICLR 2026
3.3.3 DEPTH OF Qin AND Qout
Both Qin and Qoa modules are implemented as query transformers, with each layer comprising cross-attention layer. We investigate the optimal depth for these modules by varying the number of layers d E {1, 2, 4). As reported in Table 6bb d = achieves the lowest Fréchet Inception Distance (FID), while increasing
yiclds no further improvement. However, considering the quality-latency trade-off, we adopt d = as the default, which substantially improves throughput with only a modest impact on image quality. This choice offers a favorable balance for practical deployment.
3.4 VISUALIZATION OF REFINEMENT
As described in Section 2.2 at each decoding step, TensorAR outputs block of it consecutive to- kens. The first token the block is committed to the final sequence, while the remaining tokens are provisional and refined in subsequent steps. This commit-and-refine strategy induces zig-zag, coarse-to-fine progression across positions (Sun et al.] 2025): previously emitted tokens (except the first in each block) are iteratively improved as new tokens are introduced. To illustrate this behavior, Figure g visualizes the evolution of outputs produced by Janus-Pro-7B model with window size of 4. Applying TensorAR yields higher visual quality and stronger instruction following than the baseline. The images become progressively sharper and semantically richer as refinement proceeds. These qualitative results corroborate the effectiveness of the refinement mech- anism. Additional visualizations are provided in the appendix.
Prompt
Janus -rous-Pro-78
Finetuned
TensorAR
stands with another nan ...
there is single arched
yellow taxis, red buses
Figure 8: Visualization of the refinement process of TensorAR against its base model: Janus-Pro- Pro-7B with a window size h: 4. We mark the text that Janus-Pro-77 fails to generate in red and point to the corresponding object generated by TensorAR via red arrow. All these prompts are from the DPG-Bench benchmark. Best viewed in zoom.
4 CONCLUSION
In this paper, we present TensorAR, to the best of our knowledge, the first visual autoregressive framework that integrates an explicit refinement mechanism into the decoding process. TensorAR extends the conventional next-token prediction paradigm to next-tensor prediction by introducing two lightweight plug-in modules, enabling iterative revision of recent outputs. Crucially, it functions as drop-in augmentation to standard autoregressive transformers, requiring no modifications to the base architecture or changes to the training procedure. Across both class conditioniii image synthesis and text-to-imen tot-ttioimmeeeeee generation, TensorAR delivers consistent improvements in quality, demonstrating the effectiveness of incorporating refinement into visual autoregressive models.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (No. U24A20291), National Key Research and Development Program of China under Grant (No. 2023YFB4403101), and Sanqin Talent Special Support Plan (No. 2024STZZK09).
REFERENCES
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 19911-17999333 1-79993, 2021.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang. Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dull---3 pdf. 2(3):8. 2023.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-a: Weak-to-strone training of diffusion transformer for 4k text-to-image generation. In European Conference on Computer Vision, pp. 74-91. Springer, 2024.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501. 17811, 2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Shi Guang, and Haoqi Fan. Causal diffusion transformers for generative modeling. arXiv preprint arXiv:2412.12095, 2024.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pp. 12873-12883, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024.
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens. arXiv preprint arXiv:2410. 13863, 2024.
Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14333 2024.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object- focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152. 2023.
Jiatao Gu, Yuyang Wang, Yizhe Zhang, Qihang Zhang, Dinghuai Zhang, Navdeep Jaitly, Josh Susskind, and Shuangfei Zhai. Dart: Denoising autoregressive transformer for scalable text- to-image generation. arXiv preprint arXiv:2410.08159, 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Towards non-autoregressive language models. arXiv preprint NXXV:22022.0339, 3(4):5, 2021.
Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Um for enhanced semantic alignment. arXiv preprint 1555v:2403.05/35, 2024.
11

---

Published as a conference paper at ICLR 2026
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Doyup Lee, Chiheon Kim. Sachoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11523-11532 2022.
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37: 2024a.
Zhimin Li, Jianwei Zhang. Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, et al. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding. arXiv preprint arXiv:2405.08748, 2024b.
Zijie Li, Henry Li, Yichun Shi, Amir Barati Farimani, Yuval Kluger, Linjie Yang, and Peng Wang- Dual diffusion for unified image generation and understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2779-2790, 2025.
Enshu Liu, Xuefei Ning, Yu Wang, and Zinan Lin. Distilled decoding 1: One-step sampling of image auto-regressive models with flow matching. arXiv preprint arXiv:24/2. 17153, 2024a.
Enshu Liu, Qian Chen, Xuefci Ning, Shengen Yan, Guohao Dai, Zinan Lin, and Yu Wang. Distilled decoding 2: One-step sampling of image auto-regressive models with conditional score distilla- tion. arXiv preprint arXiv:2510.2100 2025.
Wenze Liu, Le Zhuo, Yi Xin, Sheng Xia, Peng Gao, and Xiangyu Yue. Customize your visual autoregressive recipe with set autoregressive modeling. arXiv preprint arXiv:2410.105: 1,
Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvin An open-source project toward democratizing auto-regressive visual generation. arXiv preprint arXiv:2409.044//0 2024.
Ziqi Pang, Tianyuan Zhang, Fujun Luan, Yunze Man, Hao Tan, Kai Zhang, William T Freeman, and Yu-Xiong Wang. Randar: Decoder-only autoregressive visual generation in random orders. arXiv preprint ar21:2242..01827, 2024.
Sucheng Ren, Qihang Yu, Ju He, Xiaohui Shen, Alan Yuille, and Liang-Chieh Chen. Beyond next- token: Next-x prediction for autoregressive visual generation. arXiv preprint arXiv:2502.20388, 2025.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference machine learn- ing, pp. 2256-2265. pmlr, 2015.
Mingzhen Sun, Weining Wang, Gen Li, Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, Yi Zhou, Qian He, and Jing Liu. Ar-diffusion: Asynchronous video generation with auto-regressive diffusion. In Proceedings of the Computer Vision and Patlern Recognition Conference, pp. 7364- 7373, 2025.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.00525 2024.
Chameleon Team. Chameleon: Mixed-mold farly-fusion foundation models. arXiv preprint 0058888 2024.
12

---

Published as a conference paper at ICLR 2026
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng. and Liwei Wang. Visual autoregressive modeling: Scalable image generation via sext-scale prediction. Advances in neural information processing systems, 37:84839 777848339-8865, 2024.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.1888 18869, 2024.
Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang- Chieh Chen. Maskbit: Embedding-free image generation via bit tokens. arXiv preprint arXiv:2409. 16211, 2024.
Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models scalable and unified multi-mode generators. arXiv preprint arXiv:2412.04332. 2024.
Yicheng Xiao, Lin Song, Yukang Chen, Yingmin Luo. Yuxin Chen, Yukang Gan, Wei Huang, Xiu Li, Xiaojuan Qi, and Ying Shan. Mindomni: Unleashing reasoning generation in vision language models with rgpo arXiv preprint arXiv:2505. 13031, 2025a.
Yicheng Xiao, Lin Song, Rui Yang. Cheng Cheng, Zumnan Xu, Zhaoyang Zhang, Yixiao Ge, Xiu Li, and Ying Shan. Haploomni: Unified single transformer for multimodal video understanding and generation. arXiv preprint wrxiv:2506.0297. 2025b.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijic Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. 12528, 2024.
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vggan. arXiv preprint 1xXv:2210000627, 2021.
Lijun Yu. José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, al. Language model beats diffusion- tokenizer key to visual generation. arXiv preprint 1rXiv:2310.05737, 2023.
Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chihh Chen. Randomized autoregres- sive visual generation. arXiv preprint arXiv:2411.0077,, 2024.
Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. arXiv preprint 23X1v 2023.
13


Reviews:
[{"review_id": "1SHbiaBe9P", "reviewer": "Reviewer_rrSZ", "summary": "The paper introduces TensorAR, a plug-and-play autoregressive image generation framework that replaces traditional next-token prediction with next-tensor prediction, enabling iterative refinement of previously generated content while preserving causality. By incorporating a discrete tensor noising mechanism inspired by diffusion theory and lightweight input/output modules, TensorAR consistently improves generation quality and instruction-following ability across class-conditional and text-to-image tasks without altering the base architecture or training paradigm.", "questions": "1. Fig. 7 shows the visual comparison between Janus-Pro-7B and Janus-Pro-7B+TensorAR. The visualization is nice. As I know, Janus-Pro is an understanding and generation unified model. Does Janus-Pro + TensorAR support unified understanding and generation? If yes, how about the generation performance?\n2. The name TensorAR may not directly reflect the core methodology. I think \"GroupAR\" may be better aligned with the method.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The overall paradigm is very interesting, which naturally combines the traditional AR image generation with the diffusion model.\n(2) The method is effective. Extensive experiments over Open-MAGVIT and LlamaGEN have proven its effectiveness.", "weaknesses": "1. The illustration in Fig. 1 is not intuitive.\n2. The llamagen baselins is a little weak. AS I know, SimpleAR (https://github.com/wdrink/SimpleAR) is a stronger baseline. The experiments could be improved by utilizing SoTA baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vg6lCt6Abm", "reviewer": "Reviewer_ojZ8", "summary": "This paper proposes TensorAR, a novel autoregressive method for image generation that introduces a refinement mechanism by shifting from next-token to next-tensor prediction. TensorAR predicts overlapping tensor windows, allowing later predictions to revise earlier ones. The method is designed as a plug-and-play extension to existing AR models. Experiments on class-conditional and text-to-image generation show consistent improvements in FID.", "questions": "1. Have you ever tried a larger k value? To what extent can FID be optimized when k approaches the sequence length.\n2. Can you explain that why is the exponential noise schedule superior?  Is there an intuitive or theoretical justification, or is it purely empirical?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper is well-written and provides a thorough and accurate explanation of the main methods.\n2. The core idea of \"next-tensor prediction\" is simple yet powerful. It provide a new approach to bridge the gap between autoregressive generation and refinement-based paradigms.\n3. TensorAR requires no modification to the base AR architecture or training objective, making it highly practical and easy to integrate with existing models.\n4. The author conducted experimental verifications on various autoregressive models, including Open-MAGVIT2, LlamaGen and Janus-Pro.", "weaknesses": "1. Lack of a stronger explanation or demonstration of refinement: The paper claims that tokens are \"refined\" over multiple steps, but it does not provide direct evidence  that the model actually revises its predictions meaningfully during refinement steps, rather than simply generating forward. TensorAR improves the T2I generation effect as shown in Figure 7. The baseline fails on \"a person stands with another man\" but TensorAR succeeds—is this due to refinement of earlier tokens, or improvement in the ability to follow the text after additional fine-tuning?\n2. The visualization results for the T2I generation are missing: The results of LlamaGen+TensorAR presented by the author in Table 1 (0.61) far exceed those of LlamaGen itself (0.31). The improvement in its visualization results should be more significant than that of Janus-Pro-7B, but no comparison of the visualization results are given.\n3. Incorrect interpretation of the results of the ablation experiment: The author explains Table 5b in section 3.3 by saying: \"... d = 2 achieves the lowest\nFr´echet Inception Distance (FID) ...\", But this does not match the result given in the table.\n4. The names of the indicators in the table are misaligned. In Table 5a, \"IS\" and \"FID\" are written in reverse.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jFaf85n0ZP", "reviewer": "Reviewer_nvDe", "summary": "The paper introduces an enhanced paradigm for autoregressive image modeling, where instead of generating a single token at each step, the model is trained to produce multiple tokens in a sliding window manner. As a result, each token is generated multiple times across steps, allowing for iterative refinement. However, because some target output tokens are presented in the input, the model may learn to simply copy inputs to outputs. To mitigate this shortcut, the authors propose adding noise to the input tokens. Experiments demonstrate that applying the proposed method to LlamaGEN and Open-MAGVIT2 leads to improved image generation quality on ImageNet.", "questions": "I reviewed this paper in a previous venue. This version has addressed most of my earlier concerns, particularly regarding the text-to-image generation experiments and the improved balance between generation quality and latency. Therefore, I continue to support the paper and keep recommending a positive score. Below, I list my new questions for this round of review.\n\n* There are numerous citation-related typos throughout the paper. For example:\n    * Missing spaces before citations, e.g., \"frameworks(Pang et al., 2024\" (line 36), \"standard AR models(Pang et al., 2024\" (line 42)\n    * Incorrect citation formatting — \\citep should be used instead of \\citet in many cases, e.g., \"multimodal integrationWu et al. (2024); Team (2024).\" (line 40), \"Discrete diffusion models Sohl-Dickstein et al. (2015)\" (line 164), \"existing studies Zheng et al. (2023)\" (line 169).\n\n* One limitation of auto-regressive models is their slow sampling speed. A promising direction to mitigate this issue is to distill pre-trained auto-regressive models into one-step samplers using Distilled Decoding (https://arxiv.org/abs/2412.17153, https://arxiv.org/abs/2510.21003), which aims to retain the high generation quality of AR models while achieving fast sampling. (For the second paper, I understand that the authors are not expected to be aware of or discuss it given its very recent publication; I include it here only for completeness.) It would be interesting to discuss whether TensorAR is compatible with such an approach (i.e., applying distilled decoding on TensorAR models).\n\n\n* What is the rationale behind using \"1/(k/2)\" in the exponent of the exponential schedule? In general, any decreasing function of k could be applied in the exponent, so it would be helpful to clarify why this particular form was chosen.\n\n* Compared to the previous submission, the performance of TensorAR on LlamaGEN models has improved substantially, which is great to see. I notice that one difference is that the TensorAR models were made larger (according to Table 2). I have a few follow-up questions:\n(1) Which parameters were modified to increase the model size?\n(2) Was this the only change, or were there other factors contributing to the improved performance?\n\n* In the previous version, experiments were also conducted on RAR models (which was a valuable addition), but they appear to have been removed in the current version. Could the authors clarify the reasoning behind this decision?\n\n* Figure 4 does not appear to match the results reported in Table 2. Specifically, the smallest models in Table 2 have FIDs of 5.46 (base model) and 4.71 (TensorAR), whereas the highest points in Figure 4 show FIDs around 3.5–4 (base model) and 3–3.5 (TensorAR). My assumption is that different models were selected for Table 2 and Figure 4. It would be helpful if the authors could clarify this discrepancy.\n\n* If I understand correctly, all TensorAR models in the experiments are fine-tuned from pretrained AR models. Would TensorAR still perform well when trained from scratch, or does it rely on a well-trained base model to achieve good performance? It would be great to include some discussion on this point, though additional experiments are not necessary.\n\n* In Table 5(a), the column headers for IS and FID appear to be reversed.\n\n* In Table 6, it would be helpful to include the results of the base models to make it easier to support the statement that \"all four schedules yield substantial gains over the base configuration\" (line 420).\n\n* Line 436 claims that \"d = 2 achieves the lowest Fr´echet Inception Distance (FID)\". However, in Figure 5(b), d=1 has the best FID.\n\n* Line 53: \"AR\" should be \"VAR\"\n\n* I am a bit confused about the title of Section 2 — is “TensorAR-T2I” a typo? Should it instead be “TensorAR”?\n\n* Line 224: it would be better to change \"(x_i,...,x_{i+k-1}^ * )\" to \"(x_i, x_{i+1}^ * , ..., x_{i+k-1}^ * )\" (i.e., adding \"x_{i+1}^*\"). Otherwise, it is unclear if \" * \" should be applied on x_{i+1}, ..., x_{i+k-1}.\n\n* Eq 4: The index of x (2, ... k-1) is incorrect, given that Line 228 discusses \"x_t, ..., x_{t+k-1}\"\n\n* In Eq 5, the function q has j as a condition. However, in Eq. 4, this condition is missing. I think it would be more rigorous to have this condition.\n\n* Eq. 5: It would be more accurate to place the summation before the expectation, since the variables involved in the expectation (i.e., those in the subscript of E) depend on the definitions of i and j.\n\n* Line 262 states that \"when k equals the total number of image patches T, TensorAR becomes equivalent to a discrete variant of a diffusion process.\" This is not exactly accurate, since the denoising order differs: in TensorAR, the clean tokens are revealed from left to right, whereas in discrete diffusion, the order can be random.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well-written.\n\n* The proposed method is simple and easy to implement. It can be plugged into most of the existing image AR models.\n\n* The paper demonstrates the effectiveness of the method across multiple image AR models and multiple tasks (class-conditioned generation and text-to-image generation).", "weaknesses": "* The paper contains many typos, though they are minor and should be easy to fix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "56fDFSoIOQ", "reviewer": "Reviewer_JNGB", "summary": "In this work, the authors present TensorAR, an improvement over standard AR vision generative models. Unlike standard AR where one token is generated at a time, TensorAR predicts overlapping tensor windows. This allows correction of predicted tokens in inference. The authors also proposed noise mechanism to avoid information leakage in training. Empirical results on class-conditioned image generation and text-to-image generation show that TensorAR tuning on AR models improve performance while adding relatively small computation overhead.", "questions": "Please see Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The motivation to allow correction in AR image generation is valid and effective in improving generative models. \n2. TensorAR is easy to implement and a plug-and-play module for standard AR models. \n3. Empirical results demonstrate effectiveness of TensorAR on most benchmarks.", "weaknesses": "1. In abstract, the authors mention TensorAR to predict overlapping tensor windows. However, the definition of the tensor window is vague. Adding explanation to clarify this can help audience better understand the work. \n2. How is w_j in Eq.5 defined in training?\n3. In Table 2, are numbers of the baselines (ie, Open-MAGVIT2 and LlamaGEN) evaluated with or without finetuning? It would help better illustrate the improvement of TensorAR if compared with baselines that are finetuned for same iterations but with standard AR strategy. \n4. Do authors have quantitative measure of additional computational overhead (eg, FLOPs) with TensorAR compared to standard AR models (ie, Open-MAGVIT2 and LlamaGEN)? \n5. Do authors have quantitative measures of the portion of tokens that are flipped in inference? I'm curious whether the improvement roots from actually flipping the prediction or the additional information in each step. \n6. In Table 5a, the column names of IS and FID seem to be wrong.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)