Visual Autoregressive Modeling for Image Super-Resolution
Authors: Yunpeng Qu
Kun Yuan
Jinhua Hao
Kai Zhao
Qizhi Xie
Ming Sun
Chao Zhou
Venue: ICML 2025 poster (2025)
Keywords: visual autoregressive modeling; image super-resolution
Primary area: applications->computer_vision
TL;DR: This paper introduces VARSR, a novel visual autoregressive framework for image super-resolution.

Abstract: Image Super-Resolution (ISR) has seen significant progress with the introduction of remarkable generative models.
However, challenges such as the trade-off issues between fidelity and realism, as well as computational complexity, have also posed limitations on their application.
Building upon the tremendous success of autoregressive models in the language domain, we propose \textbf{VARSR}, a novel visual autoregressive modeling for ISR framework with the form of next-scale prediction.
To effectively integrate and preserve semantic information in low-resolution images, we propose using prefix tokens to incorporate the condition.
Scale-aligned Rotary Positional Encodings are introduced to capture spatial structures and the diffusion refiner is utilized for modeling quantization residual loss to achieve pixel-level fidelity.
Image-based Classifier-free Guidance is proposed to guide the generation of more realistic images.
Furthermore, we collect large-scale data and design a training process to obtain robust generative priors.
Quantitative and qualitative results show that VARSR is capable of generating high-fidelity and high-realism images with more efficiency than diffusion-based methods.
Our codes are released at \url{https://github.com/quyp2000/VARSR}.

Visual Autoregressive Modeling for Image Super-Resolution
Yunpeng Qu 12
Kun Yuan?
Jinhua Hao? Kai Zhao?
Qizhi Xie 12 Ming Sun
Chao Zhou
Abstract
methods leverage generative model priors tackle ISR, with approaches based on GAN models (Zhang et al., 2021; Wang et al., 2021) and diffusion models (Ho et al., 2020). While these methods have achieved significant advance- ments, ISR, as an ill-posed problem, struggles to balance the realism and fidelity of the restoration results. GAN- based methods achieve higher fidelity metrics, but limita- tions in generation capability and the goal of over-fidelity optimization make challenging to reproduce vivid and re- alistic textures (Wu et al., 2024a). Diffusion methods, based on their strong generative priors, can generate extremely rich image details. However, the random noise sampling approach and the gap between generative priors and LR distribution pose challenges for pixel-Jevel fidelity (Yu et al., 2024). Building upon previous works, we aim explore further existing frameworks enhance fidelity and realism.
Image Super-Resolution (ISR) has seen signifi- cant progress with the introduction of remarkable generative models. However, challenges such as the trade-off issues between fidelity and re- alism, as well as computational complexity, have also posed limitations on their application. Build- ing upon the tremendous success of autoregres- sive models in the language domain, we propose VARSR, novel visual autoregressive modeling for ISR framework with the form of next-scale prediction. To effectively integrate and preserve semantic information in low-resolution images, we propose using prefix tokens incorporate the condition. Scale-aligned Rotary Positional En- codings are introduced to capture spatial struc- tures, and the Diffusion Refiner is utilized for modeling quantization residual loss to achieve pixel-level fidelity. Image-based Classifier-free Guidance is proposed to guide the generation of more realistic images. Furthermore, we collect large-scale data and design a training process to obtain robust generative priors. Quantitative and qualitative results show that VARSR is capable of generating high-fidelity and high-realism im- ages with more efficiency than diffusion-basee methods. Our codes are released at https: //github. httph//...o./gmgg//////////
Introducing the generative capability of large models to enhance ISR is prevalent trend. With the rich semantic priors of T2I models pretrained on massive datasets, many works (Yang et al., 2025; Wu et al., 2024b) have applied the powerful generative models ISR tasks and achieved signif- icant results. However, for ISR tasks demanding pixel-level fidelity, the diffusion model presents certain limitations, in- cluding the high computational complexity of iterative infer- ence and potential issues with semantic hallucinations (Kim et al., 2025). With the success of autoregressive (AR) mod- cling in Large Language Models (LLMs) (Touvron et al., 2023; Achiam et al., 2023), visual autoregressive modeling (VAR) has gained great attention, exemplified by VQGAN (Esser al., 2021) and DALL-E (Reddy al., 2021), which apply discrete token prediction for image generation. VAR (Tian et al., 2024) takes step further by quantizing images into scale-wise token maps and generating images through next-scale prediction, leading to excellent results across mul- tiple generation tasks (Tang et al.; Zhang et al., 2024; Yao et al., 2024). This novel approach provides new insights into ISR, showcasing potential advantages over diffusion methods: (1) The coarse-to-fine to-fine next-scale prediction con- forms Markov unidirectional modeling, providing better structural preservation and naturally adapting to ISR tasks; (2) The reduced number of inference steps and lower com- plexity in preceding scales result in higher efficiency.
1. Introduction
Image Super- Resolutino (ISR) aims to generate realistic high-resolution (HR) images from their degraded low- resolution (LR) counterparts. Traditional ISR methods fo- cus on restoring LR images by assuming simple and known degradations (Dong et al., 2016; Zhang al., 2018b), which limits their practicality in real- world scenarios with complex distortions (Gu et al., 2019a; Zhang et al., 2023b). Recent
'Tsinghua University, Beijing, China Kuaishou Tech- nology, Beijing, China. Correspondence to: Kun Yuan <yuankun03 @ kuaishou.com>>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
However, unlike other controllable generation tasks, ISR re- quires generation with pixel-level fidelity alongside seman-

---

Visual Autoregressive Modeling for Image Super-Resolution
ties reservation, presenting challenges in integrating VAR into ISR tasks: (1) How to efficiently and effectively incor- porate LR condition information to generate high-fidelity images? (2) AR- base approaches map images into ID token sequences for processing. To achieve enhanced spa- tial structure preservation in ISR, how to better represent the positional relationships between tokens? (3) AR-based approaches quantize continuous latent to discrete tokens for training with cross-entropy loss. How to avoid the loss of in- formation caused by the quantization process to improve the pixel-level fidelity of ISR? (4) Optimization objectives for fi- delity may impact the generation of rich image details. How to perceive low-quality distortions and guide the generation of more realistic images within the AR framework?
Generative priors are essential for addressing severe degra- dation in ISR tasks, capturing image structure and real-world distribution (Ho et al., 2020; Blattmann et al., 2022; Mou et al., 2024). While GAN-based methods show impressive results (Chen et al., 2022; Zhang et al., 2021; Wang et al., 2021), training stability and generative priors limit their ability to produce realistic details. Diffusion models are popular in ISR for strong generative capabilities (Saharia et al., 2022; Kawar et al., 2022; Qu et al., 2025), but may generate unrealistic details or hallucinations (Aithal et al., 2024; Narasimhaswany et al., 2024). This may stem from their simplification of the Markov process, restricting the access to antecedent denoised trajectories (Gu et al., 2024).
Visual Autoregressive Models. Building upon the tremen- dous success of LLMs (Touvron et al., 2023; Achiam et al., 2023), visual autoregressive models utilize discrete quantiz- ers such as VQVAE (Van Den Oord et al., 2017) to transform image patches into index-wist tokens, generating images based on next-token prediction (Yu et al., 2022; Wang et al., 2024b; Lee et al., 2022; Chang et al., 2022). However, the prediction of flattened tokens may lead to the loss of spatial structure. VAR (Tian et al., 2024) shifts from the next- token prediction to the next-scale prediction, significantly enhancing image generation quality and offering excellent scalability. VAR-based methods such as STAR (Ma et al., 2024), ControlAR (Li et al., 2024c) have expanded to other conditional generation tasks (e.g., class-to-image (C2I), text- to-image (T2)), yielding results comparable to diffusion models and validating the potential of VAR (Zhang et al., 2024; Yao et al., 2024; Roheda, 2024; Chen et al., 2024b).
In this paper, we propose a novel Visual AutoRegressive modeling for image Resolution (VARSR) framework, with next-scale prediction. To enhance the integration of semantic information from LR images, we propose using Prefix Tokens as a proceeding scale to incorporate the LR condition as global guidance. To better capture the spa- tial structure between tokens, we utilize the Scale-aligned Rotary Positional Encodings (SA-RoPE) to calibrate the spatial positions of LR images and token maps. To predict the residual loss of the discrete quantizer, the Diffussion Re- finer is introduced to model continuous distributions and achieve finer pixel-level restoration. To generate more real- istic images, we define negative samples to learn lualquality distortions and propose Image-based Classífier-free Guid- ance (CFG) to guide the distribution for generating richer textures. Furthermore, leverage powerful generative pri- ors, we collect large-scale data and plan comprehensive training process. Our contributions are as follows:
3. Methods
1. We first introduce VAR into the ISR field and pro- pose VARSR, which specifically designed to address LR conditions, spatial structure representation, quan- tization loss prediction, image- based CFG, and other issues to apply to ISR tasks.
3.1. Preliminary: Visual Autoregressive Modeling
Next-token Prediction is relied on by traditional AR mod- els to generate patches at different positions in an image. Images are mapped into latent by a visual autoencoder and then tokenized into a series of token maps (I1, 22 ..., .T) using a discrete quantizer. AR model predicts the token zr based on its preceding sequence (21, ..., 1) and condition c. The conditional probability can be expressed:
2. We collect large-scale, high-quality image dataset to establish a robust base model and design training pipeline for fine-tuning downstream ISR tasks.
3. Through quantitative and qualitative analysis, VARSR demonstrates a strong capability in generating high- fidelity and high-realism images, achieving the best performance on multiple image quality metrics and 10x efficiency improvement of diffusion methods.
p (21, F2, ..., TT) II (1x 21, T2, C),, (1)
Previous work (Tian et al., 2024) has pointed out that "next- token prediction" is insufficient for highly structured images, as it will disrupt the spatial structure and defy the unidirec- tional dependence assumption of autoregression.
2. Related work
Generative Priors for ISR. Existing ISR methods typi- cally focus on blind recovery without assuming specific degradations (Huang et al., 2020; Bell-Kligler et al., 2019).
Next-scale Prediction is reformulated based on these analy- ses (Tian et al., 2024), where the basic unit of autoregression becomes the token map of the entire scale. The visual en-
2

---

Visual Autoregressive Modeling for Image Super-Resolution
coder (e.g, VQVAE) first embeds the image / into feature map f Rxxwxd and then quantizes the feature map f into K multi-scale token maps (r1, r2:....K) TK).
three main stages: conditional control generation, visual autoregression, and diffusion refinement. In the first stage, to address the Ist problem, we use an image encoder to map LR images to conditional tokens, which guide the gen- eration of VAR by serving as Prefix Tokens (Sec. 3.2.1). In the second stage, a pretrained VAR model is employed. To address the 2nd problem. we implement Scale-align rotary positional encoding (SA-RoPE) in the transformer to calibrate the spatial positions of tokens different scales (Sec. 3.2.2). In the third stage, to address the 3rd prob- lem, a lightweight Diffusion Refiner estimates quantization residuals from generated discrete tokens to enhance image details (Sec. 3.2.3). To address the 4th problem, we further propose Image-based bbasd classifter-free guidance (CFG) as an additional control learn distortions and generate better- quality images without additional training (Sec. 3.2.4).
fK= - f Σ upsample((ooup((..m))
(15) rk = arg min Jookup(V. f(13)) vE[V]
2
where V is the VQVAE codebook, The E [V] is the token map at scale k and lookup(V, v) retrieves vectors from codebook V based on index VAR predicts the next scale rkK based on previous outcomes and the condition:
K II p(r (T-1,)),,,
p rK)
3.2.1. CONDITIONAL CONTROL
VAR with next-scale prediction advances AR models greatly. For ISR tasks demanding fidelity and realism, VAR's pro- gressive generation with coarse-to-fine coherence aligns with human perception and the Markov unidircctional as- sumption, ensuring high structural fidelity and aesthetic quality, Hence, applying VAR to ISR has broad prospects.
The key challenge in ISR is how to effectively integrate LR information for controllable generation. Diffusion methods often use ControlNet (Zhang et al., 2023a) due to its superior performance over simpler approaches like concatenation or addition with noise. However, for AR models, ControlNet increases the computational burden and may lead to conflicts between LR priors and prefix-scale information.
3.2. VARSR framework
Considering the next-scale prediction form of VAR, we adopt more efficient way to introduce LR conditional con- trol called Prefix Tokens. LR image It is encoded through an image encoder E to be mapped into conditional tokens re E(rr)). Te is used as the initial scale and placed at the start of all tokens in the form (re, F1, F2, .... F.. ) where re is of size h x w like the final scale rx. VAR iteratively predicts the next scale by considering conditional tokens and the previous scales to effectively integrate LR priors.
We aim to enhance ISR via the VAR models. Unlike other controllable generation tasks (e.g., C21, T2I), ISR necessi- tates maintaining semantic integrity and achieving precise pixel-level restoration. Therefore, when using the VAR, there still exist four critical issues requiring resolution:
In what manner can the conditions derived from LR images be efficiently and effectively furnished to VAR to achieve high-fidelit semantic restoration?
In addition, HR images are categorized into two classes based on their quality during training. High- and low-quality categories are respectively associated with positive embed- ding negative embedding for quality-centered control in VAR through modulate layer. The objective is to impose Image-based CFG, as will be elaborated on in Sec. 3.2.4.
Tokens are concatenated into a ID sequence in VAR, leading to the loss of spatial structures, which affects the fine-grained local restoration. How can the posi- tional relationships of tokens be represented?
VAR processes the image into discrete tokens through quantization, resulting the significant loss of high- frequency details and impacting the pixel-Ievel restora- tion. How to reduce the impact of quantization loss?
3.2.2. AUTOREGRESSIVE TRANSFORMER
As shown in Fig. 2, our VARSR utilizes a visual transformer for predicting multi-scale tokens. In training, the tokens r'e of the previous K scales are interpolated to next scale:
The fidelity-oriented optimization objective may affect the generation of image details. How can we perceive distortions and quality factors in images to generate more realistic and higher-quality images?
A = interpolate(lookupp/
E R he4s Xmx41x4 xd (4)
Conditional tokens and interpolated tokens are concatenated as (Te, f1, f,,, 1 to predict the corresponding output (fi, f2, fx) for each scale. A block-wite wise attention mask confines each r% to focus solely on its preceding tokens rs k with ra accessible to all tokens to offer L.R priors.
To address these problems, we propose a framework called Visual AutoRegressive modeling for image Super- Resolution ( (VARSR). As depicted in Fig. 1, consists of
3

---

Visual Autoregressive Modeling for Image Super-Resolution Resolution
Quality-conterdd
Continuous Reconstruction
Next-scule predicted Diverete
Results
Negatio
FreheMing
Modulate
Image
Degradation
Pipeline
Scalable Visual
Scales
Diffusion Steps
Autoregressive Transformer
MLP
Tixed
Image Encoder
Trainable
Quility certered
Conditional tokens
Fi
fg-1
Noise
Control
Interploted Discrete tokens
LR Image
Conditional Control
Visual Autoregression
Diffusion Refiner
Figure 1. VARSR framework, which can be divided into three parts: (1) LR image is as Prefix Tokens as condition. (2) VAR generates discrete tokens based on next-scalc prediction. (3) Diffusion Refiner predicts the continious tokens as quantization residuals.
Quality-entterdd Control
Where Ry, Oc E RCxC are the standard rotation matrix for RoPE and the zero matrices, respectively. Our SA-RoPE will be applied to the multiplication in self-attention to in- corporate positional information for query and key of the two tokens Tim: Typ
Modulate
Offict
Offects
Multi-Head
Feed- Forward
Atlf-Arteation
Layer
(6) Where m, represent their positional information, includ- ing the scale and position. It is worth noting that we also perform scale alignment on the LR condition re to incor- porate the spatial structure of LR for next-scale restoration, enhancing the structural fidelity.
T q k. (Re(Zm) qmm)) ) (R w.,)
Input Takens
Autoregressive Transformer Block
Scale-aligned Ratary Position Embedding
Attention mask
Traneformmaiion
Widd-Sim
1
Conditiunal Token
Lastly, the positive negative embedding of the quality- centered control is introduced via the modulate layer, guid- ing the autoregression through the generation of offsets.
Token
Scale Token
7x
Masked
Figure Internal structure the autoregressive transformer. SA- RoPE represents the spatial structure. Quality-centered control generates offsets for autoregression.
3.2.3. DIFFUSION REFINER
In Fig. 3, the image is quantized into discrete tokens, repre- sented as indices of the codebook V. However, this discrete quantization introduces quantization loss, leading to the loss of high- fighffreqeency textures and restricting the upper bound of restoration, VAR can only predict the quantized dis- crete vectors of the image. Such loss is problematic for ISR, which demands precise pixel-level recovery. MAR (Li et al., 2024a) and HART (Tang al.) utilize a diffusion loss to map predicted tokens to continuous probability distribu- tion, mitigating quantization loss. VARSR adopts the above approach of using diffusion refiner to convert predic- tions of categorical vector distribution into continuous- valued space through a diffusion loss, thereby enhancing the upper bound VAR's capacity. The quantization residual is defined as continuous tokens 2:
Absolute positional encodings (APE) in transformers can- not capture 2D structure represent the inter-scale rela- tionships, which are crucial to incorporate spatial structure from LR images. Inspired by previous works using RoPE (Su et al., 2024) to represent the positions token maps (Ma et al., 2024; Tang al.), we propose an advancement with 2D RoPE align spatial structures of LR images and multi-scale token maps, termed Scale-aligned RoPE. For the embedding o (i,j), we split the channels into two for representing 2D positions and normalize encodings for alignment:
(i.j) E RC of token rk (3.j) scale at position
R
2
RC(((10) =
(5)
upsamplc(lookup(V.rj)
0 0
R

---

Visual Autoregressive Modeling for Image Super- Super-Resolution
We introduce a simple Diffusion Refiner, composed of lightweight MLP (Li et al., 2024a). The Diffusion Refiner solely serves to map the probability distribution of quantized residuals, and does not have the capabilities of an additional ISR model. It leverages the final scale hidden states zk as conditional control to generate a continuous distribution from noise Et.
Substituting Eq. 11 into Eq. 10 yields Eq. 12. In CFG, guiding scale Xg balances diversity and realism.
V, log log Vogog cn) + +V (o V (oggp(Iree (12)
During inference, Image-based CFG are represented as:
[(ax, x) E. - [lls
Eo (ze 4, (xxx)1
(8)
i= F(re: + Xa F(re:
(13)
where E is the sampled noise, L is the timestamp and ca(-) is the mapping function of the Diffusion Refiner. During inference, the sampled continuous tokens are combined with the predicted token maps for the final output.
where F()) is the mapping function of our VARSR.
3.3. Scaling up Database
Large-scale Data Collection. ISR methods require ample data to acquire generative priors and semantic understanding capabilities (Yu et al., 2024). Diffusion methods use Stable Diffusion (SD) (Rombach et al., 2022) trained on billions of text-image pairs for plentiful image priors. The open-source VAR base model falls short of our needs, as it could only generate 256 x 256 images and is limited in generated image quality, necessitating scaling high-quality training data for robust priors. Classical datasets often struggle to meet our high standards for image quality (e.g., ImageNet (Deng et al., 2009)) or quantity (e.g., DIV2K (Agustsson & Timofte, 2017), DIVSK (Gu et al., 2019b)). Therefore, we collect new large-scale dataset with 4 million high-quality and high-resolution images across over 3k categories, ensuring rich details and clear semantics.
3.2.4. IMAGE-BASEE CLASSIFIER- GUIDANCE
VAR, GANs, and Diffusion models all target fidelity as the optimization objective in ISR tasks, which may result in generated images being overly smooth and lacking in detail. It tends to retain distortions such as blur from LR images, leading to lower human-perceinn quality.
Messifier-FFee Guidance (CFG) (Ho & Salimans, 2022) has been widely used in T2I diffusion models, leveraging nega- tive prompts to generate more realistic images by directed guiding of probabilistic distributions. Instead of simply set- ting the condition to null (Li et al., 2024d; Ho & Salimans, 2022) or using textual prompts, we aim to better perceive distortions and low-quality factors in images, hence propos- ing Imuge-bused CFG to enhance generation quality. By learning low-quality image distributions during training, it allows us to guide the probability distribution during sam- pling towards generating higher-quality images, thereby expanding the upper bound.
Negative Samples. In Sec. 3.2.4, our VARSR utilizes neg- ative embedding as inverse control to generate low-quality images. Therefore, we provide low-quality images as nega- tive samples corresponding to negative embedding to learn low-quality distortions. We sample 50k low-quality images from various manually annotated image quality assessment (IQA) datasets (e.g., KonIQ10K (Hosu et al., 2020), CLIVE (Ghadiyaram & Bovik, 2016)) and image aesthetics assess- ment (IAA) dataset AVA (Murray et al., 2012) as negative samples added to our database.
In training, HR images are divided into high- and low- quality classes, with positive and negative embeddings Cp: Gn for control. In inference, to generate a higher-qualiny image I with condition r. and cp. the distribution
pp)) p(cplre: D(((Ir)
p(cplra)
(9)
p(Trr))= = pe)pTrr) cp) + ((ccllr))p(Theecc.
Image Preprocess. Psychophysics research suggests that the richness of details in visual content impacts human per- ception of quality and aesthetics (He et al., 2014). Therefore, instead of the commonly used random cropping, we resize images to 1.25 times the size for input and center-crop them during training. The aim is to enhance the comprehen- sive coverage of the foreground, which typically contains richer semantics and textures than the background, enabling VARSR to capture a broader range of visual attributes.
Eq. 9 can be simplified to:
p(I(Te, = p(enlre) p(cplrer (n
p(pplra)
1 p(cplra; 1)
p(
(10)
V, logg((Trra = Vr log 1 T)
+V, log Cn),
Furthermore, by applying the Bayesian formula, we can obtain:
3.4. Training Procedure
p(cplra) Cp) 1 1-plcplres I) p(culre) Cn) (11)
p(cplras I)
Our entire training process is the same as the diffusion- based models and can be divided into three-step process
5

---

Visual Autoregressive Modeling for Image Super-Resolution
Boass Perestler: Quality
Ilueaa Perogdder 10 Qutl
AON DOR
VQYAE Knosder
Eacide
Daosder
Image Tokenisation
Scale Drepont (Original)
Leae 1o0
Scale Dropour
Seafes incraner iill
Figure 4. Limitations of current full-reference metrics (egg, PSNR, SSIM, LPIPS). VARSR has generated images of higher perceptual quality for humans, yet it lags behind in certain metries.
Figure 3. Image tokenization process of VQVAE. The quantizer converts the image latent to multi-scall discrete tokens while rep- resenting the quantization loss continuous tokens.
for comprehensive evaluation. We create the synthetic validation set DIV2K- VAL by randomly cropping 3k patches from the DIV2K (Agustsson & Timofte, 2017) validation set, and for real-world evaluation, DrealSR (Cai et al., 2019) and RealSR (Wei et al., 2020) are center-cropped. Following (Wang et al., 2024a), all HR images have a resolution of 512 x 512, and LR images are 128 x 128.
for application in ISR tasks: (1) training the VAE, (2) pre- training on C21/T2I tasks, and (3) finetuning on ISR. We train our model in all three stages to acquire stronger generative prior for the ISR task.
VQVAE. In Fig. 3, the visualizations of the original VAR quantizer (Tian et al., 2024) reveal that image semanties concentrated in the final few scales. This limits ISR, as earlier scale generations may lack information for later scales. Therefore, after the initial full training of the VQ- VAF, we added an additional training stage for the quantizer. Specifically, we follow the scale random dropout strategy in previous works (Li et al., 2024b; Kumar et al., 2024), where multi-scale quantized results are randomly discarded with probability Pa- We freeze other parts and only train the quantizer. In Fig, 3, applying scale dropout preserves more semantic information in the earlier scales.
Implemental Details. We train our VARSR following the procedure in Sec. 3.4, using a GPT-2 style (Radford et al., 2019) transformer with 24 blocks as the base model and Diffussion Refiner with blocks. We accelerate training by using pretrained VAR (Tian et al., 2024). We utilize Adam (Loshchilov & Hutter, 2017) optimizer with batchsize=128, weight decay=5e-2, and learning rate=5e-5, VQVAE, C2I pretraining, and ISR finetuning run for 10k, 40k, and 20k iterations, respectively. The loss balancing coefficient À 2.0, and the dropout ratio pa is 0.1. The guid- ance scale A, linearly increases 6.0 as the scale increases. Experiments are performed 32 NVIDIA V100 GPUs.
C21 Pretraining. We first train a powerful base VAR model on the C2I task to establish robust generation pri- ors. Our training utilizes our large-scale dataset of over 3k categories, incorporating class information via start tokens and modulate layers in the transformer (Tian et al., 2024).
Metrics. reference-based metrics, including PSNR, SSIM (Wang et al., 2004) (Y channel), (Zhang et al., 2018a), and DISTS (Ding et al., 2022), are used for fidelity evaluation. FID (Heusel et al., 2017) measures the distribution distance between generated and reference images MANIQA (Yang et al., 2022), MUSIQ (Ke et al., 2021), and CLIPIQA (Wang et al., 2023) are non-reference IQA metries.
ISR Finetuning. We fine-tune the C2I pre-trained base model for the downstream ISR task to create the final VARSR. Both C2I Pretraining and ISR finetuning stages utilize the same loss with a coefficient À to balance the cross-entropy loss of tokens and diffusion loss of the refiner.
4.2. Comparison with SOTA
To verify the effectiveness, we compare our VARSR with other well-known SOTA GAN-based and Diffusion-based ISR methods', including BSRGAN (Zhang et al., 2021). Real-ESRGAN (Wang et al., 2021), SwinIR-GAN (Liang et al., 2021), DASR (Liang et al., 2022), LDM (Blattmann et al., 2022), StableSR (Wang et al., 2024a), DiffBIR (Lin et al., 2023), SinSR (Wang et al., 2024c), PASD (Yang et al., 2025) and SeeSR (Wu et al., 2024b).
4. Experiments
4.1. Experimental Setups
Datasets. We train VARSR on our large-scale dataset with negative samples using Real-ESRGAN's degradation pipeline (Wang et al., 2021) to synthesize LR-HR image pairs. Both synthetic and real-world datasets are utilized
'lll methods are tested based on their official code and models.
6

---

Visual Autoregressive Modeling for Image Super-Resolution
Table 1. Comparison with SOTA methads on synthetic and real-world benchmarks. Red and blue colors are the best and second-best.
GAN-based Real-ESR
Diffusion-based DiffBIR
AR-hased VARSR
Datasct
Metries
BSRGAN
SwinIR
DASR
LDM
StableSR
SinSR
PASD
SocSR
PSNR
24.42
24.30
23.77
24.46
21. 21.66
23,26
23.49
24.22
23.59
23.56
23,91
SSIMT
0.6164
0.6324
0.6186
0.6253
0,4752
0.5670
0.5568
0.5922
0.5899
0.5981
0.5980
L.IPS
0.3511
0.3267
0.3910
0.3696
0.4887
0.3228
0.3638
0.3429
0.3611
0.3283
0.3260
DISTS!
0.2369
0.2245
0.2291
0,2533
0.2693
0.2116
0.2177
0.2157
0.2134
0,2008
0.2218
DIV2K-Val Val
FID
50.99
44,34
44.45
57.37
55.04
28.32
34,55
42.17
39.74
28.89
35,51
MANIQAT
0.3547
0.3756
0.3411
0,3104
0.3589
0,4598
0.4101
0,4440
0.5046
0.5340
CLIPIQA
0.5253
0.5205
0.5213
0,4960
0,5570
0.6752
0.6731
0.6411
0.6573
0.0959
0.7347
M
60.18
59.76
57.21
53,96
57
65.19
65.57
61.46
66.58
68.35
71.27
PSNRT
26.38
25.68
25.88
25.47
25. 66
24.69
24.94
24.86
25.21
25.31
24.61
SSIM
0.7651
0.7614
0.7671
0.7575
0.6934
0.7090
0.6664
0.7191
0.7140
0.7284
0.7169
L
0.2656
0.2710
0.2614
0.3240
0.3367
0.3003
0.3485
0.3472
0.2986
0.2993
0.3504
DISTS!
0.2124
0,2060
0.2061
0,2267
0,2324
0.2134
0.2257
0,2500
0.2125
0,2224
0.2470
RealSR
FID1
141.25
135,14
132.80
133.33
133.34
131.72
127.59
142.31
139.42
126.21
137.55
MANIQAT
0.3763
0.3736
0.3561
0.2470
0.3375
0.4167
0.4378
0.3985
0.4418
0.5370
0.5570
CLIPIQAT
0.5114
0.4487
0.4433
0.3198
0,6053
0.6200
D.6396
0,6162
0.6019
0.6618
0.7006
MUSIQ
63.28
60.37
59,28
41,21
56.32
65,25
64,32
60.57
66.61
69.99 09.56
71,26
PSNRT
28.70
28.61
28,20
29.75
27.78
27.87
26.57
28.26
27.45
28.13
28,16
SSIM
0.8028
0.8052
0.7983
0.8262
0.7152
0.7427
0.6516
0.7443
0.7539
0.771
0.7652
LPIPS
0.2858
0.2819
0.2830
0.3099
0.3745
0.3333
0.4537
0.3743
0.3331
0.3142
0.3541
DISTS
0.2144
0.2089
0.2103
0.2275
0.2417
0.2297
0.2724
0.2495
0.2322
0.2230
0,2526
DReatSR
FIDL
155.62
147.66
146.38
155.36
164.87
148.18
160.67
173.42
173.40
147,00
155.87
MANIQAT
0.3441
0.3435
0.3311
0,2809
0.3342
0.3897
0.4602
0.3843
0.4551
0.5077
0,5362
CLIPIQAT
0.5061
0.4525
D.4522
0,3813
0.5984
0.6445
0.6302
0.6365
0.6893
0.7240
MUSIQ
57.16
54,27
53.01
42,41
51.37
58.72
61.06
55.28
63.69
64.75
68.15
Quantitative Comparisons. The quantitative comparisons are shown in Tab. 1. First, VARSR outperforms other SOTA methods by a wide margin in three no-reference IQA metrics: MAINIQA, CLIPIQA, and MUSIQ, reflecting the capability to produce high-quality and realistic images. Second, in reference-based metrics (e.g., PSNR, SSIM), our VARSR approximates that of the best diffusion-based methods, yet still lags behind GAN methods. This is due to the trade-off between realism and fidelity, as VARSR and diffusion methods generate more textures and details, which may reduce fidelity metrics, especially for lower-quality HR images.
Table 2. Complexity comparison.
Methods
Params
Steps
Inference Time
StableRR
1409. IM
200
18.70.
PASID
1900.4M
20
6.07s
DiffBIR
1716.7M
50
5.85s
SeeSR
2283.7M
50
7.24s
VARSR(OUrS)
110199
10
0.59s
capability, aligning with the IQA metrics in Tab. 1. For ex- ample, in the 2nd and 4th row, VARSR distinguishes itself as the sole method that can correctly understand the semantics and generate houses and leaves with clear textures. In the 1st, 3rd, 5th, and 6th rows, VARSR demonstrates superior structural fidelity over other methods, generating images with well-defined structures and clear semantics. Addition- ally, VARSR excels in creating intricate animal fur, detailed leaves, and vivid textures. The results highlight VARSR's s robust ability to produce realistic and semantic-preservat images, even under severe degradation of LR images.
In Fig. 4, VARSR's restoration exhibits higher quality in hu- man perception, yet lags behind in certain reference-based metrics, as overly smoothed low-quality images tend to score better. This phenomenon highlights the limitations of current reference-basee metrics, which have been confirmed in many studies (Wang et al., 2024a; Wu et al., 2024a). The- oretical derivations have verified the inherent contradiction between fidelity and quality (Blau& & Michaeli, 2018).
Complexity Analysis. Diffusion models have longer infer- ence times due to multi-step noise sampling, while VARSR generates token maps at different scales with low iteration counts, ensuring minimal inference time. Additionally, with fewer token counts at early scales, the complexity of earlier steps is very low, unlike diffusion methods with consistent complexity per iteration. In Tab. 2, compared to diffusion models, VARSR only requires 0.59s to generate an image, which is 10.1% of the second-ranked DiffBIR. In addition, other optimization methods can also be applied to VARSR
Qualitative Comparisons. In Fig. 5, we visualize some of the ISR results from the test set. First, our VARSR consis- tently generates images with rich details and clear semantic information, showcasing the strong generative prior capa- bility of autoregressive methods. Second, the image quality generated by GAN-based methods lags behind diffusion- based methods and our AR-based VARSR, highlighting the significance of generative priors. Third, VARSR also outperforms diffusion- methods in detail generation
7

---

Visual Autoregressive Modeling for Image Super-Resolution Resolution
LR
HR
BSRGAN
Real-ESRGAN
StableSR
PASD
SecSR
VARSR(Ours)
Figure 5. Qualitative comparisons with different SOTA methods. Zoom in for a better view.
with cross-attention (CA) (Yang et al., 2025). In Tab. 3, Prefix Tokens mode outperforms in all metrics. In Fig. 6, our method is the only one capable of generating the correct traffic light colors and clear text. These results showease the effectiveness of the proposed Prefix Tokens pattern in ex- tracting sufficient LR information for conditional guidance.
|1/
||/
Blieetly AM
ContralNet
ComtanNaaiCAA
Profis Iskenn(Ours)
Scale-aligned RoPE We validate the effectiveness of SA- RoPE by contrasting with two scenarios: (1) Utilizing the original APE; (2) Applying SA-RoPE solely on discrete tokens. In Tab. 4, following the gradual application of SA-RoPE, fidelity metrics have shown significant improve- ment. In Fig. 7, it is evident that applying SA-RoPE can better preserve the spatial structure, generating clear text and architectural details faithful to the original image.
Figure Effectivess the Condition Mode.
to further improve efficiency, such as using knowledge dis- tillation to reduce the number of scales for inference.
4.3. Ablation Study
LR Condition. We use the form of Prefix Tokens to provide conditional control from LR images, comparing this mode with the other three LR condition modes: (1) Adding LR features directly in the transformer; (2) ControlNet (Zhang al., 2023a) with 1/2 depth; (2) ControlNet combined
Diffasion Refiner. We use the Diffission Refiner to gener- ate continuous tokens as the prediction of quantization resid- uals, supplementing the discrete tokens of the autoregressive transformer. Tab. 5, after adding the Diffusion Refiner, both fidelity and image quality have significantly improved.
8

---

Visual Autoregressive Modeling for Image Super- Resolutin
Table Ablation on the Condition Mode.
Condition
DreaSR
RealSR DISTS)
Mode
SSIM
I.PISS]
DISTSL
MANIQAT
MUSIQ1
SSIMT
LPIPS
MANIQA1
MUSIQ1
Directly Add
0,6902
0.4011
0,3010
0,5188
66.46
0,6712
0.3742
0,2553
0.5221
69.90
ControlNet
0.7188
69.70
ControlNet-CA
0.7314
0.3784
0.2630
0.5224
65.97
0.7084
0.3577
0.2552
0.5270
69.83
Prefix: Tokens
0.7682
0.3541
0.2526
0.5362
68.15
0.7169
0.3504
0,2470
0.5570
71.26
Table 4. Ablation on the Scale-aligned RoPE.
Exp
DrealSR
RealSR
Condition
X
X
would
rotec
Rould
rotec
Rould
rotec
Rould
rotec
Rould
rotec
Discrete
ct
stthe
tte
rt
zt
the ouenm J tne obu.auu.
the ohea
the olcaye
SSIM
0,7424
0,7603
0.7652
0.6992
0.7039
0.7169
proteetion
whie protection
protect
Whie protection
LPIPS
0.3691
0.3645
0.3541
0.3654
0.3672
0.3504
ensing can weing can
wiiing can
wiinn canbe
ansing
DISTST
0.2617
0.2642
0.2526
0.2552
0.2576
0.2470
Ba wibh
ANry with D.eemer saken
Ruft with
MANIQAt
0.5238
0.5490
0.5362
0.5307
0.5555
0.5570
MUSIQ1
68.14
68.41
68.15
71,08
71.32
71.26
Figure 7. Effectivess of the Scale-aligned RoPE.
Table 5. Ablation on the Diffusion Refiner.
DreaiSK w/o Refiner
RealSR
Metrics
w/ Refiner
w/o Reliner
w/ Reliner
0.7583
0.7652
0.7118
0,7169
LPIPS
0.3532
0,3541
0,3515
0.3504
DISTS
0.2539
0.2526
MANIQAT
0.5297
0,5362
0,5399
0,5570
MUSIQU
67.99
68.15
70.74
71.26
Table 6. Ablation on the Image-based based CFG.
Stemn Mehe
Continato Jokeer
Wibore TNlen Takeo
DrealSR CFG
RealSR
Metries
wlo
w/ CFG
Figure 8. Visualization of Continuous Tokens and Discrete Tokens.
SSIM
0.8004
0.7652
0.7436
0.7169
LPIPS
0.2961
0.3541
0.2977
0,3504
DISTS.
0.2161
0.2526
0.2103
0,2470
MANIQA
0,4052
0.4326
MUSIQU 59.70
67.63
66.32
71.26
In Fig. 8, we visualize discrete tokens and continuous to- kens, showing that continuous tokens can effectively capture high-frequency details lost during quantization, resulting in images with richer semantic textures. A lightweight network mapping, and a larger
1.R
Fideliy
Guidance Scale incresses
Reslinic
suffices for the discrete-to-comtinee refiner does not yield significant gains (Li et al., 2024a).
Figure 9, Effect of the Image-based CFG with guidance scale X..
Image-based CFG. In Sec. 3.2.4, we introduce the Image- based CFG to generate more realistic and higher-qualh images. Tab. shows a notable enhancement in percep- tual quality metrics, after incorporating the CFG. In Fig. 9, we show visualizations where images become clearer as the guidance scale >, increases. CFG results in signifi- cantly richer textures in the generated images, substantially improving perceptual quality to meet human preferences while maintaining correct semantics. However, excessive X, values may introduce artifacts not present the original image. Therefore, our Image- based CFG achieves a balance between fidelity and realism by controlling Xş. In practice,
X 6.0 is optimal, and it should increase linearly with scale for finer texture details in larger scales.
5. Conclusion
We explored VAR in ISR tasks and proposed the VARSR framework. To ensure pixel-level fidelity and realism, we made improvements in LR conditioning, structure repre- sentation, quantization prediction, and CFG guidance. We collected large-scale dataset and designed a training pro- cess. Extensive experiments validate the performance of VARSR in generating high-fidelity and high-quality images.

---

Visual Autoregressive Modeling for Image Super-Resolute
Impact Statement
Chen, L., Wang, Z., Ren, S., Li, L., Zhao, H., Li, Y., Cai, Z.., Guo, H., Zhang, L.,, Xiong, Y., et al. Next token predic- tion towards multimodal intelligence: A comprehensive survey. arXiv preprint arXiv:2412.
This paper introduces the visual autoregressive into the im- age super-resolution field. Therefore, any potential societal consequences or impacts related ISR tasks apply here, as work introduces ideas that enhance ISR tasks with high efficiency and practicality.
2024b.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, Icee, 2009.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L.,, Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S.,
Ding, K., Ma, K., Wang, S., and Simoncelli, E. P. Image quality assessment: Unifying structure and texture sim- ilarity. IEEE Trans. Pattern Anal. Mach. Intell., 44(5): 2567-2581, 2022.
Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Agustsson, E. and Timofte, R. NTIRE 2017 challenge single image super-resolution: Dataset and study. In CVPR Workshops, pp. 1122-1131. IREE Computer Soci- cty, 2017.
Dong, C., Loy, C. C., and Tang, X. Accelerating the super- resolution convolutional neural network. In ECCV (2), volume 9906 of Lecture Notes in Computer Science, pp. 391-407. Springer, 2016.
Aithal, S. K., Maini, P., Lipton, Z., and Kolter, J. Z. Un- derstanding hallucinations in diffusion models through mode interpolation. Advances in Neural Information Pro- cessing Systems, 37:134614-134661 2024.
Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883 2021.
Bell-Kligler, S., Shocher, A., and Irani, M. Blind super- resolution kereel estimation using an internal-gan Ad- vances in Neural Information Processing Systems, 32, 2019.
Gadre, S. Y., Ilharco, G., Fang, A., Hayase, J., Smyrnis, G., Nguyen, T., Marten, R., Wortsman, M., Ghosh, D., Zhang. J., Orgad, E., Entezari, R., Daras, G., Pratt, S. M., Ramanujan, V., Bitton, Y., Marathe, K., Mussmann, S., Vencu, R., Cherti, M., Krishna, R., Koh, P. W., Saukh, O., Ratner, A. J., Song, S., Hajishirzi, H., Farhadi, A., Beaumont, R., Oh, S., Dimakis, A., Jitsev, J., Carmon, Y., Shankar, V., and Schmidt, L. Datacomp: In search of the next generation of multimodal datasets. In NearIPS, 2023.
Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High- resolution image synthesis with latent diffusion models. In CVPR, pp. 10674-10685. IEEE, 2022.
Blau, Y. and Michaeli, T.
The perception-distortion trade- off. In Proceedings of the IEEE conference computer vision and patlern recognition, pp. 0228-6237, 2037,,200 2018.
Ghadiyaram, D. and Bovik, A. C. Massive online crowd- sourced study of subjective and objective picture quality. IEEE Trans. Image Process., 25(1):372-38) 2016
Cai, J., Zeng, H., Yong, H., Cao, Z., and Zhang, L. Toward real-world single image super-resolution: A new bench- mark and new model. In ICCV, pp. 3086-3095. IEEE, 2019.
2016.
Gu, J., Lu, H., Zuo, W., and Dong, C. Blind super-resolution with iterative kernel correction. In CVPR, pp. 1604-1613, Computer Vision Foundation / IEEE, 2019a.
Chang, H., Zhang, H., Jiang, L., Liu, C., and Freeman, W. T. Maskgit: Masked generative image transformer. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 11315-11325, 2022.
Gu, J., Wang, Y., Zhang, Y., Zhang, Q. Zhang, D., Jaitly, N., Susskind, J., and Zhai, S. Dart: Denoising autoregressive transformer for scalable text-to-image generation. arXiv preprint arXiv:2410.088.52 2024.
Chen, C., Shi, X., Qin, Y., Li, X., Han, X., Yang, T., and Guo, S. Real-world blind super-resolution via feature matching with implicit high-resolution priors. In Pro- ceedings of the 30th ACM International Conference on Multimedia, pp. 1329 1338, 2022.
Gu, S., Lugmayr, A., Danelljan, M., Fritsche, M., Lamour, J., and Timofte, R. DIV8K: diverse 8k resolution im- age dataset. In ICCV Workshops, pp. 3512-3516. IEEE, 2019b.
Chen, C., Mo, J., Hou, J., Wu, H., Liao, L., Sun, W., Yan, Q. and Lin, W. Topiq: A top-down approach from se- mantics to distortions for image quality assessment, IEEE Transactions on Image Processing, 2024a.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
10

---

Visual Autoregressive Modeling for Image Super-Resolutee
He, L.. Gao, F., Hou, W., and Hao, L. Objective image quality assessment: a survey. Int. J. Comput. Math., 91 (11):2374-2388, 2014.
Li, T., Tian, Y., Li, H., Deng, M., and He, K. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37:56424- 56445, 2024a.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by two time-scale update rule converge to local nash equilibrium. In NIPS, pp. 6626-6637, 2017.
Li, X.. Qiu, K., Chen, H., Kuen, J., Gu, J., Raj. B., and Lin, Z. Imagefolder: Autoregressive image generation with folded tokens. arXiv preprini arXiv:2410.01756 2024b.
Ho, J. and Salimans, T. Classifier-free free diffusion guidance. arXiv preprint arXiv:2207. 0202207..///9,, /2598, 2022.
Li, X., Qiu, K. Chen, H., Kuen, J., Lin, Z., Singh, R., and Raj. B. Controlvar: Exploring controllable visual au- toregressive modeling. arXiv preprint ar XXVv:2406.09750, 2024c.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. In NeurIPS, 2020.
Hosu, V., Lin, H., Szirányi, T., and Saupe, D. Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Trans. Image Process., 29:4041-400 4056, 2020.
Li, Y., Zhang, K., Liang, J., Cao, J., Liu, C., Gong, R., Zhang, Y., Tang, H., Liu, Y., Demandolx, D., Ranjan, R., Timofte, R., and Gool, L. V. LSDIR: large scale dataset for image restoration. In CVPR Workshops, pp. 1775-1787. IEEE, 2023.
Huang, Y., Li, S., Wang, L., Tan, T., et al. Unfolding the alternating optimization for blind super resolution. Advances in Neural Information Processing Systems, 33; 5632-5643, 2020.
Li, Z., Cheng, T., Chen. S., Sun, P., Shen, H., Ran, L.,, Chen, X., Liu, W., and Wang, X. Controlar: Controllable image generation with autoregressive models. arXiv preprint arXiv:2410.02705, 2024d.
Kang, L., Ye, P., Li, Y., and Doermann, D. Convolutional neural networks for no-reference image quality assess- ment. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1733-1740, 2014.
Liang. J., Cao, J., Sun, G., Zhang, K., Gool, L. V., and Tim- ofte, R. Swinir: Image restoration using swin transformer. In ICCVW, pp. 1833-1844. IEEE, 2021.
Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks. Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410 2019.
Liang. J., Zeng. H., and Zhang, L. Efficient and degradation- adaptive network for real-world image super-resolution. European Conference on Computer Vision, pp. 574- 591. Springer, 2022.
Kawar, B., Elad, M., Ermon, S., and Song, J. Denoising diffusion restoration models. Advances in Neural Infor- mation Processing Systems, 35:23593-23600 2022.
Lin, X., Hc, J., Chen, Z., Lyu, Z., Fei, B., Dai, B., Ouyang, W., Qiao, Y., and Dong, C. Diffbir: Towards blind im- age restoration with generative diffusion prior. CoRR, abs/2308. 15070, 2023.
Ke, J., Wang, Q.. Wang, Y., Milanfar, P., and Yang, Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5148-515 2021.
Loshchilov, I. and Hutter, Fixing weight decay regular- ization in adam. 2017.
Kim, S., Jin, C., Diethe, T., Figini, M., Tregidgo, H. F., Mul- lokandov, A., Teare, P., and Alexander, D. C. Tackling structural hallucination in image translation with local diffusion. In European Conference on Computer Vision, pp. 87-103. Springer, 2025.
Ma, X., Zhou, M.. Liang. T., Bai, Y., Zhao, T., Chen, H., and Jin, Y. Star: Scale-wise text-to-image genera- tion via auto-regressive representations. arXiv preprint 206.1:2406.10793 10797, 2024.
Kumar, R., Seetharaman, P., Luebs, A., Kumar, I., and Ku- mar, K. figh-fiielity audio compression with improved rvggan. Advances in Neural Information Processing Sys- tems, 36, 2024.
Mou, C., Wang, X., Xie, L.. Wu, Y., Zhang, J., Qi. Z., and Shan, Y. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelli- gence, volume 38, pp. 4296-4304, 2024.
Lee, D., Kim, C., Kim, S., Cho, M., and Han, W.S. Autore- gressive image generation using residual quantization. In Proceedings of the IEFE/CVF Conference on Computer Vision and Puttern Recognition, pp. 11523-11532. 2022.
Murray, N., Marchesotti, L.,, and Perronnin, F. AVA: A large- scale database for aesthetic visual analysis. In CVPR, pp. 2408-241 IEEE Computer Society, 2012.

---

Visual Autoregressive Modeling for Image Super-Resolution
Narasimhaswamy, S., Bhattacharya, U., Chen, X., Dasgupta, I, Mitra, S., and Hoai, M. Handiffuser: Text-to-image generation with realistic hand appearances. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 2468-2479, 2024.
Shazeer, N. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv: /91 1.02 2019.
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:1 068:127063, 2024.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 4195-4205, 2023.
Su, S., Yan, Q. Zhu, Y., Zhang, C., Ge, X., Sun, J., and Zhang, Y. Blindly assess image quality in the wild guided by a self-adaptive hyper network. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 3667-3676, 2020.
Qu Y., Yuan, K., Zhao, K., Xie, Q. Hao, J., Sun, M., and Zhou, C. Xpsr: Cross-modal priors for diffusion- based image super-resolution. In European Conference Computer Vision, pp. 283-303. Springer, 2025.
Tang, H., Wu, Y., Yang, S., Xie, E., Chen, J., Chen, J., Zhang, Z., Cai, H., Lu, Y., and Han, S. Hart: Efficient visual generation with hybrid autoregressive transformer. In The Thirteenth International Conference on Learning Representations.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Tian, K., Jiang, Y., Yuan, Z., Peng, B., and Wang, L. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839-84865, 2024.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, Learning transferable visual models from natural language supervision. In ICML, volume 139 of Proceedings of Machine Learning Research, pp. 8748-8763. PMLR, 2021.
Timofte, R., Agustsson, E., Van Gool, L., Yang. M.-H., and Zhang, L. Ntire 2017 challenge single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 1 14--125 2017.
Reddy, M. D. M., Basha, M. S. M., Hari, M. M. C., and Penchalaiah, M. N. Dall-e: Creating images from text. UGC Care Group I Journal, 8(14)711-7, 2021.
Touvron, H.,, Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint 2rXiv:2302./333 13971, 2023.
Roheda, S. Cart: Compositional auto-regressive transformer for image generation. arXiv preprint arX1v:241..10180 2024.
Van Den Oord, A., Vinyals, O., et al. Neural discrete rep- resentation learning. Advances in neural information processing systems, 30, 2017.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffiusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Wang. J., Chan, K. C., and Loy, C. Exploring clip for as- sessing the look and feel of images. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pp. 2555-2563, 2023.
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., and Norouzi, M. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713 2022.
Wang. J., Yue, Z., Zhou, S., Chan, K. C., and Loy, C. C. Exploiting diffusion prior for real-world image super- resolution. International Journal of Computer Vision, 132(12):5929-5949 2024a.
Schuhmann, C. Clip+mlp aesthetic score predictor. https: //github. chrr httm/cchhitttpiiccchmeeeet improved aesthetic aesthetie-prediicoor 2022.
2022.
Wang, X., Xie, L., Dong, C., and Shan, Y. Real-esrgan: Training real-world blind super-resolution with pure syn- thetic data. In ICCVW, pp. 1905-1914. IEEE, 2021.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S., Crow- son, K., Schmidt, L., Kaczmarczyk, R., and Jitsev, LAION-SB: an open large-scale dataset for training next generation image-text models. In NeurIPS, 2022.
Wang. X., Zhang, X., Luo, Z., Sun, Q. Cui, Y., Wang, J., Zhang, F., Wang, Y., Li, Z., Yu, Q., et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869, 2024b.
12

---

Visual Autoregressive Modeling for Image Super-Resolution
Wang, Y., Yang, W. Chen, X., Wang, Y., Guo, L.,, Chau, L.-P., Liu, Z.,, Qiao, Y., Kot, A. C., and Wen, B. Sinsr: diffusion-bassee image super-resolution in single step. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 25796-25805, 2024c.
Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. Sig- moid loss for language image pre-training. In ICCV, pp. 11941-11952. IEEE, 2023.
Zhang, K., Liang, J., Gool, L. V., and Timofte, R. Designing practical degradation model for deep blind image super- resolution. In ICCV, pp- 3771-4780. IEEE, 2021.
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4): 600-612, 2004.
Zhang, L., Rao, A., and Agrawala, M. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023a.
Wei, P., Xie, Z., Lu, H., Zhan, Z., Ye, Q., Zuo, W., and Lin, L. Component divide-and-conquee for real-world image super-resolution. In ECCV (8), volume 12353 of Lecture Notes in Computer Science, pp. 101-117. Springer, 2020.
Zhang, Q. Dai, X., Yang, N., An, X., Feng, Z., and Ren, Var-clip: Text-to-image generator with visual auto- regressive modeling. arXiv preprint arXiv:2408. 181, 2024.
Wu, R., Yang, T., Sun, L., Zhang, Z., Li, S., and Zhang, L. Seesr: Towards semantics-aware real-world image super- resolution. In CVPR, pp. 04446725467. IEEE, 2024a.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR. pp. 586-595, Computer Vision Foundation IEEE Computer Society, 2018a.
Wu, R., Yang, T., Sun, L., Zhang, Z., Li, S., and Zhang, L. Seesr: Towards semantics-aware real-world image super- resolution. In Proceedings of the IEEE/CVF conference compuler vision and paltern recognition, pp. 25456- 25467, 2024b.
Zhang, R., Gu, J., Chen, H., Dong, C., Zhang, Y., and Yang. W. Crafting training degradation distribution for super-
the eecuracy--eneralizattie trade-off in resolution. In ICML, volume 202 of Proceedings of Machine Learning Research, pp. 41099. PMLR, 2023b.
Yang. Q. Chen, D., Tan, Z., Liu, Q. Chu, Q. Bao, J., Yuan, L., Hua, G., and Yu, N. HQ-50K: A large- scale, high-quality dataset for image restoration. CoRR, 0bs/2306.05349 2023.
Yang. S., Wu, T., Shi, S., Lao, S., Gong, Y., Cao, M., Wang, J., and Yang, Y. MANIQA: multi-dimension attention network for no-reference image quality assessment. In CVPR Workshops, pp. 1199-1199. IEEE, 2022.
Zhang, Y., K., Li, K., Wang, L., Zhong, B., and Fu, Y. Image super-resolution using very deep resid- ual channe attention networks. In ECCV (7), volume 11211 of Lecture Notes in Computer Science, pp. 294- 310. Springer, 2018b. URL https: /ooii org/10. 1007/978- 3- 030 01 22342232222 18.
Yang, T., Wu, R., Ren, P., Xie, X., and Zhang. L. Pixel- aware stable diffusion for realistic image super-resolution and personalized stylization. In European Conference on Computer Vision, pp. 74-91. 91. Springer, 2025.
Yao, Z., Li, J., Zhou, Y., Liu, Y., Jiang, X., Wang, C., Zheng, F., Zou, Y., and Li, L. Car: Controllable autore- gressive modeling for visual generation. arXiv preprint arXiv:2410.04671, 2024.
Yu, F., Gu, J., Li, Z., Hu, J.. Kong, X., Wang, X., He, J., Qiao, Y., and Dong, C. Scaling up to excellence: Prac- ticing model scaling for peoto-realistic image restoration in the wild. Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 25669-25680, 2024.
Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., Hutchinson, B., Han, W., Parckh, Z., Li, X., Zhang, H., Baldridge. J., and Wu, Y. Scaling autoregressive models for content- rich text-to-image generation. Trans. Mach. Learn. Res., 2022, 2022.
13

---

Visual Autoregressive Modeling for Image Super-Resolution Resolution
A. Implementation Details
A.I. Details VARSR Framework
Table 7. Reconstruction results of the VQVAE on RealSR dataset.
Methods
Compression ratio
PSNR
SSIM
LPIPSI.
DISTS
FID4
MANIQA f
CLIPIQA
VQVAE (w/ quantization) VQVAE (w/o quantization)
16
34.33
0.9380
0.0428 0.0752
0.0484
17.19
0.3432
0.4615
16
29.86
0.9006
0.0688
34.27
0.3172
0,4327
VQVAE Tokenizer. We employ VQVAE (Van Den Oord al., 2017) the discrete tokenizer for VARSR, following the basic settings of VAR (Tian al., 2024). The downsampling factor of VQVAE is 16x, and all scales share the same codebook with size V 4096. Our VQVAE consists of two training stages: the initial full training and the application of the scale dropout strategy for the quantizer. Due to the Diffusion Refiner in our VARSR, which can predict quantization loss to some extent, we follow the alternating training strategy of HART (Tang et al.) for the initial full training. This involves incorporating continuous tokens for reconstruction with probability of 50% (i.e., bypassing the quantization process), ensuring that VQVAE can reconstruct images in both scenarios. The reconstruction results both scenarios are illustrated Tab. 7.
Image Encoder. We a pyramid-style image encoder extract information from degraded LR images and downsample it by factor of 16 obtain 32 x 32 feature space vectors. Therefore, the image encoder consists of layers, each composed of two convolutional layers and an activation SiLU layer, including the last convolutional layer with stride of 2 2 to reduce the size of the feature maps by half. All parameters of our image encoder are trainable.
Autoregressive Transformer. We utilize Transformer based on the standard GPT-2 (Radford et al., 2019) architecture with a depth of 24 layers and a width of 1536. The modulate layer adopts the standard AdaLN (Peebles & Xie, 2023) form, which is added separately in each block to generate the offsets. The tokens are divided into 10 scales corresponding to resolutions ranging from 16 x 512 512, with prefix tokens from LR added at the beginning. In the attention calculation, queries and keys reflect spatial positional information through our Scale-aligned RoPE.
During inference, KV-cache (Shazeer, 2019) is utilized to enhance the speed. Our VARSR supports the generation of higher-resolution images by concatenating them in a tiled manner, which is completely consistent with the diffusion-based method. Specifically, we uniformly divide the high-resolution image to be generated into overlapping grids. VARSR generates SR results for each grid separately and then tiles them together to obtain the complete image restoration result. We can batch process different grids parallel to accelerate the inference process, significantly increasing computational speed.
Difffusion Refiner. Our Diffiusion Refiner adheres to the specifications of (Li et al., 2024a), naturally supporting applications CFG form. The Diffusion Refiner comprises highly lightweight residual MLP as (He et al., 2016), with each residual block consisting of LayerNorm, linear layer, an activation SiL.U layer, and another linear layer, merging with residual connection. Our model consists of 6 layers, with channel dimension of 1024. The hidden states from the autoregressive transformer serve conditional controls and are introduced alongside timestamps T through the modulate module. Our noise schedule follows cosine shape, with 1000 steps during training, and is resampled with 10 steps during inference, thus requiring minimal inference time. The parameter count the lightweight Diffusion Refiner is less than 40M, accounting for 3% of the 1.IB model.
A.2. Details of Large-scale Dataset
Table 8. Comparison of our large-scale dataset with other well-known datasets.
Dataset
Scale
Resolution
MANIQA f
CLIPIQAt
MUSIQU
Aesthetic Scoret
DIV2K
800
2k
0.43
0.62
71.15
5.57
Flickr2K
2650
2k
0.41
0.58
70.83
5.22
Large-scale data
Over 4M
2k
0.48
0.66
72.31
5.62
14

---

Visual Autoregressive Modeling for Image
Dataset Collection
@halligoceneerd Control
Scaling-up Dataset 4M high-quudlity images
Image pool
4M high- quailty images 50% lon-quailty onages
Lun-quality imsugss
Segative Samples Selection
IQATAA Datasets Kodid-TOk CLIVE FLIVE AYA...
Figure 10. Database Collection. We collect large-scale, high-quality dataset, and select 50k low-quality images from IQA and IAA datasets as negative samples, each with different quality-centeree controls attached.
collect billions of images from public datasets (e.g., LAION (Schuhmann et al., 2022), DataComp (Gadre et al., 2023)) and internal datasets. Employing progressive filtering and semantic balance strategy, we have constructed dataset comprising million high-quality, high-resolution images.
Progressive Filtering. Images progressively filtered using the following sub-metrics, with thresholds set for each: image metadata (including resolution, aspect ratio, and bits per pixel), IQA scores (comprising MANIQA (Yang et al., 2022), MUSIQ (Ke et al., 2021), CLIPIQA (Wang et al., 2023), and Aesthetic score (Schuhmann, 2022)), and texture richness (evaluated by the ratio of the power spectrum of high-frequency components (Yang et al., 2023) and the detection of blurred and flat regions (Li et al., 2023)). We establish the threshold for each metric at the top 70th percentile of its distribution within the DIV2K (Agustsson Timofte, 2017) and Flickr2K (Timofte al., 2017) datasets, ensuring that the images exhibit abundant details and high-frequency textures.
Table 9. Domain coverage of our harge-scale dataset.
Categories
Indoor
Outdoor
Nature
Human
Plant
Object
Animal
Text
Food
Cartoon
Others
Rates
10%
9%
13%
28%
8%
9%
6%
4%
8%
2%
3%
Semantic Balance. To achieve diversity and balance of images across various domains, we implement semantic clustering based on the CLIP (Radford et al., 2021) and SigLIP (Zhai al., 2023) models, and quantitatively select sufficient number of images according to predefined semantic categories, while specifically supplementing those categories with fewer images. As shown in the Tab. 9, we ensure that the dataset covers wide range of category scenes and a relatively balanced proportion (Scenes with a broader semantic scope correspond to higher proportion of images). The scenes include portraits, people, food, animals, natural landscapes, cartoons, cityscapes, indoor and outdoor scenes, thereby ensuring comprehensive coverage of visual concepts and richness in scene content.
B. Additional Experimental Results
B.1. Ability of Base Model
Table 10. Comparison of the base generative model on the C2I task.
Methods
Dataset
MANIQA f
CLIPIQAt
MUSIQ1
Original VAR
ImageNet
0.3250
0.5496
60.92
Original VAR
Large-scale data
0.4852
0.7015
72.74
VARSR
Large-scale data
0.5634
0.7314
74.32
To enhance the generative capabilities of the base model in the C2I task for downstream task migration, we pre-train the VAR model on our large-scale, high-quality dataset. In this section, we compare the capabilities of our VARSR base model
15

---

Visual Autoregressive Modeling for Image Super- Resolutin
Original VAR on ImageNet
VARSR Large-scale data
Figure 11. Visualization of C21 generation results from closely related categories.
trained on our large-scale dataset with the original VAR model (Tian et al., 2024) trained on ImageNet (Deng et al., 2009). As we are particularly interested the model's ability to generate detailed textures and ensure that it can still produce richly detailed images for downstream ISR task migration, we use non-reference IQA metrics for evaluation. In addition, is unfair to reference metries such FID for testing due the differences in datasets.
Quantitative results shown in Tab. 10, and in Fig. 11, visualize some generated results from closely related categories. After using our high-quality large-scale dataset, the generated images show significant improvements in both objective metrics and subjective human observations. Furthermore, by incorporating our improvements (e.g.. Scale-aligned RoPE, Diffusion Refiner) into the original VAR model, i.e., using the VARSR model, the quality further improved. This validates the effectiveness our enhancements for original C2I task. The results above validate the capabilities of our base model, which can generate images with rich details and clear semantics. It possesses strong generative prior that can be leveraged by ISR tasks.
B.2. Comparisons with SOTA Methods
Table 11. Results of user study on real-world images.
Methods
BSRGAN
Real ESRGAN
StableSR
PASD
SeeSR
VARSR(Ours)
Selection Rates
0.35%
0.7%
3.5%
17.3%
21.05%
57.1%
User Study. In order to comprchensively assess the performance of our VARSR in real-world scenarios. we conduct user study on 100 randomly sampled LR real-world images from DreatSR (Cai et al., 2019) and RealSR (Wei et al., 2020). We compare our VARSR with five other GAN-based and diffusion-based ISR methods, including BSRGAN (Zhang et al., 2021), Real-ESRGAN (Wang et al., 2021), StableSR (Wang et al., 2024a), PASD (Yang et al., 2025), and SeeSR (Wu et al., 2024a). For each image, participants were presented with both the LR image and the restoration results of all ISR methods, and were then asked to indicate their choice for the best ISR result for the LR image. We invited 20 visual researchers to participate in the user study, and in total, we obtained 20 100 selection results. As shown in Tab. 11, our VARSR achieves the highest selection rate of 57.1 %, far surpassing other methods, demonstrating the powerful capability of VARSR in real-world scenarios to generate realistic images that align with human aesthetics.
Comparisons Real-world Images. To evaluate the performance of our VARSR in in-the-wild scenarios, test different approaches on the RealL R200 dataset (Wu et al., 2024a), which comprises 200 real-world images collected from previous studies (Lin et al., 2023; Wang et al., 2021) and from the internet. RealLR200 dataset contains many highly degraded real-world images, such as historical photographs and extreme compression artifacts. Due to the absence available reference HR images, we utilize only three no-reference IQA metrics: MANIQA (Yang et al., 2022), MUSIQ (Ke et al., 2021), and CLIPIQA (Wang et al., 2023).
16

---

Visual Autoregressive Modeling for Image Super-Resolution
Table 12. Quantitative comparison with SOTA methods on RealLR200 dataset with no reference images. Red and blue colors are best and second-bes performance.
GAN-based Real-ESR
Diffusion-hasee DiffBIR
AR-based VARSR
Dataset
Metrics
BSRGAN
SwinIR
LDM
StableSR
PASD
SccSR
MANIQAt
0.3671
0.3633
0.3741
0.3049
0,3688
0,4288
0,4295
0.4844
0,5177
RealLR200
CLIPIQAT
0.5698
0.5409
0,5596
0.5253
0.5935
0.6452
0.6325
0.6553
0.7513
MUSIQU
64.87
62.96
63.55
55.19
63.29
62.44
66.50
68.37
71.92
As shown in Tab. 12, our VARSR achieves the best performance among all metrics, which is consistent with the results on other synthetic and real-world datasets. In Fig. 15, we visualize some ISR results, demonstrating that VARSR can generate images with more realistic details compared other methods. The results above validate the strong restoration capability of VARSR in real-world scenarios, showcasing its practical application value.
Table 13. Quantitative comparison with SOTA methods on other quality-based metrics. Red and blue colors are the best and second-bog performance.
GAN-based Real-ESS
Diffusion- based DifiBIR
AR- based
Dataset
Metrics
BSRGAN
SwinIR
LDM
StableSR
PASD
SeeSR
VARSR
CNNIQA
0.5492
0,5652
0,5402
0.5579
0,6274
0,6413
0.6269
0,6613
0.6661
DIV2K-Val Val
HyperlQAt
0.5682
0.5586
0.5235
0.5225
0,6100
0.6164
0.6158
0.6666
0.7031
TOPIQ1
0.5413
0,5182
0.4796
0.4695
0.5923
0.6105
0.6165
0.6793
0.7020
CNNIQA 0.5513
0,5624
0.5281
0.5637
0,6029
0.6077
0.5938
0.6594
0.6692
RealSR
HyperlQAt
0.5617
0.5231
0.5093
0.4936
0.5703
0,5690
0.6001
0.6746
0.7038
TOPIQT
0,5502
0,5137
0,4882
0,4762
0,5579
0,5580
0,5920
0.6854
0.6991
CNNIQA
0.4989
0.4849
0.5017
0.5367
0.5518
0.6025
0.5794
0.6132
0.6445
DRealSR
HyperlQAt
0.5305
0,4938
0,5074
0,5050
0.5537
0.5992
0.6008
0.6583
0.6866
TOPIQI
0.5058
0.4622
0.4694
0,4807
0,5330
0.5831
0.5963
0,6534
0.6800
Additional IQA Metrics. In order to further validate VARSR's s ability to generate high-quality images that conform to real distributions, we select a broader range of IQA metrics to evaluate VARSR, including CNNIQA (Kang et al., 2014), HyperIQA (Su et al., 2020), and TOPIQ (Chen et al., 2024a). These IQA metrics were not used for the data filtering of our training dataset in 8. As shown in the Tab. 13, VARSR continues to achieve SOTA results, surpassing other methods. This validates the strong capability and robustness of VARSR generate high-quality and content-rich images, rather than deceiving specific metrics.
Table 14, Quantitative comparison with SOTA methods with smaller finetuning data.
DrealSR SeeSR
RealSR SeeSR
Metrics
DifiBir
PASD
VARSR(LSDIR)
DiflBir
PASD
VARSR(LSDIR)
SSIMT
0.6516
0.7539
0.7711
0.7536
0.6664
0.7140
0.7284
0.7042
LPIPS.
0.4537
0.3331
0,3142
0.3716
0.3485
0.2986
0.2993
0,3614
DISTS.
0.2724
0.2322
0,2230
0,2620
0.2257
0.2125
0.2224
0.2540
MANIQA
0.4602
0.4551
0.5077
0.5368
0.4602
0.4551
0.5077
0.5539
MUSIQT
61.06
63.69
64.75
67.74
61.06
63.69
64.75
70.49
Finetuning Smaller Datasets. The primary objective of using our large-scale data is for pretraining to acquire generative priors, similar to the powerful base models like Stable Diffusion (Rombach et al., 2022). Therefore, in this section, we present the results of finetuning VARSR on a smaller dataset after undergoing large-scale pretraining, We utilize the LSDIR (Li et al., 2023) dataset and 10k images from FFHQ (Karras et al., 2019) as high-quality data for finetuning ISR tasks, which consistent with SeeSR, with a total of 95k high-quality images. In Tab. 14, when pretraining with our large-scale data and finetuning with smaller data, VARSR still performs exceptionally well, far surpassing other SOTA methods in perceptual
17

---

Visual Autoregressive Modeling for Image Super-Resolution
quality metrics. This is consistent with our conclusion when training large-scale data, validating the superiority of the VARSR framework.
B.3. Ablation Study
Table 15. Ablation on the Training Database.
DrealSR
RealSR
Metries
DIV2K
Large-scale Data
DIV2K
Large-scale Data
SSIMI
0.7750
0.7652
0.7284
0.7169
LPIPS
0.3441
0,3541
0.3376
0,3504
DISTS
0,2434
0,2526
0.2321
0.2470
MANIQAT
0,4885
0.5360
0.4990
0.5570
MUSIQT
64.68
68.15
67.36
71.26
Table 16. Ablation on the Image Preprocess Strategy.
DrealSR Random Crop
RealSK
Metries
Resize&Crop
Random Crop
Resize&Crop
SSIM
0.7772
0.7652
0.7304
0.7169
LPIPS
0.3279
0.3541
0.3317
0.3504
DISTS
0.2335
0.2526
0.2286
0.2470
MANIQA
0.4577
0.5360
0.4639
0.5570
MUSIQI
63.19
68.15
66.62
71.26
LR
IIR
Trianed
Trianed Dataset
(a) Comparison of VARSR trained different dataset.
Trianed with Resize& Resize&CemterCrop (b) Comparison VARSR trained on image preprocess method.
LR
HR
Trianed
Random Crop
Figure 12. Comparison of different dataset and image preprocess methods.
Large-scale Dataset. Tab. 15, we validate importance of training VARSR with large-scale, high-quality data. separately test the model trained on our large-scale dataset (over 4M images) and the model trained using the DIV2K (Agustsson & Timofte, 2017) series dataset (12k images, following the setting of (Yang al., 2025)). The model trained on the DIV2K dataset utilized the original VAR (Tian et al., 2024) trained ImageNet (Deng et al., 2009) the pretrained model. Training with large amount of high-quality data results in significant performance improvement, as also confirmed by the visual results in Fig. 12(a). This demonstrates the effectiveness and necessity of using large-scale, high-quality data.
Image Preprocess. As depicted in Sec. 3.3, we preprocess the training images by first resizing and then cropping to include more foreground semantic information. Tab. 16 and Fig. 12(b), we compare this preprocessing method with the
18

---

Visual Autoregressive Modeling for Image Super-Resolution
commonly used random cropping strategy. Our strategy achieves higher perceived quality, producing richer details, which demonstrates that high-frequency semantic information be better preserved through our preprocessing.
Table 17. Ablation on the VQVAR Scale Dropout Strategy.
DrealSR
RealSR
Metrics
w/o dropout
Pa = 0.1
w/o dropout
Pd 0.1
SSIM
0.7567
0.7652
0.7155
0.7169
LPIPS
0.3667
0.3541
0.3587
0,3504
DISTS
0.2634
0.2526
0,2511
0.2470
MANIQAT
0.5410
0,5362
0.5414
0.5570
MUSIQU
67.98
68.15
70.83
71.26
VAE Dropout
VAE Dropout
Figure Effectivess of VQVAE Scale Dropout Strategy.
VQVAE Dropout Strategy. In Tab. 17, we compare the results of training the VARSR with the VQVAE tokenizer under two conditions, representing whether the VQVAE uses the scale dropout strategy during training with a dropout ratio Pa = 0.1. The tokenizer using the dropout strategy achieves significant improvements in all metrics, particularly fidelity-based metrics, as the preceding scales can provide richer semantic information. As shown in Fig. 13, when the scale dropout strategy is not applied, all the fine texture details are generated in the final scale, which may result in some undesired artifacts in generated images. The condition with the dropout strategy applied can consistently generate images that are faithful to the original image.
B.4. Limitations
I.R
VARSR(Ours)
LR
VARSR(Ours)
Figure Limitations VARSR. to limitations in scope training data coverage, may be challenging restore the correct semantics in certain extreme scenarios.
Although our VARSR base model is pretrained on the large-scale dataset of 4M images, the semantic coverage may still
19

---

Visual Autoregressive Modeling for Image Super-Resolution
have limitations compared the billions of data used by Stable Diffusion (Rombach et al., 2022). is undeniable that some may not be covered well. When the original image contains rare semantics and is severely damaged, it may not be possible to correctly restore the content. As shown in Fig. 14, the LR image's severe distortion impedes the accurate restoration of the famous character Spider- Man in the left case, while the generated image of an ancient woman in the right case features simplistic and unrealistic head adornments.
C. More Visualization Results
In Fig. 16 and Fig. 17, provide additional comparison visualization results with other methods, showcasing the robust capability of VARSR in generating high-fidelit and high-realistic images.
20

---

Visual Autoregressie Modeling for Image Super- Resolutin
LR
BSRGAN
Real-ESRGAN
StableSR
VARSR(Ours)
DiABIR
PASD
SccSR
VARSR(Ours)
BSRGAN
Real-ESRGAN
StubleSR
VARSR(Ours)
DinBIR
PASD
SccSR
VARSR(Ours)
LR
BSRGAN
Real-ESRGAN
StableSR
VARS)(Ours)
DiABIR
SceSR
VARSR(OurS)
LR
BSRGAN
Real-ESRGAN
StableSR
VARS)(Ours)
DitBIR
PASD
SceSR
VARSR(Ours)
Figure 15, Comparisons with different methods on real-world images. Zoom in for better view.
21

---

Visual Autoregressive Modeling for Image Super- Resolutinn
BSRGAN
Real-ESRGAN
StableSR
(ARS)(OUrr)
HR
PASD
SeeSR
VARSR(Ours)
LR
BSRGAN
Real-ESSRGA
StableSR
VARSR(OUrs)
HR
PASD
SecSR
VARSR(OurS)
LR
BSRGAN
Real-ESRGAN
StableSR
VARSR(OUFs)
HR
PASD
SeeSR
VARSR(Ours)
BSRGAN
Real-ESRGAN
StableSR
VARSR(Ours)
HR
SeeSR
VARSR(Ours)
Figure 16. Additional qualitative comparisons with different SOTA methods (Part 1). Zoom in for better view.
22

---

Visual Autoregressive Modeling for Image Super-Resolution Resolution
LR
BSRGAN
Real-ESRGAN
StubleSR
1OOD ZACA RO
XOO BAAAK X5
F.950p
VARSR(Ours)
1IR
PASD
SecSR
VARSR(Ours)
LR
BSRGAN
Real-ESRGAN
StubleSR
VARSR(Ours)
HR
PASD
VARSR(Ours)
LR
BSRGAN
Real-ESRGAN
StableSR
(Our)
HR
PASD
SccSR
VARSR(Ours)
BSRGAN
Real-ESRGAN
StubleSR
Ours)
HR
PASD
SeeSR
VARSR(Ours)
Figure 17. Additional qualitative comparisons with different SOTA methods (Part 2). Zoom in for better view.
23


Reviews:
[{"review_id": "DL447i7bPD", "reviewer": "Reviewer_XdmU", "summary": "This paper presents VARSR, a Visual Autoregressive Model for Image Super-Resolution (ISR). VARSR leverages autoregressive modeling with next-scale prediction, prefix tokens for integrating low-resolution conditions, Scale-aligned Rotary Positional Encoding (SA-RoPE) for preserving spatial structure, and a Diffusion Refiner for pixel-level fidelity. The model is trained on a large-scale dataset of over 4 million high-quality images. Experiments show that VARSR outperforms existing methods in both fidelity and realism, offering higher computational efficiency compared to diffusion-based models.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A1. Training with different datasets.**\n \nThanks for your comments. In fact, baseline methods are trained on different datasets (e.g., SeeSR uses LSDIR and PASD uses DIV2K/FFHQ, with a tenfold difference in data quantity). There are also differences in the pretrained models,  which are important in providing generative priors for ISR (e.g.,  diffusion methods using Stable Diffusion and GAN methods trained from scratch). In lines 258-270, the primary objective of using large-scale data is for pretraining to acquire generative priors. The open-source VAR base model falls short of our needs, as it could only generate $256*256$ images and is limited in generated image quality. However, diffusion methods leverage the powerful Stable Diffusion as their base model, which is pretrained on billions of image-text pairs, far surpassing our scale of 4 million. Thus, in evaluating ISR methods, previous works (e.g., SeeSR/PASD) typically compare based on the performance of the models themselves.  \n\nBased on your advice, we present VARSR results trained on the same datasets as baselines. In the Tab below, when pretraining with our large-scale data and fine-tuning with the same LSDIR as SeeSR, VARSR still performs exceptionally well, far surpassing other methods in perceptual quality metrics. This is consistent with our conclusion when training on large-scale data, validating the superiority of the VARSR framework. \n\n||Metrics|PSNR|SSIM|LPIPS|DISTS|MANIQA|CLIPIQA|MUSIQ|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|DRealSR|DiffBIR|26.57|0.6516|0.4537|0.2724|0.4602|0.6445|61.06| \n||PASD|27.45|0.7539|0.3331|0.2322|0.4551|0.6365|63.69| \n||SeeSR|28.13|0.7711|0.3142|0.2230|0.5077|0.6893|64.75| \n||VARSR (trained on LSDIR)|27.87|0.7536|0.3716|0.2620|**0.5368**|**0.7206**|**67.74**| \n\n**A2. Advantages of VARSR in preserving semantics.** \n\nVAR has the advantage over diffusion models in better human-perceived semantic fidelity and quality.\n\n(1) Firstly, in terms of fidelity metrics as PSNR and SSIM, VARSR performs similarly to SeeSR and outperforms all other diffusion methods on DIV2K and DrealSR datasets (Tab.1). However, existing fidelity metrics have limitations in measuring human perceptual quality. Fig.4 shows that images with higher human perceptual quality may not score well on fidelity metrics, as overly smoothed images tend to perform better. These limitations have been confirmed in previous studies (e.g., SUPIR in CVPR2024, Pipal in ECCV2020), and mathematical derivations verify the contradiction between fidelity and quality (The Perception-Distortion Tradeoff in CVPR2018). \n\n(2) Secondly, user study and numerous examples support VARSR's superiority in maintaining spatial structure and preserving semantics. In Tab.10 (user study), the results of VARSR align more closely with human preferences. In Fig.5/15/16/17, examples illustrate that VARSR excels in generating textures faithful to the original image, outperforming diffusion models. In the last two rows of Fig.5, only VARSR accurately restores architectural semantics and smooth details. In the 2nd and 3rd cases of Fig.17, VARSR generates clear foliage and walnuts, while diffusion methods exhibit illusion issues (SeeSR produces fabricated content). These results highlight VARSR's superior preservation of semantics and spatial structure.\n\n**A3. Efficiency Comparison.**\n\nThanks for your comments. We believe that the superior efficiency of VARSR lies in the advantages of the VAR framework over the standard diffusion framework. Therefore, there is no need to compare with models that are optimized to enhance efficiency for a specific framework. This is because these efficiency designs can also be applied to VARSR to further improve efficiency. For example, one-step diffusion models (e.g., OSEDiff in NIPS2025 and SinSR in CVPR2024) often undergo knowledge distillation to match the performance of multi-step generation. Our VARSR can also adopt this approach to simplify the inference steps (e.g., reduce the inference scales).\n\nTab. 3 and the explanations (lines 380-384/407-413) demonstrate VARSR's notable efficiency enhancements over diffusion models, validating our motivation. Compared to the one-step diffusion method SinSR (refer to A1 to Reviewer vbZD), VARSR outperformed in most metrics, showcasing its effectiveness.\n\n**A4. Motivation.**\n\nVAR provides a novel and effective approach for addressing ISR tasks, offering advantages over diffusion models rather than a straightforward substitute. Our work is just an initial attempt, and there is vast potential for further leveraging VAR in ISR tasks. In addition to the advantages in preserving structural features and efficiency, as you mentioned, VAR aligns the generative paradigm for both vision and text. Thus, we believe that VAR has the potential to be integrated with LLMs, enabling the direct utilization of human preferences to guide ISR through optimization forms such as DPO or GRPO, which is a promising avenue for future research.", "claims_and_evidence": "- The paper compares VARSR with several state-of-the-art methods (GAN-based, diffusion-based, and autoregressive-based), and VARSR consistently performs well in both quantitative and qualitative evaluations across different datasets. It achieves superior results on no-reference IQA metrics, such as MANIQA, CLIPIQA, and MUSIQ, and performs on par with diffusion-based methods in terms of reference-based metrics like PSNR and SSIM.\n\n- The paper demonstrates that VARSR is more computationally efficient than diffusion models, requiring only 0.59s for inference, significantly reducing the number of steps needed compared to diffusion-based methods.", "methods_and_evaluation_criteria": "Nan", "theoretical_claims": "Nan", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "- The paper builds upon prior research in autoregressive image generation (e.g., VQGAN, DALL-E) and image super-resolution (e.g., GAN-based methods, diffusion models). It extends the work on autoregressive models by introducing next-scale prediction and addressing issues specific to ISR tasks, such as pixel-level fidelity and the preservation of spatial structure. \n- I keep up with the literature in this area.", "essential_references_not_discussed": "Nan", "other_strengths_and_weaknesses": "- The originality of the approach lies in its integration of autoregressive modeling with image super-resolution, making it capable of achieving high fidelity and realism with computational efficiency. The extensive experiments, including a user study, validate the proposed method's performance in real-world scenarios, highlighting its potential practical application.", "other_comments_or_suggestions": "- When comparing with other generative models, it appears that the authors did not re-train these models on the newly proposed large-scale dataset. As shown in Table 9, VAR benefits from training on the large-scale dataset. Therefore, directly comparing it with other generative models trained on ImageNet is not a fair comparison.\n- The motivation for introducing VAR into the super-resolution field is not clearly explained. The authors claim that VAR preserves structure better than the Markov process in diffusion models, but the final results show that its reference-based metrics are generally weaker than those of diffusion-based methods, which seems contradictory. Additionally, the authors argue that VAR is more efficient than DDPM, but there are already many efficient designs for diffusion models and one-step generation methods. Therefore, the claim of efficiency improvement for basic diffusion models alone is insufficient to support the motivation of this paper. I believe VAR is an excellent work, but its primary contribution lies in aligning the generative paradigms of vision and text via autoregressive methods. Using it as a simple replacement for diffusion models lacks a clear motivation.", "questions_for_authors": "Please see weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fOx4VWJScD", "reviewer": "Reviewer_vbZD", "summary": "This paper proposed a image super resolution framework that is built on top of the VAR framework -- next-scale prediction. They modify the original VAR architecture to digest tokens from low res image inputs. And then, they leverage the original VAR architecture to upsample until the highest granularity. Given the quantized signals, they further add one diffusion refiner to close the quantization gap, i.e., producing continuous tokens from the VAR prediction. In the above process, scaled roPE is introduced, and a modified CFG is presented. Experiment-wise, they compare mostly against generative image super resolution prior, including GAN and diffusion based priors. They demonstrate they are higher quality and more efficient.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A1. Comparison with more SOTA.** \n\nBSRGAN and Real-ESRGAN are still commonly used SOTA GAN-based models due to their excellent performance. Other ISR works (e.g., SeeSR, PASD) also chose them as GAN-based baselines for comparison.\n\nGigaGAN does not provide open-source models or code for testing on its homepage. Therefore, we further conduct a comparison with more recent SOTA DASR (GAN-based, ECCV2022) and SinSR (diffusion-based, CVPR2024). As shown in the Tab, the results are consistent with the findings in the paper, with VARSR leading by a significant margin in perceptual quality metrics, validating the strong performance of VARSR.\n\n||Metrics|PSNR|SSIM|LPIPS|DISTS|FID|MANIQA|CLIPIQA|MUSIQ|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|DIV2K|DASR |24.46|0.6253|0.3696|0.2533|57.37|0.3104|0.4960|53.96| \n||SinSR| 24.22|0.5922|0.3429|0.2157|42.17|0.4101|0.6411| 61.46| \n||**VARSR**|23.91|0.5890|0.3260|0.2218|35.51|**0.5340**|**0.7347**|**71.27**| \n|RealSR|DASR|25.47|0.7575|0.32401|0.2267|133.33|0.2470|0.3198 | 41.21| \n||SinSR |24.86|0.7191|0.3472|0.2500|142.31|0.3985|0.6162|60.57 | \n||**VARSR**|24.61|0.7169|0.3504|0.2470|137.55|**0.5570**|**0.7006**|**71.26**| \n|DRealSR|DASR|29.75|0.8262|0.3099|0.2275|155.36| 0.2809|0.3813|42.41| \n||SinSR|28.26|0.7443|0.3743|0.2495|173.42|0.3843|0.6302|55.28| \n||**VARSR**|28.16|0.7652|0.3541|0.2526|155.87|**0.5362**|**0.7240**|**68.15**| \n\n**A2. Necessity of Diffusion Refiner.**\n\nAs mentioned in lines 043-051 of our paper, we believe that VAR is effective for ISR tasks. Similar to other improvements made to VAR-related components (e.g. Prefix Tokens, SA-RoPE), the Diffusion Refiner is also an attempt to enhance the upper bound of VAR in ISR tasks.\n\nAs highlighted in lines 185-195, quantization of image discrete vectorization leads to loss.  Even if VAR accurately predicts all quantized discrete tokens, the generated image will still have quantization losses that limit its upper bound of restoration. Therefore, we specifically proposed a refiner to convert predictions of categorical discrete vector distribution into a continuous-valued space, thereby enhancing the upper bound of VAR's representational capacity. This Refiner utilizes the loss of diffusion form to model the probability of quantization residuals instead of the complete latents to accelerate convergence. The idea of using a diffusion refiner for discrete-to-continuous mapping has been validated in many previous works: MAR (NIPS2025) and HART (ICLR2025).\n\nIn Tab.5, the introduction of the diffusion refiner led to improvements in all metrics, especially in perceptual quality metrics. Notably, MANIQA achieved an average improvement of 2.2%, and SSIM improved by 0.82%. Fig. 8 shows that the diffusion refiner reintroduced many image details lost in quantization (e.g., textures of the clothes and the flowers). Furthermore, the diffusion refiner is an extremely lightweight module, with only 37M parameters accounting for just 3% of the 1.1B model. We believe that the Diffusion Refiner is effective in enhancing the representational capacity upper bound of VAR in ISR tasks with a very small param increase.\n\n**A3. Necessity of CFG.**\n\nSimilarly, the introduction of CFG is also aimed at expanding the upper bound of the image quality generated by VAR, thereby generating more realistic images through guided sampling.\n\nAs mentioned in lines 152-156,  VAR, GANs, and Diffusion models all target fidelity as the optimization objective in ISR tasks, which may result in generated images being overly smooth and lacking in detail.  It tends to retain distortions such as blur from LR images, leading to lower human-perceived quality. To address this, we propose an image-based CFG that follows the principles of the standard CFG. By learning low-quality image distributions during training, it allows us to guide the probability distribution during sampling towards generating higher-quality images, thereby expanding the upper bound of the image quality.\n\nImage-based CFG validates the introduction of a new form of CFG into the VAR framework, striking a balance between realism and fidelity similar to the diffusion model. Results in Tab.6 and Fig.9 confirm the effectiveness of Imaged-base CFG, resulting in significantly richer textures in the generated images, substantially improving perceptual quality to meet human preferences while maintaining correct semantics.", "claims_and_evidence": "yes.", "methods_and_evaluation_criteria": "Method-wise, I am confused on the necessity of the diffusion refiner, and the introduction of CFG together with its positive and negative embedding. If the message is VAR is great for ISR, then the framework should focus on pushing more on the upperbound of VAR related components. However, now it ends up instead mixing diffusion with VAR, which makes the key message vague to me. \n\nEvaluation criteria makes sense.", "theoretical_claims": "no theoretical claims.", "experimental_designs_or_analyses": "I am not sure BSRGAN or Real-ESRGAN is SOTA, which was published back in 2021. Latest GAN baselines, such as GigaGAN(CVPR 2023,https://mingukkang.github.io/GigaGAN/), is not compared directly. Similarly, diffusion-based baseline is missing, SinSR (https://github.com/wyf0912/SinSR, CVPR 2024).", "relation_to_broader_scientific_literature": "repurpose VAR for Image super resolution.", "essential_references_not_discussed": "They have discussed the essential prior work, e.g., VAR and VQVAE", "other_strengths_and_weaknesses": "Strengths:\n1. The authors did a fantastic job in ablation over their design choices. \n2. The qualitative and quantitative results show their greater quality against the selected baselines. And it makes their ablation justification easy to follow. \n3. The metrics are carefully selected and discussed.\n\nWeakness:\n1. As is stated above. I am confused on the necessity of the diffusion refiner, and the introduction of CFG together with its positive and negative embedding. If the message is VAR is great for ISR, then the framework should focus on pushing more on the upperbound of VAR related components. However, now it ends up instead mixing diffusion with VAR, which makes the key message vague to me.\n\n2. The role of diffusion diffuser is very marginal. As can be seen from the Table 5, despite improvement, it is very tiny scale better. For this level of improvement, an additional level of VAR might just suffice without the need to introduce the diffusion part. Then, the need of CFG is also questionable to me.", "other_comments_or_suggestions": "none", "questions_for_authors": "none", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tX61mv62nh", "reviewer": "Reviewer_yFzJ", "summary": "This work presents a novel generative model for image super-resolution, leveraging visual autoregressive modeling. The generative process is conditioned on the low-resolution input, treated as a prefix token. To enhance performance, the authors introduce scale-aligned positional encoding and a diffusion-based refinement step to mitigate quantization losses. The proposed method surpasses previous diffusion-based models in both fidelity and efficiency.\n\n## after rebuttal: weak accept", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A1. Effectiveness of CFG.**\n\nCFG leads to a certain reduction in fidelity metrics but significantly improves the perceptual quality of the image. Existing fidelity metrics have limitations in accurately measuring human perceptual quality, especially when the original HR image quality is low. In Fig.4,  generated images of higher perceptual quality for humans can lag behind in certain fidelity metrics, as overly smoothed low-quality images tend to score better on these metrics.\n\nThese limitations have been confirmed in many previous studies (e.g., SUPIR in CVPR2024 and Pipal in ECCV2020), and mathematical derivations have verified the inherent contradiction between fidelity and quality (The Perception-Distortion Tradeoff in CVPR2018). In Fig.9, the introduction of CFG results in significantly richer textures in the generated images, substantially improving perceptual quality to meet human preferences while maintaining correct semantics.\n\n**A2. Other IQA Metrics.**\n\nThanks for your valuable comments. First, as mentioned in A1, fidelity and IQA metrics can exhibit certain contradictions. Using our large-scale datasets for training can generate images that retain semantics and have higher quality.\n\nSecond, the good performance of VARSR in IQA metrics does not originate from specialized data filtering but from its ability to generate high-quality images. We further conduct evaluation using other well-adopted IQA metrics that were not used for data filtering, including CNNIQA (CVPR2014), HyperIQA (CVPR2020), and TOPIQ (TIP2024). As shown in the Tab below, VARSR continues to achieve SOTA results, surpassing other methods.\n\n||Metrics|BSRGAN|RealESR|SwinIR|LDM|StableSR|DiffBIR|PASD|SeeSR|VARSR(Ours)|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|DIV2K|CNNIQA|0.5492|0.5652|0.5402|0.5579|0.6274|0.6413|0.6269|0.6613|**0.6661**|\n||HyperIQA|0.5682|0.5586|0.5235|0.5225|0.6100|0.6164|0.6158|0.6666|**0.7031**|\n||TOPIQ|0.5413|0.5182|0.4796|0.4695|0.5923|0.6105|0.6165|0.6793|**0.7020**| \n|RealSR|CNNIQA|0.5513|0.5624|0.5281|0.5637 | 0.6029|0.6077|0.5938|0.6594|**0.6692**|\n||HyperIQA|0.5617|0.5231|0.5093|0.4936|0.5703|0.5690|0.6001|0.6746|**0.7038**|\n||TOPIQ|0.5502|0.5137|0.4882|0.4762|0.5579|0.5580|0.5920|0.6854|**0.6991**|\n|DRealSR|CNNIQA|0.4989|0.4849|0.5017|0.5367|0.5518|0.6025|0.5794|0.6132|**0.6445**|\n||HyperIQA|0.5305|0.4938|0.5074|0.5050|0.5537|0.5992|0.6008|0.6583|**0.6866**|\n||TOPIQ|0.5058|0.4622|0.4694|0.4807|0.5330|0.5831|0.5963|0.6534|**0.6800**|\n\n**A3. Complexity of the training setup.**\n\nOur model is no more complex than diffusion methods, both requiring the same three-step process for application in ISR tasks: (1) training the VAE, (2) pre-training on C2I/T2I tasks, and (3) fine-tuning on ISR. The apparent simplicity of training in previous diffusion methods stems from directly using pre-trained Stable-Diffusion and its VAE as base models. These base models have already undergone training in the first two steps, requiring only fine-tuning for the ISR task.\n\nHowever, the open-source VAR base model falls short of our needs, as it could only generate $256*256$ images and is limited in generated image quality. Therefore, we need to conduct training in all three stages, making it appear more complex. We intend to open-source the base models trained in the first two steps to ease training burdens for future research and contribute more to the community.\n\n**A4. Clarification of Diffusion Refiner.**\n\nOur proposal of the Diffusion Refiner does not mean introducing an additional ISR process but offers a mapping of continuous residual distributions. As highlighted in lines 185-195,  quantization of image discrete vectorization leads to loss, thereby restricting the upper bound of restoration, as VAR can only predict the quantized discrete vectors of the image. Therefore, we specifically proposed a refiner to convert predictions of categorical vector distribution into a continuous-valued space through a diffusion loss, thereby enhancing the upper bound of VAR's capacity. Such an idea has been validated in previous works: MAR (NIPS2025) and HART (ICLR2025).\n\nThe Refiner solely serves to map the probability distribution of quantized residuals with VAR features as a condition, and does not have the capabilities of an ISR model. A lightweight network (only 37M parameters accounting for 3% of the 1.1B model) suffices for the discrete-to-continuous mapping. In the Tab below, a larger refiner does not yield significant gains, which was also confirmed in MAR (NIPS2025). Thus, we believe that a standard SR model as the refiner will not lead to improvement.\n\n||Refiner|PSNR|SSIM|LPIPS|DISTS|FID|MANIQA|CLIPIQA|MUSIQ|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|RealSR|6-layer|24.61|0.7169|0.3504|0.2470|137.55|0.5570|0.7006|71.26|\n||18-layer|24.51|0.7184|0.3492|0.2478|138.75|0.5521|0.7043|70.97|", "claims_and_evidence": "The authors identify the challenges of adapting autoregressive models for image super-resolution and propose tailored solutions for each issue.", "methods_and_evaluation_criteria": "Yes, please refer to experimental design for more details.", "theoretical_claims": "No, as there are no proofs or theoretical claims.", "experimental_designs_or_analyses": "Yes. The experimental validation is consistent with previous work, and sufficient ablations regarding the contributions are provided. However, the data collection and filtering processes remain questionable.", "relation_to_broader_scientific_literature": "This paper explores a novel generative framework recently popularized by VAR, while also tackling efficiency, adding further value to the approach.", "essential_references_not_discussed": "Essential references are discussed.", "other_strengths_and_weaknesses": "Strengths:\n- Overall, this is a strong and well-written paper, supported by extensive experimental results.\n\n- The authors clearly articulate their motivation and provide thorough explanations of their methodology, which significantly enhances the efficiency of generative image super-resolution, which is a valuable contribution.\n\nWeaknesses:\n- The use of classifier-free guidance noticeably reduces SSIM, likely affecting PSNR similarly. Established perceptual metrics like LPIPS also show a negative impact.\n\n- The larger dataset appears to negatively influence SSIM and LPIPS. Since a filtering stage based on MUSIQ/MANIQA was applied, it seems that the selected images are optimized specifically for these metrics, raising concerns about data representativeness.\n\n- The overall performance of VARSR seems strong specifically on the metrics used for pre-filtering the data. Compared to previous methods like SeeSR, which use smaller and lower-quality datasets, this brings the value of the collected data into question and the fairness of evaluation.\n\n- The training setup is complex and resource-intensive, aiming to design a more efficient diffusion-based alternative — though diffusion processes are still integral to the approach.", "other_comments_or_suggestions": "None.", "questions_for_authors": "- How does the model perform when using a standard super-resolution model as the refiner instead of a diffusion model? Does the efficiency also improve?\n\n- How does the model perform on IQA metrics that were not used for data filtering?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GU777VSYcj", "reviewer": "Reviewer_ZhDb", "summary": "This paper proposes VARSR, a visual autoregressive framework for image super-resolution (ISR), addressing the trade-off between fidelity and realism. By leveraging next-scale prediction, prefix tokens for LR conditioning, scale-aligned rotary positional encodings (SA-RoPE), and a diffusion refiner for quantization residual modeling, VARSR achieves promising results in perceptual quality while maintaining computational efficiency. The authors also introduce a large-scale dataset and an image-based classifier-free guidance (CFG) mechanism to enhance realism. Extensive experiments demonstrate VARSR’s superiority over GAN- and diffusion-based methods in both qualitative and quantitative metrics, with significant efficiency gains.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A1. Implementation details.** \n\nThank you for your suggestions. We will elaborate further on these details in our supplementary materials, for example, the MLP architecture of the diffusion refiner, which consists of linear and activation layers. Additionally, our code will be open-sourced to help readers double-check and verify our implementation details.\n\n**A2. User Study.**\n\nWe conducted a larger-scale user study involving 20 participants to evaluate 100 images. The scale of this study is already sufficiently large compared to previous research (e.g., PASD in ECCV2024 involved 15 participants evaluating 40 images). The experimental results further validate our assertion that our VARSR attains the highest selection rate of 57.1%, significantly surpassing alternative methods. This underscores the potent capability of VARSR in real-world settings to produce lifelike images that harmonize with human aesthetics.\n\n| Methods | BSRGAN | Real-ESRGAN | StableSR | PASD | SeeSR | VARSR(Ours) |\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n| Selection Rates | 0.35% | 0.7% | 3.5% | 17.3%| 21.05% | **57.1%** |\n\n**A3. Domain coverage of our large-scale dataset.**\n\nTo achieve diversity and balance in images from different domains, we conducted semantic clustering and supplemented specific category data. As shown in the Tab below, we ensured that the dataset covers a wide range of category scenes and a relatively balanced proportion (Scenes with a broader semantic scope correspond to a higher proportion of images). This includes portraits, people, food, animals, natural landscapes, cartoons, cityscapes, indoor and outdoor scenes, ensuring comprehensive coverage of visual concepts and richness of scene content.\n\nIt is undeniable that due to the limited number of images, some rare scenes may not be covered well, such as the semantic scenes discussed in Sec. B.4 Limitations of our Supp..\n\n| Categories | Indoor | Outdoor | Nature | Human | Plant | Object | Animal | Text | Food | Cartoon | Others |\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n| Rates | 10% | 9% | 13% | 28% | 8%| 9% | 6% | 4% |  8% | 2% | 3% |\n\n**A4. Highly degraded scenarios.**\n\nIn Sec. B.2 of our Supp., we conducted evaluations on the RealLR200 dataset and achieved SOTA results compared to previous methods. The RealLR200 dataset already contains many highly degraded real-world images, such as historical photographs and extreme compression artifacts. \n\nWe further provided restoration results in these extreme degradation scenarios, which can be accessed at https://figshare.com/articles/figure/extreme_pdf/28668452?file=53243348. It can be observed that VARSR can still produce faithful and high-quality results in such extreme degradation scenarios, validating the robust capability of VARSR.\n\n**A5. Tiling for High-resolution images.**\n\nVARSR adopts a tiling approach in generating high-resolution images, which is completely consistent with the diffusion based method. Specifically, we uniformly divide the high-resolution image to be generated into overlapping grids, with each grid having a standard resolution of $512\\times 512$. VARSR generates SR results for each grid separately and then tiles them together to obtain the complete image restoration result.\n\nTherefore, the computational cost increases linearly with the increase in output resolution. However, we can batch process different grids in parallel to accelerate the inference process, significantly increasing the actual computational speed. Importantly, due to VARSR and the diffusion model using the same tiling approach, the tenfold efficiency advantage over the diffusion model still exists when handling images of various resolutions.\n\n**A6. Clarification of Diffusion Refiner.**\n\nPlease see A4 to Reviewer yFzJ.\n\n**A7. Spatial Misalignment.**\n\nIn the field of image restoration, our goal is to recover the original HR image from severely degraded LR images, where degradation refers to distortions such as blur, noise, compression, and other artifacts that may occur during transmission or capture processes, as defined in the classic work SRCNN (TPAMI2015) in ISR field. \n\nThe spatial distortions you mentioned, such as rotation or flipping, typically do not fall within the scope of distortions that need to be addressed in ISR field.  When spatial distortions such as rotation occur, we can employ preprocessing methods (e.g., corner detection) to restore spatial positioning, followed by the image restoration process. Therefore, we can assume that LR and HR images are always aligned in spatial scale. Our proposed SA-RoPE effectively conveys this spatial structural consistency, thereby enhancing the ISR performance.", "claims_and_evidence": "The paper provides strong empirical evidence to support its claims, including quantitative results, qualitative comparisons, ablation studies, and user evaluations. These claims are well-aligned with the broader goals of ISR and generative modeling, and the evidence demonstrates VARSR’s effectiveness in addressing key challenges in the field.", "methods_and_evaluation_criteria": "The methods and evaluation criteria in the paper are well-designed and comprehensive. VARSR’s innovations (e.g., next-scale prediction, prefix tokens, SA-RoPE, diffusion refiner, CFG) are rigorously validated through quantitative metrics, qualitative comparisons, ablation studies, and human evaluation. The use of both reference-based and non-reference metrics ensures a balanced assessment of fidelity and perceptual quality, while the user study provides valuable insights into real-world applicability.", "theoretical_claims": "The theoretical claims in the paper are well-supported by both theoretical justifications and empirical evidence. The use of a large-scale dataset and a robust training pipeline further enhances the model’s generative priors, making it a strong candidate for real-world ISR applications.", "experimental_designs_or_analyses": "The experimental designs and analyses in the paper are comprehensive and well-structured. They effectively validate VARSR’s performance through quantitative metrics, qualitative results, ablation studies, and human evaluation. The use of both synthetic and real-world datasets ensures a robust evaluation, while the ablation studies provide valuable insights into the contributions of each component.", "relation_to_broader_scientific_literature": "- Image Super-Resolution (ISR)\n\n ISR is a well-studied problem in computer vision, aiming to reconstruct high-resolution (HR) images from low-resolution (LR) counterparts. Traditional methods (e.g., SRCNN, VDSR) focus on pixel-level fidelity but struggle with real-world degradations and perceptual quality.\n\n- Autoregressive Modeling\n\n Autoregressive models, popularized in language modeling (e.g., GPT, LLaMA), have recently been adapted to vision tasks (e.g., VQVAE, DALL-E). These models generate images by predicting tokens sequentially, often in a coarse-to-fine manner.\n\n- Generative Models\n\nGenerative models, including GANs, VAEs, and diffusion models, have revolutionized image synthesis and restoration. Each has strengths (e.g., GANs for realism, diffusion for detail) and weaknesses (e.g., GAN instability, diffusion inefficiency).", "essential_references_not_discussed": "The references are relatively comprehensive.", "other_strengths_and_weaknesses": "- Strengths\n\n1) First application of visual autoregressive (VAR) modeling to ISR, introducing next-scale prediction as a core mechanism.\n\n2) The large-scale dataset (4M images) and training pipeline (C2I pretraining + ISR finetuning) enhance generative priors.\n\n3) Comprehensive evaluations across synthetic (DIV2K-Val) and real-world datasets (RealSR, DRealSR) using both reference (PSNR, SSIM) and non-reference metrics (MANIQA, CLIPIQA, MUSIQ).\n\n4) VARSR outperforms SOTA methods (e.g., PASD, SeeSR) in perceptual metrics (e.g., +6.7% MANIQA, +3.7% CLIPIQA) while matching diffusion models in fidelity.\n\n5) Efficiency: 10× faster inference than diffusion methods (0.59s vs. 5.85s for DiffBIR).\n\n- Weaknesses\n\n1) Some implementation details are under-explained, e.g., the exact architecture of the diffusion refiner, the interaction between SA-RoPE and multi-scale tokens, and the training dynamics of the VQVAE with scale dropout.\n\n2) The user study (50 images, 20 participants) is relatively small-scale. Expanding this would strengthen claims about human preference.\n\n3) The large-scale dataset’s curation process (e.g., semantic balance, filtering thresholds) is described briefly. A deeper discussion of potential biases or limitations (e.g., domain coverage) is needed.\n\n4) Limited testing on niche or highly degraded real-world scenarios (e.g., historical photographs, extreme compression artifacts).", "other_comments_or_suggestions": "None.", "questions_for_authors": "1) How does the computational cost of VARSR scale with output resolution? The paper mentions tiling for higher resolutions but lacks empirical analysis.\n\n2) Could the diffusion refiner be replaced with a lightweight alternative (e.g., GAN-based) without sacrificing performance?\n\n3) The prefix token approach assumes LR and HR scales are spatially aligned. How does VARSR handle severe misalignment (e.g., rotation or perspective distortion in LR inputs)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces visual autoregressive modeling to image super-resolution. The paper originally received 2xWeakReject and 2xWeakAccept. The main concerns include unfair comparison, unclear motivation, insufficient evaluations, reliance on diffusion, etc. The authors have provided rebuttals. Afterward, all reviewers keep their ratings. In the discussion phase, Reviewer yFzJ acknowledge that although this method currently relies on the diffusion refiner, it first introduces VAR to ISR and establishes a strong baseline for future research. Considering the rebuttal and discussions from all reviewers, ACs recommend accepting this paper. The authors are suggested to carefully revise the paper and incorporate newly conducted experiments according to the comments and discussions.
Decision: Accept (poster)