Texture Vector-Quantization and Reconstruction Aware Prediction for Generative Super-Resolution
Authors: Qifan Li
Jiale Zou
Jinhua Zhang
Wei Long
Xingyu Zhou
Shuhang Gu
Venue: ICLR 2026 Poster (2026)
Keywords: generative super-resolution; vector-quantization
Primary area: generative models
TL;DR: A VQ-based method for generative super-resolution.

Abstract: Vector-quantized based models have recently demonstrated strong potential for visual prior modeling. However, existing VQ-based methods simply encode visual features with nearest codebook items and train index predictor with code-level supervision. Due to the richness of visual signal, VQ encoding often leads to large quantization error. Furthermore, training predictor with code-level supervision can not take the final reconstruction errors into consideration, result in sub-optimal prior modeling accuracy. 
 In this paper we address the above two issues and propose a Texture Vector-Quantization and a Reconstruction Aware Prediction strategy. The texture vector-quantization strategy  leverages the task character of super-resolution and only introduce codebook to model the prior of missing textures. While the reconstruction aware prediction strategy makes use of the straight-through estimator to directly train index predictor with image-level supervision. Our proposed generative SR model TVQ&RAP is able to deliver photo-realistic SR results with small computational cost.

Published as a conference paper at ICLR 2026
TEXTURE VECTOR-QUANTIZATION AND RECONSTRUC- TION AWARE PREDICTION FOR GENERATIVE SUPER- RESOLUTION
Qifan Jiale Zou Jinhua Zhang Wei Long Xingyu Zhou
Shuhang Gu'
University of Electronic Science and Technology of China
qifanli. 11g.@ggmammm... com shuhanggue com
ABSTRACT
Vector- Vector-quantized based models have recently demonstrated strong potential for visual prior modeling. However, existing VQ-based methods simply encode visual features with nearest codebook items and train index predictor with code-level supervision. Due to the richness of visual signal, VQ encoding often leads large quantization error. Furthermore, training predictor with code-level supervision can not take the final reconstruction errors into consideration, result in sub-optimal prior modeling accuracy. In this paper address the above two issues and propose a Texture Vector- Susior--unntzzttonn and a Reconstruction Aware Prediction strategy. The texture vector- qector-quaniization strategy leverages the task character of super- resolution and only introduce codebook to model the prior of missing textures. While the reconstruction aware prediction strategy makes through estimator to directly train index predictor with image-level supervision. Our proposed generative SR model (TVQ&RAP) is able deliver photo-realistic SR results with small computational cost.
of the straight-
Modeling complex footure space containing both structures and textures.
Testure Codebouk: Modeling simple feature space by structures inherently LR
Tt
Leodng
712-112
1gaz22
10-2524
Reature dis diseinngging enctuding
1g-322
Fouure
Structures inheveutly in I.R.
F0-256
onconlng
(a) Vanilla Vector Quantization
(b) Texture Vector Quantization
TVQ
Figure I: Vanilla VQ vs. Texture VQ. Vanilla VQ directly encode the entire visual feature space, large codebook is required to capture complex combinations of structure and texture information. Our Texture VQ focuses on modling textures absent LR inputs, thereby mitigating the difficulty of visual encoding for generative super-resolution. Notably, TVQ achieves significantly better reconstruction performance than the vanilla method across a range of codebook sizes. Experimental details can be found in Section 4.3.
INTRODUCTION
Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low- resolution (LR) counterparts. Classical SR methods target minimizing the Root Mean Square Error (RMSE) between HR estimation and ground truth image (Liang et al., 2021: Zhang et al., 2024; Long ct al., 2025), tend to produce overly smooth results (Ledig et al., 2017). To mitigate this limitation, generative SR (GSR) methods introduce impressive generative modeling techniques, e.g. generative adversarial networks (GANs) (Wang et al., 2018; 2021; Zhang et al., 2021) and diffusion- based models (Rombach al., 2022; Yue et al., 2023; Wang et al., 2024b; Zhang et al., 2025), to obtain the
"Corresoonding author
Project page: https: / /gg thub. nom Shw.shhaanggg///// /TVO/OO

---

Published as a conference paper at ICLR 2026
capability of prior distribution modeling, has been thriving research topic due to its highly practical value in generating photo-realistic SR results.
Recently, besides the LAN-based and Diffusson- based generative modeling techniques, another category of generative visual modeling ap- proaches, i.e. the vector-quantized variational autoencoder (VQ-VAE), has shown advantages in modeling accuracy and efficiency in image generation tasks (Van Den Oord al.. 2017; Esser et al., 2021; Ramesh et al., 2021; Lee et al., 2022; Tian et al., 2025). At the core of VQ-based model is a visual codebook, with which visual features are encoded as their cor- responding nearest codebook items and visual prior modeled by training codebook index pre- dicting networks. Despite their great success in visual prior modeling, the existing VQ-based methods still suffer from the following two lim- itations. First, most of the existing VQ-based methods directly replace visual features with nearest codebook items, due to the richness and diversity of natural images, a large codebook is often required to fulfill the requirement of cod- ing accuracy (see Figure (a)). However, the incorporation of large codebook not only in- troduces heavy memory footprint but also esca- lates training difficulty. Second, in the existing VQ-based methods, visual prior is captured by training the index predicting network with code- level supervision, i.e. minimizing cross-entropy between predicted and target probability. This makes index prediction accuracy the primary op- timization target, which in practice does not strictly align with image quality. As a result, such an indirect training paradigm ignores the different levels of reconstruction impacts introduced by different incorrect codes, penalizing all predictions that deviate from the ground-trute truth index even if the predicted code yields visually plausible result (see Figure 2 (a)), which may cause optimization stagnation and ultimately result in sub-optimal prior modeling.
Eipil X
Predictor
Different incorrest codes distines positiony are uniformly treuted as
Different incorrest codes the same positions are unlformly treated as X.
Different incorrect codes couve varging visual impacts.
(a) Vanilla Cude-Level Cross-Entrops Prediction
Ling
STE
Prodictur
organat
(b) Image-Leve Reconstruction aware Prediction
Figure 2: (a) Code-level loss ignores the visual impacts caused by the predicting results and penal- izes all non-ground ton-grouun--ruth predictions equally. (b) Our reconstruction aware training strategy guides the predictor according to the visual impacts intro- duced by different code predictions.
In this paper, we propose novel VQ-based generative super resolution framework with Texture Vector- custor-Qaanizztion (TVQ) and Reconstruction A ware Prediction (RAP) strategies. Inspired from classical dictionary learning methods (Matsui et al., 2017; Zeyde et al., 2010; Gu et al., 2015), which remove low-frequency intensity component to improve the representation capability of dictionary, our TVQ strategy introduces visual texture codebook instead vanilla codebook for predictive prior modeling. Concretely, decompose image into the structure and the texture components; the structure component can be casily estimated by the LR input, and we only exploits texture codebook to encode the remaining texture features. Removing structure information could significantly reduce the diversity of feature space, therefore alleviating the coding error introduced by VQ and consequently improving prior modeling accuracy. An illustration of our Texture VQ strategy versus vanilla VQ paradigm can be found in Figure 1. Moreover, besides TVQ, another important innovation of our paper lies in our predictor training scheme. As we have discussed previously, most of the existing VQ-based methods (Van Den Oord et al., 2017; Esser et al., 2021; Zhou et al., 2022) train index predictor with code-level supervision which ignores the consequences of selective predicting errors. i.e. the final reconstruction emoor While, we proposes a reconstruction aware training paradigm which illustrated
directly exploits image-level reconstruction supervision for training the predictor. in Figure 2, the predictor directly takes the quality of the reconstructed image into consideration, aligning the optimization target with image quality and is expected to better capture the visual prior for generating high-quality visual data. Building upon proposed strategies, proposed model is able to achieve state-of-thhe -ftthe-rr GSR results with less computational footprints.
2

---

Published as a conference paper at ICLR 2026
The contributions of this paper summarized as follows: (i) We present tailored visual prior modeling framework for generative super-resolution, which takes inspiration from classical dictionary learning method and establish texture codebook to mitigate the encoding difficulty of highly complex visual signal. (ii) We propose an advanced training strategy for predictive visual prior modeling, which directly take the final image-level reconstruction accuracy instead of intermediate code-level predicting accuracy as target to train index predictor. (iii) We conduct comprehensive experiments on both synthetic and real-world datasets, our method able to achieve state-of-the-art generative super-resolution results with less computational footprint; detailed ablation studies are also provided validate the effectiveness of our innovations.
2
RELATED WORKS
2.1 VECTOR QUANTIZATION METHODS
The seminal VQ-VAE (Van Den Oord et al., 2017) introduced a learnable codebook discretize continuous latent representations, providing a foundation for subsequent generative modeling ap- proaches. Building upon this, VQGAN (Esser et al., 2021) incorporated adversarial losses during training, significantly improving the visual quality of reconstructed images. However, despite these advancements, the overall performance of VQ-based models remains limited by the expressive capacity of the codebook. To address this challenge, various strategies have been proposed to enhance the representational power of VQ models. These include RQVAE (Lee et al., 2022) with multi-stage recursive encoding for fine details, VIT-VQGAN (Yu et al., 2021) leveraging larger codebook and lower compression ratio for higher fidelity, and MoVQ (Zheng et al., 2022) using multi-channel quantization to boost codebook expressiveness. While these techniques improve representation capacity, they often introduce trade-offs such as increased model complexity and computational cost. Moreover, existing VQ-based methods typically use per-code cross-entropy loss for code prediction, which limits the model's ability to capture the underlying distribution of visual data.
2.2 IMAGE SUPER- RESOLUTIOO
Image super-resolution (SR) is a longstanding ill-posed problem that remains fundamental challenge in low-level vision. Traditional SR methods (Dong et al., 2012: Gu al., 2015) rely on handcrafted priors and domain-specifie knowledge to recover HR details. With the advent of deep learning, data-driven approaches have become dominant the SR domain (Dong et al., 2015; Wang et al., 2020). Early SR methods (Liang et al., 2021; Zhang et al., 2024; Long et al., 2025) optimized pixel-wise losses (e.g., mean squared error) to achieve high PSNR, but often produced overly smooth results lacking realistic textures (Ledig et al., 2017). To address this limitation, photorcalistic SR approaches adopt generative models such as GANs (Wang et al., 2018; 2021; Zhang et al., 2021) and diffusion models (Rombach et al., 2022: Yue et al., 2023; Wang et al., 2024b; Zhang al., 2025; Wu et al., 2024; Yang et al., 2024) to better capture complex image priors, leading to the reconstruction of more natural and detailed textures. Despite significant advances, GAN-based methods continue to face challenges such as training instability and difficulty balancing perceptual quality with fidelity. Diffusion-based SR methods (Yue et al., 2023; Wang et al., 2024b; Zhang al., 2025; Wu et al., 2024: Yang et al., 2024) often incur substantial computational costs during inference, which further diminishes their practicality for real-world applications.
2.3 VQ-BASED IMAGE
More recently, VQ-based super-resolution methods have emerged as promising alternatives by incorporating discrete generative priors to enhance reconstruction quality. However, since most of these methods inherit from VQ-based generative models, they face common limitations such as under-expressive codebooks and indirect optimization objectives, which lead to suboptimal predictors. For example, CodeFormer (Zhou et al., 2022) is specifically tailored for facial images, limiting its generalizability. FeMaSR (Chen et al., 2022) struggles with complex scenes, often yielding suboptimal restoration quality. AdaCode (Liu et al., 2023) introduces a molti-codeebook quantization pipeline that increases both training and inference complexity. VARSR (Qu et al., 2025), despite its strong performance, depends on a complex multi-scale residual quantization mechanism and a large pretrained autoregressive predictor, and thus shares the common limitation of diffusion-based
3

---

Published as a conference paper at ICLR 2026
Cds & Crog gradients
Ling gradients
C
Codebook Lookup Moduile
Leie
Codebook
Lookap Module
X
Structure
D
X
Predictor
Greg
feature alignment
downsample
L Limg
1/0
D
X1
D
Y
(a) Texture Vector Quantization
(b) Reconstruction aware Prediction
Figure 3: Overview of the proposed Texture Vector Quantization (TVQ) and Reconstruction Aware Prediction (RAP) strategies. (a) Texture Vector Quantization, we decompose the image into the structure and texture components, and only exploit codebook generate discrete texture features; removing the structure component could significantly reduce the complexity of visual feature space, result in enhanced texture representation accuracy. (b) Reconstruction Aware Prediction, instead of training predictor through indirect code-level supervision, introduce image-level supervision which take the reconstruction error lead by different predicting results into consideration; the predictor trained select codebook items for generating high-quality image details.
methods that pretrained generative priors, such as high computational cost. In contrast, proposed framework is explicitly designed to address these limitations. By introducing a texture- focused vector quantization scheme and incorporating image-level supervision in code prediction, our method significantly enhances representational capacity while enabling direct optimization for perceptual quality. Our method produces high-quality results while maintaining model efficiency.
METHODOLOGY
In this section, we present details of our proposed generative super-resolution method. We first introduce how high-quality images are decomposed into structure and texture components facilitate learning a texture codebook for discrete texture encoding. Then, we describe our reconstruction aware prediction training strategy which straight-through estimator (STH) (Bengio al., 2013) to train index predictor with image-level supervision.
3.1 IMAGE SEPARATION FOR TEXTURE VECTOR QUANTIZATION
The VQ-based generative model (Van Den Oord et al., 2017; Esser et al., 2021; Ramesh et al., 2021; Lee al., 2022; Tian et al., 2025; Zhou al., 2022; Yu al., 2021; Zheng et al., 2022) encodes continuous visual features with a learned codebook and trains codebook index predicting network to capture visual prior. At the core of VQ-based model is a visual codebook which comprises typical visual features to encode continuous feature in vector-quantization manner. The richness and diversity of natural images makes the latent space of visual feature highly complex space, discrete representation with guaranteed reconstruction accuracy often relies on large codebook with enormous number typical features. In this paper, we study the generative super-resolution task, for which low-resolution information is available at the inference stage. The specific character of super-resolution task inspires us to remove the available structure information and only discretization the texture information for reducing the codebook complexity.
In order to decompose high-quality images into the structure and texture components, we train multiscale autoencoder which extracts feature maps with two different resolutions, i.e. F" CC H xW and FL E
F" F4 = E(X),
where X x3xH,,XW, is the input high high-quality image, E(-) is the image encoder. We expect the low-resolution feature maps FL and high-resolution feature maps F'l encode the structure and
4

---

Published as a conference paper at ICLR 2026
texture components, respectively. To achieve this goal, generate down-sampled low rew-ruooutiinn image X E e p D and train another auto-encoder on the down- wwwn-sampledd image,
F1 (X,)) x, DD=
where (-) and (-) are encoder and decoder for down- nown-sampled image X Please note that X is an extreme low-resolution image which smaller than the low-resolution image to be super-resolved in the testing phase, which means F2 only include basic structure information of the image. With the help of Fl, we could disentangle basic structure information from X by aligning FL with FJ- Consequently, as F and vector- cuittor-quanizeeed version of F are required to reconstruct high-quality image, F is learned represent the peuucture-reooeer texture information of X. With separated image components, we introduce codebook to generate discrete texture representation via vector- quantization. Denote the texture codebook by C², for each token F#, find neartest codebook item in C to establish vector-quantized texture feature F and F -ng are combined to reconstruct the original high-qualig, image with decoder
Lookup(F" (F" C²). Lastly, F'.
X = D(F og -9 F
(3)
Following the commonly used VQ-GAN (Esser et al., 2021), we adopt MSE loss, perceptual loss and GAN loss to optimize the difference between X and X. The alignment between F' and F2 achieved by minimizing the their Euclidean distance. We use the same stop-gradient strategy as in (Van Den Oord et al., 2017; Esser et al., 2021; Ramesh et al., 2021; Lee et al., 2022; Tian et al., 2025) to deal with the back-propagation issue introduced by codebook. An illustration of Image separation framework is shown in left part of Fig. 3. More implementation details can be found in the experimental section 4.1 and appendix B.
3.2 RECONSTRUCTION AWARE PREDICTION
With the above TVQ training, we are able to represent representhighhquualeee high-quality image continuous maps FL and discrete representation FH-24, where FL and F# can be combined to generate the original high-quality image. In the second stage of training, corresponding low-rcsolution input image Y. Since X in TVQ training is with lower resolution than Y, all the information in FL can be easily regressed by Y, the major difficulty of generative SR lies in predict # from Y. In vanilla VQ-based method, a probability predictor be trained to predict the probability of codebook indexes with cross-entropy loss:
aim to predict F'. and F with the
LCE =c==-E, 1 log((.),
where H are the target codes achieved by TVQ from HR image. Although that Lee is able to guide the predictor to estimate correct code for reconstructing the high-quality image, it treats all the pre- diction errors equally and neglects the final reconstruction errors lead by different prediction choices. In order to reduce the reconstruction error, which is the ultimate target of super-resolution task, introduce image-level supervision for training reconstruction aware index predictor. Considering the forward process of predictive image reconstruction, let us denote the one-hot index as:
fone- hot =On)Hot(T.),
(5)
and the decoded texture feature is achieved by: F. 11 C (feme-hott) hot). We plug - into the pre-trained decoder in equation to generate HR estimation, and backpropagate commonly used reconstruction losses including the MSE loss, the perceptual loss and GAN loss to train the index predictor. As the decoder is differentiable, the gradient can be easily back-propagated to jone-hot through :: f To deal with the OneHot operator in Eq. equation 5, we reformulate fone-hot as:
jone-hot hot i + (fone-hol hot I.) d.).dah
in the network. The above straight-through estimator (STE) trick has been widely used in various models. We use introduce image-level supervision for training code index predictor. In addition to predicting the code indices, we also need to extract structural information from the LR input. As FL is continuous and 1 is with lower resolution than LR input, we simply MSE loss between e". and its corresponding Ft. for supervision. More implementation details can be found in the experimental section 4.1 and appendix B.
5

---

Published as a conference paper at ICLR 2026
Table 1:
Quantitative results of models on ImageNet-Test. The best and second best results are highlighted in bold and underline. ("-N" behind the method represents the number of inference steps)
Methods
PSNRT
SSIM
LPIPS
DISTSL
CLIPIQA
MUSIQT
MANIQA?
FIDU
ESRGAN (Wang al., 2018)
20.67
01.488
0.485
0.3049
0.451
43.615
0.3212
73.02
BSRGAN (Zhang et al., 2021)
24.42
0.639
0.259
0.2207
0.581
54,697
0.3865
45.63
SwinlR (Liung al., 2021)
23.99
0.667
0.238
0.2058
0.564
53.790
0,3882
35.73
RealEERRGAN (Wang al., 2021)
24.04
0.665
0.254
0.2174
0.523
52.538
0,3689
41.48
FeMaSR (Chen et al., 2022)
22.35
0.606
0.243
0.2089
0.662
$5.930
0.4721
43.39
AdaCode (Liu et al., 2023)
23.30
0.626
0.237
0,2046
0.663
53,950
0.4171
40.59
LDM-15 (Rombuch
2022)
24.85
0.668
0.269
0.2101
0.510
46.639
0,3305
30.53
ResShift-15 15 (Yue et al., 2023)
24.94
0.674
0.237
0.1716
0.586
53.182
0.4191
19.53
SinSR-I (Wang et al., 2024b)
24.70
0.663
0,218
0,1808
0.611
53.632
0.4161
25.58
UPSR-5 (Zhang et 2025)
23.77
0.630
0.246
0.2017
0.633
59,227
0.4591
TVQ&RAP (Ours)
22.49
0.603
0.210
0.1784
0.730
63,873
0,5530
26.57
(a) LR image
ResShift
SinSR
UPSR
Ours
(1) Groud truth
Figure 4: Qualitative comparison between different methods on ImageNet-Test dataset.
4 EXPERIMENTS
In this section, conduct experiments to validate the effectivness of our proposed method. We firstly introduce our experimental settings, and then compare our method with recently proposed generative SR approaches. Lastly, model analysis section is presented to validate the advantages of our proposed TVQ and RAP strategies.
4.1 EXPERIMENTAL SETTINGS
Training details. We follow the experimental settings of Yue al. (2023) and train our method on the ImageNet training set (Deng et al., 2009). For training SR model with zooming factor 4, we utilize the degradation process in (Wang et al., 2021) to generate paired low-resolution (LR) and high-resolution (HR) images. The down-sampled image X for structure disentangle is obtained by down-sampling the original image with a factor of 8. The spatial size of the structure components FL and texture components FH are 32 times and times smaller than the size of HR image, i.c. H₁ = H2//// Нμ = H,/8, with channel numbers 64 and 256, respectively. We introduce texture codebook with 1024 items, and conduct our TVQ training in Section 3.1 for 450K iterations with 512x512 images. As for the reconstruction aware prediction stage in Section 3.2, to reduce training time, we firstly train the the predictor with code-level cross-entropy loss for 300K iterations, and then finetune the predictor with image-level reconstruction aware training for another 10K iterations. Detailed network architectures can found in appendix B.2.
Testing details. Following recent work (Yue et al., 2023; Wang et al., 2024b; Zhang et al., 2025). we evaluate our method on synthetic and real-world datasets. For the synthetic setting, we utilize the ImageNNet-Test dataset following Yue et al. (2023), which contains 3,000 images randomly selected from the ImageNet validation set. Additionally, we adopt two real-world datasets, RealSR (Cai al., 2019) and RealSet65 (Yue et al., 2023), to assess the generalizability of model in practical scenarios. We report several commonly used quality measure metrics following previous works, including full-reference metrics: PSNR, SSIM (Wang et al., 2004), LPIPS (Zhang al., 2018) and

---

Published as a conference paper at ICLR 2026
Table Quantitative results of models on real-world datasets. The best and second best results are highlighted in bold and underline. Notably, as Real65 lacks ground-trute references, we report only non-reference metrics following (Yue al., 2023; Wang al., 2024b; Zhang et al., 2025).
Methods
RealSR
RealSei65
PSNRT
SSIMT
LPIPS
CLIPIQA!
MUSSQQ
MANIQA1
NIQE!
CLIPIQAT
MUSSQQ
MANIQA1
NIQU]
ESRGAN (Wang al., 2018)
27.57
0.7742
0.4152
0.2362
29.037
0.2071
7.73
0.3739
42.366
0,3100
4.93
(Zhang 2021)
0.2746
63.587
0.3702
0.6160
65.583
0,3888
ReaEESRGAN (Wang 2021)
25.83
0.7726
0.2739
0,4923
59,849
0.3694
4.68
0.6081
64,125
0.3949
4.38
FeMaSR (Chen et al., 2022)
25.43
0.7540
0.2927
0.5598
58,774
0,3430
4.76
0.6821
64.116
0.1100
5.01
AddCode (Liu al., 2023)
26.26
0,7601
0.2773
0.6092
61.279
0.3567
0.6877
64.533
0,4144
StahleR--200 (Wang
20244)
26.19
0.7556
0,2506
0,4124
48.346
0.3021
5.87
0,4488
48.740
0.3097
5.75
LDM-15 (Rombach et al., 2022)
27.18
0.7853
0.3021
0.3748
48.698
0.2655
6.22
0.4313
48,602
0.2693
6.47
ResShin--15 (Yoe al., 2023)
26.80
0.7674
0.3411
0.5709
57.769
0.3691
5,93
0.6309
59.319
0.3916
5.96
SinSR-1 (Wang 2024b)
26.01
0.7083
0.4015
19.344
0.4058
6.26
0.7164
62.751
0,4358
5.94
UPSR-5 (Zhang et al., 2025)
26.44
0.7589
0.2871
0,6010
64.541
0.3828
4.02
0.6392
63.519
0.3931
4.23
TVQ&RAP (Ours)
24.71
0.7202
0.2944
0.6897
65.591
0,4337
3.97
0.7347
68.420
0,4814
4.34
(b) BSRGAN
ESRGAN
RealESRGAN
LDM-15
LSYIRAA
(a] LK
(1 ResShin-15
SinSR
UPSR
(1 Ours
(b) BSRGAN
(C) ESRGAN
(c) ReaIESRGAN
LDM-15
LR image
ResShif-11
SinSR
UPSR
Ours
Figure 5: Qualitative comparison between different methods real-world datasets.
DISTS (Ding et al., 2020), and no-reference metrics: FID (Heusel et al., 2017), NIQE (Mittal et al., 2012), CLIPIQA (Wang al., 2023), MANIQA (Yang et al., 2022), and MUSIQ (Ke al., 2021).
4.2 COMPARISON WITH OTHER GENERATIVE SXPER-RESOLUTIIN METHODS
We benchmark our approach against several representative SR methods: ESRGAN (Wang et al., 2018), BSRGAN (Zhang et al., 2021), SwinIR (Liang et al., 2021), RealESRGAN (Wang al., 2021), FeMaSR (Chen et al., 2022), AdaCode (Liu et al., 2023), StableSR (Wang et al., 2024a), LDM (Rombach et al., 2022), ResShift (Yue et al., 2023), SinSR (Wang al., 2024b) and UPSR (Zhang et al., 2025). Table and Table 2 report quantitative results the synthetic ImageNet- and two real-world validation sets, respectively. On the ImageNet- Test dataset, our method attains the highest scores for both reference-based and no-reference perceptual metrics, while incurring minimal PSNR/SSIM degradation compared to the best models. On real-world datasets, our method either the best the second best performance across the no-reference metrics. Figure 4 and Figure
7

---

Published as a conference paper at ICLR 2026
present visual examples synthetic datasets and real-world datasets: reconstructions exhibit richer details and more realistic textures, with virtually less artifacts. More comparison and visual examples provided in appendix C, F.
In addition to superior super-resolution results, another important advantage of our model lies in its efficiency. In Table 3, compare the runtime and the number of parameters of several recently proposed generative super-resolution methods, including two VQ-based methods and several sota diffusion-based methods. Following Yue et al. (2023); Wang al. (2024b); Zhang al. (2025), report runtime (ms), params (MB), and additionally several perceptual metrics on the ImageNet-Test from Table 1 for ease of comparison. As shown in Table 3, our predictive method is able to deliver photorealistic GSR results with high efficiency. In comparison to state-of- the-art multi-step diffusion based methods, i.e. ResShift-15 (Yue al., 2023) and UPSR-5 (Zhang et al., 2025), our model is able to ob- tain comparable or better results with 5.5% and 16.5% of their runtime; comparison with dis- tilled one-step method SinSR-1 (Wang al., 2024b), our method could utilize less than 60% of its runtime obtain better GSR results. In terms of parameter count, our model also demon- strates competitiveness compared with compet- ing methods.
Table 3: We compare runtime efficiency and per- ceptual performance with state-of-the-ar methods. All models are evaluated 64x64 input images using single RTX 3090 GPU. The best results are highlighted in bold.
Methods
Runtime
Params
LPIPS..
MUSIQT
CLIPIQAT
FeMaSR
57ms
34M
0.243
55,930
0.662
AdaCode
104ms
57M
0.237
53,950
0.663
ResShin 15 SinSR.1 UPSR-5
689ms
119M
0.2371
53.128
0.586
230ms
119M
0,2460
59.227
0,633
Ours
38ms
57M
0.2101
63,873
0.730
TVO
TVO 8192
Table A comparison between Vanilla Code- book and Texture Codebook. Evaluation is con- ducted on ImageNet- Test, where 'r-' denotes re- construction metrics. Experimental details can found Section 4.3.
(8B)
8100
Taining Beration
Training Itraaton (K)
Methods
HPSNR
r-PPPS,
rFFD]
PSNRI
LPIPS]
FIDI
Figure 6: Comparisons between vanilla code-
VQ
23.29
0.1271
12.81
22.87
0.2707
44.51
book and our proposed texture codebook.
4.3 MODEL. ANALYSIS
In this part, we present detailed ablation studies to analyzes the advantages of our proposed Texture Vector Quantization (TVQ) and Reconstruction Aware Prediction (RAP) strategies.
To evaluate the effectiveness the proposed tex- ture vector quantization, we conduct ablation studies with a lightweight variant of our architec- method against vanilla
Effect of Texture
ture. We compare baseline with the structure branch removed. A series of experiments examining performance across different codebook sizes and training it- erations are presented. As shown in Figure our method consistently achieves better perfor- mance under the same codebook size and train- ing iterations. Moreover, it outperforms com- peting methods even with smaller codebooks and fewer training iterations. Notably, TVQ- 256 at 100k iterations surpasses VQ-8192 at 300k, highlighting that our approach enables more efficient codebook representation, thereby enhancing prior modeling capability. To further
imago
Vallina Codebook (d) Codebook
Figure 7: Visual comparisons between the super- resolution results with vanilla codebook and our proposed texture codebook. Experimental details can be found in Section 4.3.
8

---

Published as a conference paper at ICLR 2026
evaluate the benefit of the stronger prior for SR, we perform ablation studies on the SR task, compar- ing our method with the vanilla baseline under a codebook size 1024. For both models, we use only the code-level loss to better isolate and verify the effectiveness of the texture codebook. Both reconstruction and SR performance are evaluated on the ImageNet-Test dataset. As shown Table 4, the texture vector quantization substantially outperforms the vanilla baseline by large margin, demonstrating its superior representational capacity, which is highly beneficial for SR. Two visual examples are provided in Figure 7. The model with texture codebook could generate photorealistic images with vivid textures. The above quantitative and qualitative advantages of texture codebook over the vanilla codebook clearly validated our idea of texture vector quantization.
Table 5: A comparison between Code-Level supervision only and the integration of Image-Level supervision on ImageNet-Test. Experimental details can be found in Section 4.3.
Method
Accuracyt
DISTS!
LPIPS
FID
CLIPIQAT
MUSIQU
MANIQ1
Code-level supervision only
6,8%
0.1935
0.2159
32,876
0.6971
61.687
0.5303
+Image-level supervision
4.4%
0.1784
0.2101
26.567
0.7304
63.873
0.5530
Effect of Reconstruction Aware Prediction. To assess the effectiveness of the proposed Reconstruction-Av Aware Prediction strategy, we compare the
results of two train- ing regimes: (1) models trained solely with code- level cross-entropy loss, and (2) models further fine-tuned using image-level supervision. As re- ported in Table 5, while code-level supervision achieves better index accuracy, incorporating image-level supervision yields substantial gains in both perceptual quality and structural fidelity. This indicates that code-level loss targeting in- dex accuracy does not always directly correlate with image quality, whereas the proposed recon- struction aware prediction strategy better aligns with the goal of high-quality image reconstruc- tion and thereby significantly enhances GSR results. Figure 8 presents representative visual results from our ablation study, further corrob- orating this conclusion. Models trained with image-level supervision produce more subtle detailed textures that are often lost in code-only training. These improvements are especially pronounced in regions with complex patterns or high-frequenge de- tails. The superior GSR results achieved by Reconstruction aware prediction suggest that image-level supervision provides strong and explicit gradient signals to the code prediction network, significantly enhances the predictor's ability to generate high quality reconstruction results.
(a) LR image
(c) Code-level supervision only
(d) Image-level supervision finetune
Figure 8: Visual comparisons between code-level supervision and our proposed reconstruction aware image-level supervision. Experimental details can be found in Section 4.3.
Collectively, ablation study clearly validate the effectiveness of the proposed TVQ strategy and RAP strategy. The TVQ strategy enhances the representational capacity by focusing on texture details, while the RAP strategy improves the predictor's ability to generate perceptually accurate reconstructions through direct optimization with reconstruction-avaal supervision. With the help of the two strategies, we are able obtain otate-of-the--e generative super-resolution results with less computatinal footprint. More ablation study are provided in appendix E, G.
4.4 FEATURE MAP RESOLUTION SELECTION IN OUR ARCHITECTURE
As discussed Section 3.1, we represent an image using two components. The resolutions of the structure and texture components downsampled by factors of 32x and 8x, relative the HR image. For the texture branch, we follow the prior VQ-based super-resolution method (Zhou et al., 2022), adopting an 8x downsampling strategy. This choice balances detailed representation and computational efficiency. For the structure branch, we empirically adopt larger downsampling factor of 32x, motivated by the observation that structures be effectively captured coarser resolutions.

---

Published as a conference paper at ICLR 2026
To investigate the impact of feature map resolution on SR performance, conduct focused study using a lightweight variant. Specifically, we perform experiments with downsampling factors of 128x. 64x, 32x, 16x, and 8x relative the HR image. As shown in Table 9, although 16x and 8x downsampling achieve better reconstruction performance, the 32x configuration yields the best results in SR. We attribute the poorer SR performance at 16x and 8x the excessively large feature maps, which make it sifficult-despitt the use of alignment loss-to fully suppress texture information leakage through the structure branch. On the other hand, compared 128x and 64x downsampling, the 32x setting retains relatively complete structural information, which is beneficial for effective decoupling of structure and texture features.
Table 6: a abaltion study on different dowwssmiiin rates for the structure branch our architecture. Evaluation is conducted on ImageNet-Test, where 'r-" denotes reconstruction metrics.
Methods
r-PSNR
r-LPIPS
r-DISTS 1
r-FID
PSNR
LPIPS.
DISTS
FID
128x
24.10
0.1196
0.1210
13.03
23.50
0.2279
0.1986
35.00
64x
24.70
0.1046
0.1101
10.54
23.70
0.2241
0.1969
34.08
16x
27.65
0,0525
0.0629
4.84
24.80
0,2594
0.2424
44.60
8x
33.43
0.0147
0.0239
1.78
24.57
0.4285
0.3425
72.57
32x
25.26
0.0898
0.0988
8.76
24.01
0.2220
0.1968
33.23
CONCLUSION
In this paper, propose TVQ&RAP, a VQ-based method for generative super-resolution. To reduce the quantization error introduced by visual feature vector quantization, we decompose the image into structure and texture components and propose a texture cector-quantization (TVQ) strategy which introduce texture codebook to mitigate the difficulty in discrete visual representation. Furthermore, in order better training the prediction network, we suggest reconstruction aware prediction (RAP) strategy which utilizes the final reconstruction error to train code index predictor in an end- to-end manner. With reduced difficulty in discrete visual representation and enhanced capability in detail reconstruction, super-resolution framework. Extensive experimental results on synthetic and real-world datasets are provided to evaluatet the proposed method. Our model is able achieve state-of sare--of-heeaaee art generative super-resolution results with less computational footprint. Detailed ablation analysis are also provided to validate the effectiveness of the proposed TVQ and RAP strategies.
combine our proposed TVQ and RAP to establish novel generative
REPRODUCIBILITY STATEMENT
We provide detailed hyperparameter settings in Section 4.1 and Appendix B. To further facilitate reproducibility, will release our implementation and trained model checkpoints, enabling the reported results to be reproduced.
THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used large language models (LLMs)) aid in polishing the writing. Specifically, LLMs were employed to improve grammar, clarity, and readability of the manuscript. No part of the research ideation, methodological design, experimental analysis relied on LLMs.
REFERENCES
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: /308.3432 2013.
Jianrui Cai, Hui Zeng, Hongwei Yong, Zisheng Cao, and Lei Zhang. Toward real-world single image susor-res:l A new benchmark and a new model. In Proceedings of the IEEE/CVF international conference computer vision, pp. 3086-3095, 2019.
10

---

Published as a conference paper at ICLR 2026
Chaofeng Chen, Xinyu Shi. Yipeng Qin, Xiaoming Li, Xiaoguang Han, Tao Yang, and Shihui Guo. Real-world blind super-resolution via feature matching with implicit high-resolution priors. In Proceedings of the 30th ACM International Conference Multimedia, pp. 1329-1339 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scare scale hierarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Keyan Ding, Kede Ma, Shiqi Wang, and Eero Simoncelli. Image quality assessment: Unifying structure and texture similarity. IEEE transactions pattern analysis and machine intelligence, 44(5):2567-25811 2020.
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2): 295-307, 2015.
Weisheng Dong, Lei Zhang, Guangming Shi, and Xin Li. Nonlocally centralized sparse representation for image restoration, IEEE transactions on Image Processing, 22(4):1620- 1630, 2012.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883, 2021.
Shuhang Gu, Wangmeng Zuo, Qi Xie, Deyu Meng, Xiangchu Feng, and Lei Zhang. Convolutional sparse coding for image super-resolution. Proceedings of the IEEE International Conference on Computer Vision, pp. 1823-1831, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Computer ViCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I1 14, pp. 694-711. Springer, 2016.
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5148-5157, 2021.
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681-4690, 2017.
Doyup Lee, Chiheon Kim, Sachoon Kim. Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1.523-11532 2022.
Jingyun Liang, Jiczhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. Swinir: Im- age restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1843-1844, 2021.
Kechun Liu, Yitong Jiang, Inchang Choi, and Jinwei Gu. Learning adage-adaptive codebooks for agasstag image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5373-5388 2023.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IFFF/CVF international conference computer vision, pp. 10012-10022, 2021.
11

---

Published as a conference paper at ICLR 2026
Wei Long, Xingyu Zhou, Leheng Zhang, and Shuhang Gu. Progressive focused transformer for single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2279-2288, June 2025.
Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa. Sketch-based manga retrieval using mangal09 dataset. Multimedia tools and applications, 76:21811-21835 2017.
Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making "completely blind" image quality analyzer. IEEE Signal processing letters, 20(3)200-212, 2012.
Yunpeng Qu Kun Yuan, Jinhua Hao, Kai Zhao, Qizhi Xie, Ming Sun, and Chao Zhou. Visual autoregressive modeling for image super-resolution. arXiv preprint arXiv:2501./899) 2025.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pp. 8821-8831. Pmlr, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling:
Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839-8488 2025.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Jany Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 2555 2555-2563, 2023.
Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image super-resolution. International Journal of Computer Vision, 132(12):5929-590 2024a.
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the Euro- pean conference on compuler vision (ECCV) workshops, pp. 0-0, 2018.
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1905-1914, 2021.
Yufei Wang, Wenhan Yang, Xinyuan Chen, Yaohui Wang, Lanqing Guo, Lap-Pui Chau, Ziwei Liu, Yu Qiao, Alex C Kot, and Bihan Wen. Sinsr: diffusion-based image super-resolution a single step. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 25796 25805, 2024b.
Zhihao Wang, Jian Chen, and Steven CH Hoi. Deep learning for image super-resolution: A survey.
IEEE transactions pattern analysis and machine intelligence, 43(10):3365-33877 2020.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-61, 2004.
Rongyuan Wu, Tao Yang. Lingchen Sun, Zhengqiang Zhang, Shuai Li. and Lei Zhang. Seesr: Towards semantics-awand worldwolld image super-resolution. In Proceedings of the IÉEE/CVF conference on computer vision and pattern recognition, pp. 25456-25467 2024.
12

---

Published as a conference paper at ICLR 2026
Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujuu Yang. Maniqa: Multi-dimensien attention network for no-reference image quality assessment. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1191- 1200, 2022.
Tao Yang, Rongyuan Wu, Peiran Ren, Xuansong Xie, and Lei Zhang. Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization. In European Conference on Computer Vision. pp. 74-91. Springer, 2024.
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:2110.04627 2021.
Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Resshift: Efficient diffusion model for image super-resolution by residual shifting. Advances in Neural Information Processing Systems, 36: 13294-13307, 2023.
Roman Zeyde, Michael Elad, and Matan Protter. single image scale-up using sparse- representations. In International conference on curves and surfaces, pp. 711-730. Springer, 2010.
Kai Zhang, Jingyun Liang, Lue Van Gool, and Radu Timofte. Designing a practical degradation model for deep blind image super-resolution. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4791-4800, 2021.
Leheng Zhang, Yawei Li, Xingyu Zhou, Xiaorui Zhao, and Shuhang Gu. Transcending the limit of local window: Advanced super-resolution transformer with adaptive token dictionary. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2265,, 2024.
Leheng Zhang, Weiyi You, Kexuan Shi, and Shuhang Gu. Uncertainty-guided perturbation for image super-resolution diffusion model. arXiv preprint (xXXi:25003185122 2025.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Chuanxia Zheng, Tung-Long Vuong, Jianfei Cai, and Dinh Phung. Movq: Modulating quantized vectors for high-fidelity image generation. Advances in Neural Information Processing Systems, 35:23412-234255, 2022.
Shangchen Zhou, Kelvin Chan, Chongyi Li, and Chen Change Loy. Towards robust blind face restoration with codebook lookup transformer. Advances in Neural Information Processing Systems, 35:30599-30611 2022.
13

---

Published as a conference paper at ICLR 2026
TABLE OF CONTENT FOR APPENDIX
A The Use of Large Language Models (LLMs)
14
Implementation Details
14
B.1 Training Details
14
B.2 Network Architectures
15
Comparisons pretraining-based SR methods.
15
D Experiments on High-Resolution Scenarios
15
E Feature Map Resolution Selection in Our Architecture
15
Subjective Evaluation
16
G What Is Represented in Two Feature Map
16
H Quantitative Analysis of Reduced Feature-Space Redundancy
16
Visual Comparison
17
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
We used large language models (LLMs)) aid in polishing the writing. Specifically, LLMS were employed improve grammar, clarity, and readability of the manuscript. No part of the research ideation, methodological design, or experimental analysis relied on LLMs.
B IMPLEMENTATION DETAILS
B. TRAINING DETAILS
As discuss in section 3.1, to supervise the multiscale tokenizer, following VQ-GAN (Esser et al., 2021), we adopt compound loss including: codebook loss Leodebook. commit loss Leommits MSE loss Lmse. perceptual loss Lper (Johnson al., 2016; Zhang et al., 2018), and adversarial loss Lady (Esser et al., 2021). The overall loss function formulated as:
L Luodeboook Leammit Lisse x) Leper Auav
(
where Aud is weighting factor, set to 0.75 empirically our training. For the reconstruction task of the tokenizer applied to Xj, our objective is not generate visually perfect image, but rather to extract a meaningful feature representation that captures the basic structure information. Hence, we employ basic MSE loss:
L = Cyss (X)))
(8)
As discuss in section 3.2, we supervise the super resolution pipeline using both code-level and image- level objectives. Specifically, the code-level ground truths: F'dand ! are obtained by feeding the corresponding high-resolution image into our pretrained reconstruction network. The code-level loss consists of MSE loss for regression and a cross-entropy loss for classification:
Lende | p 112 ACE (- Σ, 1 log().)).
where ÀCE is a weighting factor that balances the two losses, empirically set to 0.5 our training. For the image-level supervision, we adopt the same loss formulation as Equation 7.
14

---

Published as a conference paper at ICLR 2026
B.2 NETWORK ARCHITECTURES
Following prior work (Esser et al., 2021; Zhou et al., 2022; Liu et al., 2023), we design multiscale VQ-tokenizer composed of residual blocks (He al., 2016) and attention layers (Vaswani et al., 2017; Liu et al., 2021; Liang et al., 2021). The tokenizer encodes the image into token maps at two spatial resolutions, with downsampling factors of and 32, respectively. The texture codebook contains N = 1024 entries. The predictor is implemented using 12 Swin-Attention blocks. This modular design ensures efficiency while maintaining strong representational capacity.
C COMPARISONS TO PRETRAINING-BASE SR METHODS.
Table Comparisons with pretraining- based SR methods on RealSR.
Method
Runtime Params Memory
[PPPSS] DISTS! FIDI
MANIQAT CLIPIQAT NIQEJ
SceSR (Wu ct al., 2024)
5740ms
2524M
8.8G
0.2806
0.1781
55.58
0.6122
0.6824
4.54
VARSR (Qu et al., 2025)
322ms
1102M
11.1G
0.3232
0.2025
61.53
0.6176
0.7020
4,49
Ours
110ms
57M
1.2G
0.2944
0.1793 54.97
0.5807
0.6897
3.97
Although methods based on pretrained generative models have demonstrated impressive performance, their dependence on large, fixed backbones restricts lightweight architectures. This significantly limits their suitability for deployment in real-world, ssoorree-cosstraine environments. Moreover, such methods typically require massive model sizes and incur substantial inference costs, placing them on a distinct path from our proposed approach. Nevertheless, for completeness, we include comparisons with some state-of-arts metsaaindd-bsse methods. Since our previous method of calculating MANIQA was different from (Wu et al., 2024; Qu al., 2025), followed their testing approach and conducted the tests again. We evaluate quality metrics on uncropped image and evaluate the Runtime and the Memory on 128 x 128 inputs using single RTX 4090 GPU. As reported in Table 7, our method achieves competitive performance while using significantly fewer parameters and requiring much less inference time. Specifically, SeeSR incurs a significant computational overhead, with 52x inference time and 44x parameters, whereas VARSR also exhibits high resource demands, requiring 3x the inference time and 19x the parameters.
when adapting to
D EXPERIMENTS ON HIGH-RESOLUTION SCENARIOS
Table 8: Comparisons on DRealSR.
To evaluate our approach under high-resolution settings, we conducted additional experiments on the DRealSR dataset, which contains real- world 4K-5K images. Table 8 shows the perior performance of our method compared to recent sota methods.
Method
CLIPIQA
MUSIQ 1
MANIQA 1
NIQE
SinSR UPSR
0.6953
30.789
0,3589
5,79
33.060
0.3220
Ours
0.7377
34.102
0.4086
3.89
E FEATURE MAP RESOLUTION SELECTION IN OUR ARCHITECTURE
As discussed in Section 3.1, we represent an image using two components. The resolutions of the structure and texture components are downsampled by factors of 32x and 8x, relative the HR image. For the texture branch, we follow the prior VQ-based super-resolution method (Zhou et al., 2022), adopting an 8x downsampling strategy. This choice balances detailed representation and computational efficiency. For the structure branch, we empirically adopt larger downsampling factor of 32x, motivated by the observation that structures can be effectively captured at coarser resolutions.
To investigate the impact of feature map resolution on SR performance, we conduct focused study using a lightweight variant. Specifically, we perform experiments with downsampling factors of 128x, 64x, 32x, 16x, and 8x relative to the HR image. As shown in Table 9, although 16x and 8x downsampling achieve better reconstruction performance, the 32x configuration yields the best results in SR. We attribute the poorer SR performance at 16x and 8x to the excessively large feature maps, which make it difficult difficult-essptte the use alignment loss-to to fully suppress texture information leakage through the structure branch. On the other hand, compared to 128x and 64x downsampling,
15

---

Published as a conference paper at ICLR 2026
the 32x setting retains relatively complete structural information, which is beneficial for effective decoupling of structure and texture features.
Table 9: A abaltion study different downsampling rates for the structure branch in architecture.
Evaluation is conducted on ImageNer- Test, where 'r-' denotes reconstruction metrics.
Methods
r-PSNR
r-LPIPS}
r-DISTS 1
r-FID
PSNRT
LPIPS,
DISTS
FID
128x
24.10
0.1196
0.1210
13.03
23.50
0.2279
0.1986
35.00
64x
24.70
0.1046
0.1101
10.54
23.70
0.2241
0.1969
34.08
16x
27.65
0.0525
0.0629
4.84
24.80
0.2594
0.2424
44.60
8x
33.43
0.0147
0.0239
1.78
24.57
0.4285
0.3425
72.57
32x
25.26
0.0898
0.0988
8.76
24.01
0.2220
0.1968
33.23
F SUBJECTIVE EVALUATION
Following the evaluation protocol of VARSR (Qu al., 2025), we conduct a user study with 15 participants. Our method was compared against five representative ISR baselines (BSRGAN (Zhang al., 2021), Real-ESRGAN (Wang et al., 2021), Resshift (Yue et al., 2023), UPSR (Zhang et al., 2025), and SinSR (Wang et al., 2024b)), using 90 images selected from three datasets: ImageNet-Tess RealSR, RealSet65 (the first 30 images from each). For each image, participants were asked to select the best restoration among the six methods. This resulted in total of 1350 responses (15 participants x 90 images). The results in Table 10 demonstrate that our method achieves the highest user preference rate (48.8%), significantly outperforming other approaches.
Table 10: Results of User Study
Method
BSRGAN
Real-ESRGAN
Resshift
SinSR
UPSR
Ours
Preference (%)
0.0%
7.7%
10.0%
21.1%
12.2%
48.8%
G WHAT IS REPRESENTED IN TWO FEATURE MAP
To show analysis that what is represented in FL and F conducted an additional analysis by passing F and F separately through the decoder to obtain corresponding reconstructions. The qualitative results in Figure 9 clearly show that: the F'--onl reconstructions preserve coarse structures and smooth areas, and the F -only reconstructions retain high- figh-freguency textures without clear structural outlines. This aligns with our ideal of feature decomposition.
Full image
Ploony
Fllolly
Full image
F--only
Fflonly
Figure 9: Qualitative comparison between different methods on ImageNet-Test dataset.
H QUANTITATIVE ANALYSIS OF REDUCED PEATURE-SPACK REDUNDANCY
To support our claim that TVQ reduces redundancy in the quantized feature space, the current manuscript provides indirect evidence: under the same (or even smaller) codebook sizes, TVQ
16

---

Published as a conference paper at ICLR 2026
Table 11: K-means clustering distortion (lower is better) on latent features under the same number of clusters K.
K (clusters)
Distortion | (VQ latent)
Distortion & (TVQ latent)
Relative reduction
64
16.53
12.96
21.6%
128
14.19
10.53
25.8%
256
11.93
8.24
30.9%
512
9.55
6.08
36.4%
Table 12: Covariance statistics of latent features. Ereg denotes a diagonally regularized covariance
matrix.
Model
Total variance tr(E) 4
log det (Ereg))g) +
VQ latent
3.24 x 10a
-6.37 x 10 -1.11 x 102
TVQ latent
2.16 x 10a
achieves lower reconstruction/quantizz error than vanilla VQ (Sec. 4.3), The intuition is that, by removing structural information already available from the LR input, TVQ narrows the content that must be represented by the discrete codebook, the latent space becomes easier to approximate with finite capacity.
In the revision, further provide direct quantitative characterization of the latent space compactness from two complementary perspectives.
(a) K-means clustering distortion under fixed capacity. We treat latent vectors as point clouds and run k-means clustering on (i) the vanilla VQ latent and (ii) the TVQ texture latent using the same number of clusters K. We report the clustering distortion as the mean squared distance from each feature to its nearest cluster center. As shown in Table 11, TVQ consistently yields lower distortion than vanilla VQ across K {64, 128, 256, 512]. and the relative reduction increases with K (from 21.6% to 36.4%). For instance, at K = 512 the distortion decreases from 9.55 to 6.08. This indicates that, for a fixed discrete capucity (same number of clusters/codewords) the TVQ texture space can be approximated substantially more accurately, implying simpler and less redundant feature distribution.
(b) Covariance structure and distribution volume. We also compute the empirical covariance matrix E of latent features for both models. Compared to vanilla VQ. the TVQ texture latent exhibits a lower total variance (i.e.. tr(2) reduced by ~ 33%) and a smaller generalized variance, measured by log dee((reeg) with small diagonal regularization. The results in Table 12 suggest that, in the same ambient dimensionality, the TVQQ texture distribution concentrates more compact region of feature space, with both reduced overall variance and reduced effective volume.
Summary. Together with the lower quantization/rrconsttutiioo enror under the same codebook size reported in Sec. 4.3, the clustering and covariance analyses above provide direct evidence that removing I. .Rprredictable structure reduces redundancy in the quantized feature space. Consequently, the TVQ texture latent more compact and easier partition with finite codebook, which aligns with our motivation for the proposed design.
I VISUAL COMPARISON
We provide more visual examples of our method compared with recent state-of-the-art methods on ImageNet-Test and real- world datasets. Visual examples shown in Figure 10, 11, 12, 13, and 14.
17

---

Published as a conference paper at ICLR 2026
LR image
(b) RealESRGAN
(c) LDM-15
(d) ResShift-15
(C) SinSR
(t) UPSR
(g Ours
(h) Ground truth
LR image
(b) RealESRGAN
LDM-15
(d) ResShift-15
(e) SinSR
(1) UPSR
gg) Ours
(h) Ground truth
aa) LR image
(b) RealESRGAN
(c) LDM-15
(d) ResShift-15
SQLiARE
POLELL
(c) SinSR
( UPSR
(g) Ours
(h) Ground truth
Figure 10: Qualitative comparison between different methods on ImageNet-Test dataset.
18

---

Published as a conference paper at ICLR 2026
(a) LR image
(b) RealESRGAN
(c) LDM-15
(d) ResShift-15
(c) SinSR
(t) UPSR
(g) Ours
(h) Ground truth
image
(b) RealESRGAN
(c) LDM-15
(d) ResShift-15
(c) SimSR
(f) UPSR
(g) Ours
(h) Ground truth
L.R image
(b) RealESRGAN
(e) LDM-MS
(d) ResShifi-15
27 Si
(0) SinSR
UPSR
Ours
Ground truth
Figure 11: Qualitative comparison between different methods on Ima-eNet-Test dataset.
19

---

Published as a conference paper at ICLR 2026
(a) LR image
(b) RealESRGAN
LDM-15
(d) ResShift-15
(c) SinSR
(t) UPSR
g Ours
(h) Ground truth
Figure 12: Qualitative comparison between different methods on ImageNet- Test dataset.
(a) LR image
(b) BSRGAN
(c) RealESRGAN
(d) LDM-15
ResShift-13
(1) SinSR
(g) UPSR
(h) Ours
(a) LR image
(b) BSRGAN
(c) RealESRGAN
(d) LDM-15
(e) ResShin--IS
(1) SinSR
(g) UPSR
(h) Ours
Figure 13: Qualitative comparison between different methods two real-world datasets.
20

---

Published as a conference paper at ICLR 2026
(a) LR image
(b) BSRGAN
(C) RealESRGAN
(d) LDM-15
(c) ResShift-15
SinSR
(g)
(h) Ours
(a) LR image
(b) BSRGAN
(c) RealESRGAN
(d) LDM-15
(e) ResShift-15
(f) SinSR
(g) UPSR
(h) Ours
Figure 14: Qualitative comparison between different methods on two real-world datasets.
21


Reviews:
[{"review_id": "u3lkC29tDW", "reviewer": "Reviewer_uFn3", "summary": "This paper introduces a Texture Vector-quantization (TVQ) method to mitigate vector-quantization encoding errors by disentangling structure and texture features, thereby reducing codebook complexity. Furthermore, the paper introduces a Reconstruction Aware Prediction (RAP) strategy, which enables the direct optimization of perceptual quality by incorporating image-level supervision to train the predictor. As a result, the proposed model achieves state-of-the-art performance. It also demonstrates advantages over competing methods in terms of inference speed and computational efficiency.", "questions": "1. The paper's core claim is that removing structure information reduces feature space redundancy. To what extent is this redundancy reduced? I would suggest providing a more quantitative analysis or visualization results of feature space to substantiate this claim.\n2. How exactly does the texture codebook improve codebook efficiency? Traditional VQ methods often face challenges with low codebook utilization or codebook collapse. I would like to see an analysis to demonstrate that TVQ achieves a higher and more effective utilization of its codebook items compared to the baseline.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The core design is intuitive. It directly leverages the inherent characteristics of the super-resolution task. The low-resolution (LR) input already contains the majority of the low-frequency structure information. Therefore, it only needs to realistically synthesize the missing high-frequency texture details. \n2. This paper clearly identify two fundamental weaknesses in traditional VQ-based models and propose a specific solution for each. TVQ is proposed to solve the problem of traditional VQ methods attempt to encode the entire complex visual space, which leads to difficult discretization and significant quantization errors. RAP is proposed to train the predictor and force it to optimize for the final perceptual quality of the reconstructed image.\n3. Comprehensive experiments and ablation studies validate the method's effectiveness and efficiency.", "weaknesses": "1. Some existing methods, such as Tokenflow, have already explored the concept of improving VQ through feature disentanglement. Therefore, the novelty of the TVQ component might be considered somewhat lacking.\n2. Check grammatical and spelling errors (e.g., \"structrue\").", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i8V07r49Ds", "reviewer": "Reviewer_g6no", "summary": "This paper introduces TVQ&RAP, a novel generative SR framework that addresses limitations in existing code prediction-based models. These models often suffer from high quantization error when encoding complex visual features and sub-optimal prior modeling because their code-level training objectives are not aligned with the final goal of image quality. The proposed solution, TVQ, first disentangles the image into a structure component and a texture component, applying VQ only to the simpler texture space. This is complemented by Reconstruction Aware Prediction, a strategy that trains the index predictor using direct image-level supervision by backpropagating final reconstruction losses via STE. Experiments on synthetic and real-world datasets demonstrate that the method achieves competitive generative SR results while maintaining high computational efficiency.", "questions": "Please see weakness points above.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The paper clearly identifies two key issues in VQ-based SR: quantization error and sub-optimal training; and proposes two intuitive solutions: TVQ and RAP.\n- The model achieves competitive perceptual quality (LPIPS, MUSIQ, etc.) while being remarkably efficient, significantly faster than diffusion-based competitors.\n- The benefits of both TVQ and RAP are validated by strong, separate ablation studies.", "weaknesses": "- The paper could better articulate the key advantages of its texture-only codebook (TVQ) compared to the direct u-net skip connection mechanisms used in other VQ-SR methods like FeMaSR or CodeFormer.\n\n- Although it is claimed than TVQ space is simpler, the final code prediction accuracy is still quite low. A deeper discussion on why this occurs and its implications would be beneficial.\n\n- It would be better if the paper could discuss how TVQ is distinguished and superior to other VQ enhancement techniques, such as the RQ-VAE mentioned in the related works. For example, comparison and analysis with VARSR might be better to highlight in main text.\n\nIn conclusion, this paper is well-written and well-motivated. The method design is simple and clear, and the model shows good performance with remarkable efficiency. The two proposed components, TVQ and RAP, are independently validated by strong ablation studies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "36l9nsmjKM", "reviewer": "Reviewer_UMmU", "summary": "This paper proposes a generative super-resolution framework that improves vector-quantization (VQ)-based models by addressing their limitations in texture representation and perceptual alignment. The authors introduce Texture Vector Quantization (TVQ), which decomposes a high-resolution image into a structural component reconstructed directly and a texture component encoded via a compact codebook, allowing the codebook to specialize in modeling fine textures rather than full image content. In addition, a Reconstruction-Aware Prediction (RAP) module is presented to train the code index predictor not only with categorical cross-entropy but also with image-level reconstruction loss, enabling gradients from perceptual and adversarial losses to influence code selection. Together, TVQ and RAP improve texture realism and perceptual quality while maintaining efficient inference. Experiments on ImageNet-Test, RealSR, and RealSet65 demonstrate competitive perceptual metrics (LPIPS, DISTS, CLIPIQA) compared to diffusion-based and GAN-based SR methods, with faster runtime and reduced codebook complexity.", "questions": "Abstract clarity:\nThe abstract states that “VQ methods do not take the final reconstruction errors into consideration.” However, in standard VQ frameworks, reconstruction loss (e.g., MSE, perceptual loss) is typically a major objective used to train the encoder, decoder, and quantizer jointly.\n\n\nPage 2 (alignment with image quality):\nThe manuscript claims that existing VQ-based methods “do not strictly align with image quality.” However, methods such as VQ-GAN and its successors explicitly introduce GAN loss and perceptual loss terms to improve reconstruction realism and perceptual fidelity. Please clarify in what sense those methods “do not align” with image quality.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The proposed VQ-based super-resolution framework performs image restoration with a single forward pass, which makes it highly practical for real-time or low-latency applications. In addition, the use of a multi-scale or pyramid-based VQ scheme is conceptually sound and aligns well with the hierarchical nature of image structures, offering an efficient way to capture both global context and fine textures.", "weaknesses": "The proposed TVQ module constructs an image pyramid and learns an autoencoder to model only the high-frequency (texture) components. However, similar multi-scale or hierarchical vector-quantization schemes have already been widely explored in the literature. In particular, the structure of VQ-VAE-2 (Razavi et al., NeurIPS 2019) also employs a coarse-to-fine multi-level quantization strategy that closely resembles the idea of separating and quantizing texture-like information. Although the present work adapts this design for the super-resolution task, several more general and prior VQ-based architectures already realize comparable hierarchical encoding mechanisms. Therefore, the algorithmic novelty of TVQ itself appears moderate rather than substantial.\n\nRecent diffusion-based methods are not included in the experimental comparison. For instance, I²SB (Image-to-Image Schrödinger Bridge, Liu et al., ICML 2023) achieves very low FID on the ImageNet super-resolution benchmark. Moreover, I²SB requires only a small number of denoising steps, which makes its inference significantly faster than conventional diffusion SR models. Therefore, it would be valuable to compare the proposed method against such recent approaches to better contextualize its performance.\n\nIn addition, verifying whether Schrödinger-bridge-based SR methods have been evaluated on the same benchmarks (e.g., DIV2K, RealSR, Urban100) would strengthen the paper’s experimental validity.\n\nSome additional clarifications are necessary to improve the manuscript. Details are listed below in the Questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tnb6eEJZdK", "reviewer": "Reviewer_ZTMa", "summary": "This paper enhances vector-quantized (VQ) models for image super-resolution by addressing the limitations of high quantization errors and sub-optimal supervision. The authors introduce Texture Vector-Quantization (TVQ), which focuses the codebook on modeling missing texture priors, and a Reconstruction-Aware Prediction (RAP) strategy that directly trains the index predictor using image-level supervision via a straight-through estimator. Experimental results demonstrate improvement over previous methods.", "questions": "As mentioned above, comparing with \"vanilla VQ-based method\" is trivial and insufficient. Why does the method outperform SOTA VQ-based methods that adopt more advanced methods? A detailed and thorough analysis against SOTA VQ-based methods is required to justify the contribution of the paper.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Clarity: The paper is well-written and easy to follow.", "weaknesses": "Originality, quality, and significance: The paper aims to establish its novelty against \"vanilla VQ-based method\", which is a trivial baseline that overlooks recent advances, e.g., those mentioned in the related work like AdaCode (Liu et al., 2023) and VARSR (Qu et al., 2025). It is very unclear how the proposed method compares with the \"multi-codebook quantization\" in AdaCode and the \"multi-scale residual quantization mechanism and a large pretrained autoregressive predictor\" in VARSR. Why it performs better than state-of-the-art VQ-based methods? Without a detailed analysis and associated experiments with them, it is very difficult to determine the originality, quality and significance of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)