RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration
Authors: Sudarshan Rajagopalan
Kartik Narayan
Vishal M. Patel
Venue: ICLR 2026 Poster (2026)
Keywords: Image restoration
generative models
low-level vision
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A generative method for all-in-one image restoration that leverages visual autoregressive models.

Abstract: The use of latent diffusion models (LDMs) such as Stable Diffusion has significantly improved the perceptual quality of All-in-One image Restoration (AiOR) methods, while also enhancing their generalization capabilities. However, these LDM-based frameworks suffer from slow inference due to their iterative denoising process, rendering them impractical for time-sensitive applications. Visual autoregressive modeling (VAR), a recently introduced approach for image generation, performs scale-space autoregression and achieves comparable performance to that of state-of-the-art diffusion transformers with drastically reduced computational costs. Moreover, our analysis reveals that coarse scales in VAR primarily capture degradations while finer scales encode scene detail, simplifying the restoration process. Motivated by this, we propose RestoreVAR, a novel VAR-based generative approach for AiOR that significantly outperforms LDM-based models in restoration performance while achieving over $\mathbf{10\times}$ faster inference. To optimally exploit the advantages of VAR for AiOR, we propose architectural modifications and improvements, including intricately designed cross-attention mechanisms and a latent-space refinement module, tailored for the AiOR task. Extensive experiments show that RestoreVAR achieves state-of-the-art performance among generative AiOR methods, while also exhibiting strong generalization capabilities.

Published as a conference paper at ICLR 2026
RESTOREVAR: VISUAL AUTOREGRESSIVE GENERA- TION FOR ALL-IN-ONE IMAGE RESTORATION
Sudarshan Rajagopalan, Kartik Narayan & Vishal M. Patel Johns Hopkins University Baltimore, MD 21218, USA (sambasa2, knaraya4, vpatel 13) jhu.edu
ABSTRACT
The use of latent diffusion models (LDMs) such as Stable Diffusion has signifi- cantly improved the perceptual quality of All-in-One image Restoration (AiOR) methods, while also enhancing their generalization capabilities. However, these LDM--baed frameworks suffer from slow inference due to their iterative denoising process, rendering them impractical for time-sensitive applications. Visual autore- gressive modeling (VAR), a recently introduced approach for image generation, performs scale-space autoregression and achieves comparable performance to that of state-of-the-art diffusion transformers with drastically reduced computational costs. Moreover, analysis reveals that coarse scales in VAR primarily capture degradations while finer scales encode scene detail, simplifying the restoration process. Motivated by this, we propose VAR, novel VAR-based gen- erative approach for AiOR that significantly outperforms LDM-based models in restoration performance while achieving over 10x faster inference. To optimally exploit the advantages of VAR for AiOR, we propose architectural modifications and improvements, including intricately designed artss-attention mechanisms and a latent-space refinement module, tailored for the AiOR task. Extensive experiments show that RestoreVAR achieves state-of-the-ar performance among generative AiOR methods, while also exhibiting strong gemeralization capabilities. Project Page:
INTRODUCTION
Image restoration complex inverse problem that aims to recover clean images from degradations, such as haze, rain, snow, blur, and low-light conditions. Recently, the paradigm of All-in-One image Restoration (AiOR) has emerged, where single network is trained to handle multiple degradation types. Fxisting AiOR methods can be broadly categorized into non-generative and generative approaches. Non-generative models such as AirNet (Li et al. 2022). PrompuR (Potlapalli et al. 2024), InstructI (Conde et al. 2025), AWRaCLe (Rajagopalan & Patel, 2024), and AdaIR (Cui et al. 2024), deterministically map degraded images to their clean counterparts. While these methods offer fast inference and reliable pixel-level restoration performance, they often fail to generalize to diverse degradations encountered in real-world scenarios. To overcome this challenge, recent works have adopted generative models that aim capture the distribution of clean images and produce more perceptually realistic outputs. Early works (Chen et al. 2022 Kupyn et al.) 2018) based on GANs (Goodfellow al.) 2020) attempted this through adversarial learning, but suffered from mode collapse and unstable training. To improve fidelity and training stability, DIffUIR (Zheng al. 2024) and DA-CLIP (Luo et al. 2023) employed pixel-space diffusion models (Ho al. 2020). However, their high computational cost makes large-scale pretraining infeasible, limiting their ability to learn strong generative priors. In contrast, recent methods such as Diff-Plugin(iuu et al.) 2024). AJiaDIR(jiang et al. 2023), and Pix Wizard (Lin et al. 2024) leverage latent diffusion models (LDMs), such as Stable Diffusion (Rombach et al., 2022). By operating in a latent space, LDMs significantly reduce computational costs, enabling large-scale pretraining which equips them with strong generative priors of natural images. These priors allow LDM-based AiOR methods to deliver perceptually realistic restoration and improved generalization real-world degradations.

---

Published as a conference paper at ICLR 2026
MAze
Mechod
Driilugin (carts 2024]
Autobtt (ECOV 2024]
Porkitard (ICLB 2023)
714
Inference Time
RestorvVAR
Lowe
Kahn
AMADII (SCCV2004) 2004)
TFLOPs (log scale)
(a) Overview
(b) PSNR (dB)f
(c) Model complexity (bubhle size or parameter count)
Figure I: VAR,, our proposed based (Tian et al., 2024) scale-space generative AiOR model (a), significantly outperforms LDM-based methods shown in (b). RestoreVAL also offers drastic reductions in computational complexity as shown in (c).
Despite their advantages, LDM-based AiOR methods have some shortcomings. (1) LDMs require multiple denoising steps during inference, resulting in significantly longer runtimes compared to non-generative models. Their slow inference speeds pose challenges for applications that demand real-time processing, such video surveillance autonomous navigation. (2) LDMs rely on variational autoencoders (VAEs) (Kingma 2013) which are primarily trained for generative diversity, rather than accurate pixel-level reconstruction. Consequently, the restored images obtained from LDM-based AiOR methods exhibit loss of fine structural details, hindering their performance.
Autoregressive models have driven rapid advances in natural language processing through large language models (LLMs) such as GPT-3 (Radford et al., (219b) and LLaMA (Touvron et al., (202)). These models generate outputs by predicting the next token, conditioned on previously generated tokens. Recently, Visual AutoRegressive (VAR) Modeling (Tian al., 2024) introduced scale-space autoregression for image generation, performing next-scale prediction the latent space of multi- scale vector-quantized VAE (VQVAE). VAR achieves performance comparable to statc-of-the-al diffusion models such as DiT-XL/2 (Peebles & Xie, 2023), while operating 45x faster. Despite its success in generative tasks, the application of VAR low-level vision tasks such image restoration remains largely unexplored. To the best of our knowledge, only two prior works: VarSR (Qu et al.) 2025) and Varformer (Wang & Zhao 2024), have used VAR for image restoration. VarSR focused exclusively on the super-resolution task, while Varformer utilized intermediate VAR features to guide separate non-generative network for AiOR. In contrast, our approach is generative and fully exploits the strong priors of the pretrained VAR model by training it directly for the AiOR task. Our analysis in Sec. 3.2 also reveals that the scale-space decomposition of VAR captures degradations predominantly in coarse scales and scene-level details in fine scales, making it well-suited for AiOR.
To this end, we introduce RestoreVAR, a novel generative approach for AiOR that addresses some of the key limitations of LDM- based approaches. Firstly, Restore VAR adopts the autoregressive structure VAR, achieving state-of-the-ar generative AiOR performance with over 10x faster inference than LDM-based methods (see Fig. I Secondly, Restore VAR employs cross-attention mechanisms conditioned on the degraded image latents, enabling the model to maintain spatial consistency and minimize hallucinations. Thirdly, to mitigate the loss of fine details by the vector quantization and VAE decoding processes, we propose a lightweight (only ~ 3% overhead) non- generative latent refinement transformer which predicts de-quantized latents from the outputs of VAR. Additionally, we fine-tune the VAE decoder to operate on these continuous latents, further enhancing reconstruction quality. Finally, through extensive experiments, we demonstrate that Restore VAR achieves ttat-oofthh-aa performance among generative restoration models, while also exhibiting strong generalization
degradations. To summarize, our key contributions are:
1. We propose RestoreVAR VAR, the first VAR-based generative AiOR framework that achieves superior performance and 10x faster inference than LDM-based methods.
2. To achieve semantically coherent restoration, we introduce degraded image conditioning through cross-attention at each block of the VAR transformer.
2

---

Published as a conference paper at ICLR 2026
3. To mitigate the loss of fine details in the vector quantization and VAE decoding processes, we introduce a non non- non-generative latent refiner transformer which converts discretized latents into continuous ones, and fine-tune the VAE decoder to operate on continuous latents.
4. Extensive experiments show that Restore VAR attains state-of-the-arr performance among generative AiOR approaches, with perceptually preferable results and strong generalization.
RELATED WORKS
2.1
IMAGE RESTORATION
Early restoration models primarily addressed specific degradations (He et al. 2009 Zhang et al.
2020 Wang al. 2019 Yasarla & Patel 2019 Zhhnng et al. 2021a Nah et al. 2017). Other methods
such as Restormer (Zamir et al.) 2022). MPRNet (aair et al. 2021) | and SwinIR (Liang et al.
2021) introduced architectures for any single restoration task. However, they are restricted to handle one degradation at time, making them ineffective for multiple degradations. All-in-One image Restoration (AiOR) methods aim to tackle multiple corruptions with single model. Early approaches
include non-generative models such All-in-one (Li et al. (020) and Transweather Saahheer(aalae (Valanarasu et al.,
2022). PromptR (Potlapalli et al. 2024) used Iearnable prompts while AWRaCLe (Rajagopalan &
Pate 2024) utilized visual in-context learning to extract degradation characteristics. Other approaches
such as InstructIR (Conde et al.) 2025) adopted textual guidance, and DCPT (Hu et al. 2025) proposed
novel pre-traine pre-training strategy for AiOR. DFPIR (Tian et al.) (2025) proposed feature perturbation strategy for AiOR. Recent AiOR methods have adopted diffusion models. Pixel-space diffusion
models (PSDMs) such as DA-CLIP (Luo al. 2023) and DiffUIR (Zheng al., 2024) demonstrated improved AiOR performance but lacked robust generative priors. Recent methods have utilized the strong priors of LDMs for AiOR. Diff-Plugin (Liu et al. 2024) adopts task plugins to guide an LDM for AiOR. AutoDIR (Jiang et al. 2023) automatically detects and restores degradations using an LDM. PixWid (Lin et al., (024) is multi-task Lumina-next (Zhuo et al. 2024) based model capable of performing AiOR among other tasks. However, 1DM-based approaches are slow at inference time, a limitation we aim to overcome using visual autoregressive modeling (VAR).
2.2 AUTOREGRESSIVE MODELS IN VISION
Recent works Van Den Oord et al. 2016 Tian et al. models to vision and can be categorized as pixel-space AR (Van Den Oord et al.
2024 have extended autoregressive (AR)
2016 Van den
Oord et al.) 2016; Chen et al., 2018b), token-based AR (Van Den Oord et al. 2017
Yu al. 2023
Ramesh et al. 2021 and scale-space AR (Tian et al. 2024: Ren et al. 2024; Guo et al. 2025
Pixel-space AR predicts raw pixels one by one in raster order, as in PixelR NN (Van Den Oord et al.)
2016) and PixeICN++ (Salimans et al. 2017). but is very slow at high resolutions. Token- AR compresses images into discrete latent codes via vector quantization (e.g., VQ-VAE (Van Den Oord ct al. 2017), VQGAN (Esser et al. 2021 ) and then models code sequences with transformers (e.g.
ImageGPT (Chen et al. 2020)). This trades-off codebook size and transformer capacity against tractability for high-resolution generation. Scale-space AR, as introduced in VAR (Tian et al. [024]. generates latents from coarse fine scales and matches the quality of Diffusion Transformers (Peebles)
& Xic 2023) at a fraction of the inference cost. HART (Tang et al. 2024) scales VAR to higher resolution and uses a MLP-based diffusion refiner to convert discrete VAR latents into continuous
representations. Despite VAR's 's success in generative tasks, it remains underexplored for image restoration with only two prior works- VarSR (Qu et al. 2025) and Varformer (Wang Zhao 2024). VarSR addressed super-resolution, while Varformer used VAR's features to guide non-generative AiOR model. In contrast, Restore VAR is a generative model which directly trains VAR for AiOR.
3 PROPOSED METHOD
We first explain the working principles behind VAR for image generation. We then describe our scale-space analysis of VAR and detail Restore VAR, our proposed VAR-based approach for AiOR.
3

---

Published as a conference paper at ICLR 2026
3.1 PRELIMINARIES: VISUAL AUTOREGRESSIVE MODELLING
Visual Autoregressive Modelling, or VAR, is a novel autoregressive class-conditioned image generation method which uses GPT-2 (Rafford al. 2019a) style decoder-only transformer architecture for next-scale prediction. The VAR trans- former operates
the latent space of Given an image I E #x3 the VQ- VAE encoder outputs a latent representation fcont E IR x Hereafter, we will re- fer to fcont as the continuous latent, and the latent obtained after quantization as discrete latent. Instead directly quantizing fcont. multi-scale residual quantization using shared codebook across K spatial scales is per- formed. First, the residual and accumulated quantized (or discrete) reconstruction of fooot initialized as fies' feom and sqp := O, respectively. At each scale k = 1,..., K, an index map Fı xH W& is obtained by quan- tizing the downsampled residual feature:
multi-scale VQVAE which uses K scales,
Degraded Go-rce GT+ffine
GT
Haze
Blur Low-light Rain Snow
Tk == quantize (downsample
The indices Гą are then decoded using the codebook embeddings e(-), upsampled to match the full resolution, and refined using a convolutional module (-x(-) to obtain
Figure 2: VAR captures degradations in early scales (coarse) and scene-level details in later scales (fine). Degraded and GT are VQVAE reconstructions of the degraded and ground truth images. GT+ replaces early GT scales with degraded ones, while GT+fine replaces the late GT scales.
ha : th (upsample (e(rx))), E Hx XC
This is done to approximate the information captured the current scale which used to update the residual continuous features to be modelled by subsequent scales as {(
f(lmm))) + hi:
f(k)
:= foomt
f
This process is repeated for all scales and yields a set of index maps {ri, r2,,,, each consisting of the code-book indices of residual information an increasingly finer scale.
For training, VAR uses teacher-forcing, where the ground-truth index maps (r1, "r....,,k)) are used autoregressively predict the next scale. For each scale k, the accumulated reconstruction Squatt = -ーー Φ (ensample(((rr))) is interpolated to the resolution of scale to obtain ((A) which is then flattened into tokens, and concatenated with the remaining tokens form the input sequence. A start-of-sequence (SOS) token, derived from the class label embedding, is then prepended to this input sequence. A block-wiss causal attention mask is used ensure that predictions for scale A attend only to the previous scales. VAR is trained to minimize the cross-entropy loss between predicted logits and the ground-truth index maps, modeling the likelihood
p(1 T2. TR) II p(rk P1.7
During inference, the SOS token is created from the target class label. VAR then autoregressively predicts each index map Tk2 one scale at a time. After predicting Tk. its embedding is upsampled, refined and accumulated to form the input for the next scale, mimicking the same procedure used during training. The VAR model uses only K = 10 latent scales with key-value (KV) caching, enabling significantly faster inference compared latent diffusion models.
3.2 SCALE-SPACE ANALYSIS OF VAR
In addition to VAR's competitive performance LDMs with far superior inference speed, found that its residual scale-space decomposition focuses on degradations and scene-level details across different scales. To demonstrate this, we consider clean degraded image pairs and compute
4

---

Published as a conference paper at ICLR 2026
Gleee 2ED feck Lndirr dra.ee LINE
Logand
Jay Degaded Image
Urresdest clan clanirgge
dgg Contnaan Aanuollagy
Cresenes loue by
Agg Conditciin
YAll
(i)) Luiest refinct tuiling
reddbonk uumiinaamnal Wed
lg
Taxiler Say contleeennn femiii (60 Traiking Astorgressie Traesderne (a) Training RestoreVARl
120 lepario rundeme
Cibaasstaa Expaa
(M) YAS Dececor faneuuing
Techedding teee by blick munalerne
Prodica Bncnne
Foos
V.AU V.A.-S-kkk
Fen
Cints-enneyy Toss
Aclecnot Trenkere
Lilsu
Credowice of Fnsses
Baotze dacedor
Triccale
Prose modak
Inferenes ResorEVAR
Figure 3: Illustration of Restore for training and inference. (a) Shows the training procedure for each component of Restore and (b) shows the overall pipeline during inference.
their scale wise residual indices {reT) T==1 and (r coarse scales as = =1,... -,...,5 5 (low-resolution index maps) and fine scales as = -6..., 10 (higher- resolution index maps). The first and last columns of Fig. show reconstructions from T3 Deg and Deg This
Jk=1 respectively, where K = 10. We define
T respectively. In column 2, replace the coarse scales of r with those from rk introduces the degradation, although fine scales remain unchanged. Next, in column we replace the fine scales of GT with those from rxx details. These observations indicate that coarse scales in VAR capture degradations, while finer scales encode scene-level detail. Notably, this observation holds across multiple degradations and simplifies restoration for VAR as removing degradations requires correctly predicting only the early scales which contain small number of tokens, while scene details can be reconstructed in subsequent scales.
Deg which yields a clean image with some loss of fine
3.3 RESTOREVAR
We now describe RestoreVAR, our proposed approach that effectively adapts VAR for AiOR, leverag- ing its substantial inference speed advantage over LDMs. Given a degraded image leg E k x3 the goal is predict clean output Ielean. close to the ground-trute truth Ig.. Adapting VAR to AiOR is non-trivial due to the need for high-quality pixel-level reconstruction, which is compromised by two factors: (1) VAR's strong generative priors can cause hallucinations in the restored images without proper conditioning. (2) Vector quantization and VAF decoding introduce artifacts that hinder pixel-level restoration. Restore VAR addresses these challenges through architectural enhancements, including roos-attention to incorporate semantic guidance from the degraded image, and novel non non-generative transformer that refines discrete latents into their continuous form to preserve fine details in the restored image. We describe these components below.
3.3. AUTOREGRESSIVE TRANSFORMER ARCHITECTURE
For training. the multi-scale teacher-forcing input is constructed from the ground-truth image Ige (see Sec. B.II. The -f-quecee (SOS) token is computed from fixed label index and augmented with a global context vector derived from the degraded image (see supplementary for details). These features are flattened and concatenated into a token sequence fquant E IR LxC where the total number of tokens across all scales (see Fig. (()) The VAR transformer is then trained to autoregressively predict the next-scale indices {r {rguk=n
RR4 of the clean image.
To enable semantically consistent restoration, we inject information from the degraded image through cross-attention at each transformer block. At block i, the queries are given by the output of the feed-forward network (Zaleck E RR LD where D is the embedding dimension). while keys and values derived from the continuous latent of the degraded image, fdeg e " XxxXx This latent is reshaped into a sequence of conditioning tokens and is appropriately projected to the embedding dimension of the transformer. As shown in Sec. 4.4 conditioning continuous latents significantly
5

---

Published as a conference paper at ICLR 2026
outperforms conditioning discrete ones. To summarize, cross-attention (CA(:.)) is applied as Wblockea -block, + Di x CA (A(bblock; caml)
We initialize 9: O to retain VAR's pretrained behavior and gradually introduce conditioning. Fur- thermore, we replace absolute positional embeddings in VAR with 2D Rotary Positional Embedddings (RoPE) for scaling resolution from 256 x 256 to 512 x 512, as RoPE is well-suited for handling vary- ing sequence lengths (Su et al. 2024). We also remove AdaLN layers, reducing ~ 100M parameters with negligible impact performance. Inference closely follows that of VAR (see Sec. 3.1 except that each scale prediction is now guided by the degraded latent. The output is a sequence predicted K indices { p prea km 1. which is then used to construct the discrete restored latent fquan E 2 Wic xc The above steps are shown in Fig. (ii). More architectural details are given in the supplementary.
3.3.2 DETAIL- RESTORATION
The discrete latent (quan) predicted by the RestoreVAR transformer is decoded by the VQVAE to produce the restored image. However, qutto--uantizztionn and VAE decoding cause a noticeable loss of fine details in the pixel-space, leading distorted reconstructions. This presents a major challenge for using VAR in AiOR, as the scene semantics may not be accurately preserved. To address this, we introduce VAE decoder fine-tuning on continuous latents, and lightweight latent refinmment transformer (LRT) that converts discrete latents to continuous latents for decoding.
VAE Decoder Fine- Tuninn, HART (Tang al., 2024) addressed VAE-induced distortions by fine- tuning the VAE decoder on both discrete and continuous latents. While effective for generative tasks, the VAE decoder HART produces overly textured outputs, compromising accurate reconstruction (see supplementary). Instead, we fine-tune the decoder only on continuous latents, bypassing the quantizer. The encoder and quantizer are kept frozen, and the decoder is trained on (J Ig) pairs. To avoid overly smooth outputs, we use a PatchGAN (Isola et al. 2017) discriminator (see Sec. 4.4 and optimize the decoder using pixel-wise, perceptual, and adversarial losses
Ldee = A1 L + Ay + X + Aa
where LL is the L1 loss, LSSIM is the SSIM loss, Epeceep is the perceptual loss, Ladv is the adversarial loss and Aj are their respective weights (see Fig. Fig. (jxiv) Our fine-tuning approach yields a decoder that is well-aligned with the objectives of AiOR, achieving mean (over 1000 samples) reconstruction PSNR/SSIM scores of 28. 1 44BB/0.842 .82, outperforming both the VAR VQVAE (22. (22.59dB/O.c) 679) and HART decoders (26. 486.484B/0.804 Qualitative comparisons are given in the supplementary.
Refining Discrete Latents. Since the VAE decoder is fine-tuned for continuous latents, the predicted discrete latent, fquam must be converted into a continuous form for decoding. While HART uses a 37M parameter diffusion- based MLP for this, it incurs ~ 20% inference overhead due to iterative denoising. Instead, we propose lightweight, non-generative latent re- finement transformer (LRT) that predicts a resid- ual, which when added to Iquan produces a con- tinuous latent, fooat E "x x XWxXC XC
Continuous Figure 4: Illustration of images decoded from
Input Discrete Refiner
discrete and continuous latents, along with the refiner's 's predicted residuals.
foomt fored + LLL( LRM( 2),
where E IR LR is the output from the final Restore transformer block. = is passed through cross-attention and provides pseudo-continuous guidance to the LRT which is critical for performance (see Sec. 4.4h. The LRT is trained using L1 loss between the predicted and ground-
continuous
latents (f LLKT = (ffeee f ) Our LRT introduces only 3% additional overhead and significantly outperforms HART's refiner in PSNR and SSIM scores (see Sec. (44). The training procedure of the LRT shown in Fig. 3 (ii) and Fig. provides visual example of its predictions.
Thus, Restore combines the VAR transformer, LRT, and fine-tuned decoder to deliver fast, perceptually realistic, and structurally faithful results. Fig. (b) depicts inference of Restore
4 EXPERIMENTS
In this section, we provide implementation details, comparisons with existing Al-in--One image Restoration (AiOR) approaches, and present ablations key components of our framework.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparisons of Restore VAR with the stale-of-the-art LDM-based generative AiOR approaches, and non- -on-geeerattve methods. RestoreVAR significantly outperforms generative methods on PSNR, SSIM and LPIPS scores. The best generative approach is indicated in bold.
Method
Venue
RESIDE
Snow100k
Rain13k
LoLd1
GoPro
PSNKT SSIMT LPIPS)
PSNKT SSIMI
LPIPS)
PSNR1
SSIM?
LPIPSL
PSNKT SSIMI LPIPS,
PSNR? SSIMP LPIPS]
Non-gouerative methods
PromptiK
NeurltS
0.952
31.98
0.924
0.296
0.250
InstrienR
20CVV224
26.90
0.922
01885
0.088
0.835
0:132
28.26
0.870
0.146
AWRaCLe
2A
30.81
0.999
0.013
30.56
0.904
0.088
31.26
0.908
0.068
21.04
0.818
0.146
26.78
0.820
0.248
DCPT
25
29.10
0.968
0.017
24.11
0.766
0.203
23.67
0.863
0.106
27.92
0.877
0.169
DFPIR
CVPR
31.39
0.979
0.012
24.87
0.794
0,171
25.12
0.853
0.123
28.66
0.884
0,158
Generatioe muthiads
Dift-Plugin
C4
0.765
21,02
D.611
0.196
21.71
0.617
0:713
0.195
21.76
0:217
AutoDIR
20CN
24.48
0.780
0.811
19.00
0.515
0.347
23.02
0.642
0.162
19:43
0.766
0.135
23.55
0.700
0.168
PoxWirand
IC1RR22
21.28
0.738
0.142
21.34
0.594
0.206
21.38
0.596
0.180
15.84
0.629
0. 3.505
20,49
0.602
0.223
RAstoreYAR (Ours)
24.67
0
0J074
24.05
0.713
0.156
23.97
0.700
0,153
21.72
0.782
0.126
23.96
0.737
0.167
Input
Diff- Diff-Pluggin
AII
PixWizart
Restore
GT
RESIDE
Rain 13K
GoPro
Figure 5: Qualitative comparisons of Restore VAR with LDM-based generative AiOR approaches. RestoreVAR achieves consistent restoration with enhanced preservation of fine-details.
4.1 IMPLEMENTATION DETAILS
Each component of Restore VAR was trained independently to disentangle learning objectives. We used the VAR model of depth 16 as the transformer backbone and trained it with the AdamW optimizer (Loshchllov & Hutter 2017), learning rate (LR) of 10-1, batch size of 48, for 100 epochs. The latent refiner was trained for 100 epochs with the AdamW optimizer, LR= 10-1 and a batch size of 96. The VAE decoder was fine-tuned using a weighted loss combination (see Sec. (33.2) with empirically chosen weights: 2 = 2.0, X2 0.4, A = 0.2, and A4- = 0.01. Fine-tuning was performed for 5 epochs with learning rate of 3 x 10-4 and a batch size of 12, using Adam W. Training was conducted on 8 RTX A6000 GPUs, while inference was done on an RTX 4090 GPU.
4.2 DATASETS
We trained Restore for five tasks: dehazing, desnowing, deraining, low-light enhancement and deblurring. For dehazing, we used the RESIDE (Li et al. 2019) dataset comprising 72135 training and 500 test images. The Snow 100k dataset (Liu et al. 2018) was used for desnowing, with 50000 training and 16801 test images (heavy subset). For deraining, we used 13K (Zamir al. 2021 consisting of 13711 training and 4298 test images. The LOLvI (Wei et al. 2018) dataset was used for low-light enhancement, consisting of 485 training and 15 test images. For deblurring, we used the GoPro (Nah et al. (017) dataset comprising 2103 training and 1111 test images. We also assess generalization performance on real-world unseen and mixed degradation datasets, namely, LHP (Guo) et al. (023) (1000 images), REVIDE (Zhang al. 2021b) (284 images), TOLED (Zhou et al. 202 (30 images), POLED (Zhou et al. 202 (30 images), CDD (Guo et al.) (024) (200 images, mix of haze and rain), and LOLBlur (Zhou et al. 2022) (482 images, mix of low-light and blur). TOLED and POLED datasets contain unseen degradation of under-display camera restoration.
4.3 COMPARISONS
We compare Restore VAR with sfate-of-the-ar generative and non- sen-eeeeraaive methods for AiOR. For non-generative approaches, we include PromptIR (Potlapalli et al. 2024), InstructIR (Conde et al.
7

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative comparisons of Restore VAR against state-of-the-art non-generative and genera- tive approaches on real-world, unseen and mixed degradations. The best result among Restore VAR
and non -on-eeneraeive approaches is indicated in bold.
Method
LHP
REVIDE
TOLED
POLED
LOLBluc (L B)
CDD (H R)
Averuge
MUSIQT
CLIPIQAT
MUSIQT
CLIPKJAT
MUSIQT
CLIPSQAT
MUSIQT
CLIPIDAT
MUSIQ1
CLIPRIAT
MUSIJ
CLIPIQAT
MUSIQT
CLIPIQA"
Prompelk
56,790
0.366
51.191
0.459
43.215
0.251
34.53
0.300
33,691
0.166
65,895
0.483
49.219
0.343
InstniclR
$8,269
0.359
63.116
0,416
44.985
0.258
23.317
0.341
00.221
0.200
65.491
0.483
49.500
0.333
AWRaCLL
57.859
0.335
59.267
0.368
44.670
0.285
90.533
0.332
38.186
0.171
66.253
0.454
51.470
0.329
DCPT
58.044
0.446
0.314
38.138
37.393
68:460
0.544
51.681
0.366
DFPIR
56.453
0.450
0.276
35.668
36.277
0.349
47.611
0,310
DitPPuggia
57.351
0.420
63,483
0,406
0.311
35.096
400054
0.212
68.791
0.578
51333
0.389
AutaDir
58,085
0,380
63.918
0.416
0.341
46.642
08.575
0.571
0.386
PisWisand
0.411
68
0.420
0.339
44.
0.348
50.708
0.268
68.409
0.589
56,022
0.394
RAstoreVAR
57.652
0.414
63,562
0.883
52.374
0.338
48.118
0.276
46.644
0.214
68.941
0.572
56.317
0,383
2025) and DFPIR (Tian et al. 2025). Among generative methods, we compare with the LDM-based approaches Diff-Plugin (Liu et al. 2024), AIR (Jiang al. (023) and FixWiz (Lin al. 2024). To ensure fair comparison. we retrained PrompiR and AWRaCLe, as their official checkpoints were not trained for most of our AiOR tasks. All other methods were evaluated using their publicly released checkpoints. For AutoDIR, we report results without the structure correction module, as this module functions as an independent, non-generative restoration network (more details in supplementary). The results reported for Pix Wizard were obtained using its publicly released checkpoint. We do not compare with task-specific restoration models, Restore VAR is proposed for the AiOR setting.
2025), AWRaCLe (Rajagopalan & Patel 2024). DCPT (Hu et al.
Table 1 presents PSNR, SSIM and LPIPS scores on the RESIDE, Snow 100k, 13K, LOLv1 I and GoPro datasets. Restore VAR surpasses LDM-based AiOR methods at a fraction of their computa- AutoDIR: 8, 8,477s, PixWizard: 8.247s and RestoreVAR 0.201s, highlighting the efficacy of our framework. More detailed complexity comparisons are given in the supplementary along with derivation showing that the time complexity of VAR with maximum latent resolution n X n is (log n) lower than an LDM operating at the same latent resolution. Qualitative comparisons with LDM-based methods in Fig. 5 further illustrate that RestoreVAR produces restored images of high quality while better preserving fine details. Visual re- sults for the Snow100k and LOLv1 datasets methods achieve better scores, it is important to recognize that the performance of Restore VAR is inherently influenced by the quality of the VAE decoder; a limitation shared by all latent generative approaches. Despite this constraint, Restore VAR narrows the gap with non-generative methods while maintaining the benefits of a generative framework, i.e., perceptually realistic results and strong gen- eralization capabilities. To demonstrate these strengths, we evaluate generalization using no-reference image quality metrics (following prior works (Liu et al.,
tional cost (inference time (s) per image) Damnage-iiff----e Diff-Plugin:2.000
provided in the supplementary. While non-generative
2024 Jiang et al., 2023 Rajagopalan & Patel 2024h), and assess perceptual realism through a user study.
2021) and CLIP- IQA (Wang et al., 2023) scores in Table 2 on the real-world, unseen and mixed degradation datasets discussed in Sec. 4.2 RestoreVAR achieves higher scores than gon-generrttvee models (on average), indicating better ro- bustness under these degradations. Qualitative results for this experiment are shown in Fig. where Restore VAR consistently outperforms non-generative approaches. Due to space constraints, qualitative comparisons with PromptlR and visual results for LOLBlur are given in the supplementary. Furthermore, RestoreVAR achieves performance competitive to those of LDM-based meth- ods on these datasets. The slightly lower metries for Restore VAR can be explained by the fact that the latent diffusion backbones (Stable Diffusion (Rombach et al.) 2022)) used by the competing generative models are trained on substantially larger datasets. typically hundreds of millions of images, whereas the VAR backbone is trained on only 1M ImageNet-IK IK (Deng et al. (009) images. Despite this, Restore VAR demonstrates comparable generalization performance while offering much faster inference and improved pixel-level fidelity. To further evaluate perceptual quality, we conducted a user study in which participants rated outputs from non-generative models, AutoDIR (LDM-basee) and Restore for 50 real-world scenes. We received 36 responses with each participant scoring outputs based on scene consistency, restoration quality, and overall appeal on 5 5-point scale. Table 3 shows that Restore received the highest average ratings (across all three criteria), highlighting its ability to produce images that align closely with human preferences. More quantitative results are provided in the supplementary.
For testing generalization, we report MUSIQ (Ke et al.
Table 3: Mean scores from user study.
Method
Score
PromptlR
2.11
InstrnctI
2.93
AWRaCLe
2.33
DCPT
2.42
DFPIR
2.35
AutoDIR
3.68
RestoreVAR
4.36
8

---

Published as a conference paper at ICLR 2026
Input
InstruetR
AWRaCLe
DCPT
DFPIR
Restore
GT
LHP
REVIDA
RE
TOLED
POLED
CDD
Figure 6: Qualitative comparisons RestoreVAL with non gon-generattve methods on real, unseen and mixed degradations. Restore VAR consistently achieves better results.
To summarize, although non-generative AiOR models offer fast inference and good pixel-level fidelity, their generalization to real-world, unseen, and mixed degradations is limited. They work well only when test samples come from a distribution close to their training data, which hinders real-world applicability. In contrast, LDM-based generative approaches exhibit substantially stronger perceptual quality and generalization, but they are computationally expensive and suffer from low pixel-level fidelity. Restore addresses the limitations of LDM-based approaches while retaining their strengths. achieves far superior pixel-level fidelity as established by the multi-task restoration performance on datasets in Table Furthermore, on the real-world and unseen degradation datasets (Table 2). Restore VAR achieves perceptual quality on par with diffusion-basede methods while significantly outperforming non-generative models, demonstrating strong generalization. Moreover, Restore VAR is substantially faster than LDM-based approaches and even nears the efficiency of some generaeneeee methods (see supplementary for details), despite requiring multiple steps for inference. Together, these results demonstrate that Restore closes the fidelity gap between LDM-based generative models and non-generative approaches while retaining strong perceptual quality and generalization, and offering enhanced computational efficiency.
We discuss limitations Restore VAR the supplementary.
4.4 ABLATIONS AND ANALYSIS
Continuous Discrete Conditioning. The Re- VAR transformer conditions on the continuous latent of the degraded image (fate) While condi- tioning with discrete multi-scale latents appears more aligned with VAR's multi-scale prediction objective, it results in significantly worse performance. To demonstrate this, we train Restore VAR with discrete and continuous conditioning for 15 epochs each. As shown Fig. 7 Restore VAR with discrete condition- ing exhibits much lower validation accuracy.
Table 4: Ablations on the types of latent refin- ers. Our proposed latent refiner transformer (LRT) performs best, with minimal overhead.
Refiner Type
Time (s) Params (M) PSNR / SSIM
Refiner HART Refiner
21.71 / 0.690
0.0455
36.06
23.48 / 0.777
LRT w/o Last-Block
0.0036
14.61
21.23/00660
Proposed LRT
0.0061
22.97
24.67/0.821
Discriminator for VAE fine-tuning. As described in Sec. 3.3.2 we fine-tune the VAE decoder on continuous latents using combination of pixel-level loss and an adversarial loss. To analyze the impact the discriminator, we compare the reconstructions of VAE decoders fine-tuned with and without the adversarial loss. As shown in Fig. removing the discriminator leads to blurrier reconstruction while including it yields sharper and perceptually better looking outputs. Ablations on SSIM and perceptual losses are provided in the supplementary.

---

Published as a conference paper at ICLR 2026
cerobaece Canation Condicon
Via baliccoon
Input image
w/o Disc
W Disc
Epoch
Figure 8: Image reconstructed by VAE decoders fine-tuned continuous latents with (w) and with- out (w/o) a diseriminator (Disc).
Figure 7: Validation accuracy of Restore VAR under discrete vs. continuous conditioning.
Latent Refiner Transformer. The Latent Refiner Transformer (LRT) is critical for preserving pixel-level detail in restored images. To analyze its impact, we compare four RestoreVAR variants: (i) No refiner, (ii) HART's diffusion refnne,, (iii) LRT without final block outputs, and (iv) our proposed LRT. As shown in Table our LRT achieves the best PSNR and SSIM, while maintaining low inference time and a low parameter count.
Using no refiner yields poor PSNR/SSIM scores.
Removing the last block outputs significantly reduces performance, indicating its impor- tance pseudo-continuous guidance for finement. HART's MLP diffusion-barte re- finer performs worse than LRT while having a much higher parameter count and runs 7x slower. To assess if the LRT causes loss of perceptual quality, we calcu- lated MUSIQ and CLIPIQA scores (higher is better) with and without the LRT. Using no LRT yields MUSIQ/CLIPIQA score of 66.56/0 066.6/00477 /0.472 while incorporating the LRT re- sults in a score of 66.45/0 40.4488..88 The scores nearly identical indicating negligible im- pact on perceptual quality.
Degraded
coarse
R
Res.
Haze
Snow
Rain
Blur Low-ll light
Scale-space analysis of outputs. A core mo- tivation behind our approach is that the scale- space residual quantization of VAR captures degradations at coarse scales and scene level details at finer scales (see Sec. 3.2 Fig. 9 visualizes this for the outputs of Restore VAR by comparing (i) the degraded image, (ii) the restored image with its coarse scales replaced by those of the degraded input, (iii) the re- stored image with only fine scales replaced, and (iv) the restored image. Similar to our observations Sec. 3.2 replacing the coarse-scale components of the restored output reintroduces the degradation, whereas replacing only the finer scales does not. This once again demonstrates that degradations predominantly reside in the coarse-scal scase indices, and that Restore VAR predicts coarse- scale tokens that are mostly free of degradation while reconstructing scene-level details at the finer scales.
Figure 9: Scale-space analysis of outputs from RestoreVAR. Res. denotes the Restored output. Res. ccarsse replaces early scales of restored out- put with degraded ones, while R fine replaces the late scales. It be observed that the coarse scales contribute most to overall restoration.
More ablations are provided in the supplementary.
CONCLUSIONS
We proposed Restore VAR, fast and effective generative approach for AiOR. Built on the VAR backbone, Restore VAR benefits from VAR's strong generative priors and significantly faster inference compared to LDMs. To tailor VAR for AiOR, we introduced cross-attention mechanisms that inject semantic information from the degraded image into the generation process. Additionally, we proposed a non-generative latent refiner transformer to convert discrete latents to continuous ones, along with VAE decoder fine-tuned continuous latents, which together improve reconstruction fidelity. RestoreVAR achieves sfate-of-the-an performance among generative AiOR models, outperforming LDM-based methods while delivering over 10x faster inference and strong generalization.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/ Interior Business Center (DOI/IBC) contract number 140D0423C0076. The U.S. Government is authorized to reproduce and distribute reprints for Govermmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DOI/IBC, or the U.S. Government.
ETHICS STATEMENT
We acknowledge that we have read and adhered to the ICLR Code of Ethics. For all our experiments, used publicly available open-source datasets.
REPRODUCIBILITY STATEMENT
Our code will be made publicly available after the review process. Details reproduce the work have been provided in Secs. and 4.1 of the main paper and Sec. of the supplementary.
Sec.
REFERENCES
Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun. Learning to see in the dark. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3291-3300, 2018 Sa.
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International conference on machine learning, pp. 1691- 1703. PMLR, 2020.
Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, and Pieter Abbeel. Pixelsnail: An improved autore- gressive generative model. In International conference on machine learning, pp. 864-872. PMLR, 2018b.
Xiang Chen, Jinshan Pan, Kui Jiang, Yufeng Li, Yufeng Huang, Caihua Kong, Longgang Dai, and Zhentao Fan. Unpaired deep image deraining using dual contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2017-2026, 2022.
Marcos V Conde, Gregor Geigle, and Radu Timofte. Instructir: High-quality image restoration following human instructions. In European Conference on Computer Vision, pp. 1-21. Springer, 2025.
Yuning Cui, Syed Waqas Zamir, Salman Khan, Alois Knoll, Mubarak Shah, and Fahad Shahbaz Khan. Adair: Adaptive all-in-one image restoration via frequency mining and modulation. arXiv preprint arXiv:2403. 11.4403..144444 14614, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.
leee, 2009.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high- high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883, 2021.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-14) 2020.
Hang Guo, Yawei Li, Taolin Zhang. Jiangshan Wang, Tao Dai. Shu-Tao Xia, and Luca Benini. Fastvar: Linear visual autoregressive modeling via cached token pruning. arXiv preprint arXiv:2503.23367, 2025.
11

---

Published as a conference paper at ICLR 2026
Yu Guo, Yuan Gao, Yuxu Lu, Huilin Zhu, Ryan Wen Liu, and Shengfeng He. Onerestore: A universal restoration framework for composite degradation. In European conference on computer vision, pp. 255-272. Springer, 2024.
Yun Guo, Xueyao Xiao, Yi Chang, Shumin Deng, and Luxin Yan. From sky the ground: large- scale benchmark and simple baseline towards real rain removal. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12097-12107, October 2023.
Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1956-1963, 2009. doi: 10.1 200./CCPR.0099520655
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
JiaKui Hu, Lujia Jin, Zhengjian Yao, and Yanye Lu. Universal image restoration pre-training via degradation classification. arXiv preprint arXiv:2501.15510, 2025.
Phillip Isola, Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1124-1134, 2017.
Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, and Jinwei Gu. Autodir: Automatic all-in-one image restoration with latent diffusion. arXiv preprint vrXiv22310.10123 2023.
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5148-5157, 2021.
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 2013.
Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiří Matas. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8183-8192, 2018.
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng. Wenjun Zeng, and Zhangyang Wang. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28 (1):492-505, 2019. doi: 10. 1009/T1..018..8866
Boyun Li, Xiao Liu, Peng Hu, Zhonggin Wu, Jiancheng Lv, and Xi Peng. All-in-one image restoration for unknown corruption. 2022 IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17431-17441, 2022. doi: 10.1109/CVPR522882222..00
Ruoteng Li, Robby T. Tan, and Loong-Fah Cheong. All in bad weather removal using architectural search. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3172-3182, 2020. doi: N0.109//VVR44260.2220000
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In 2021 IEEE/CVF International Conference Computer Vision Workshops (ICCVW), pp. 1833-1844, 2021. doi: 10. 1021 00210.
Weifeng Lin, Xinyu Wei, Renrui Zhang, Le Zhuo, Shitian Zhao, Siyuan Huang, Huan Teng, Junlin Xie, Yu Qiao, Peng Gao, et al. Pixwizard: Versatile image-to-imane visual assistant with open- language instructions. arXiv preprint arXiv:2409.15278, 2024.
Yuhao Liu, Zhanghan Ke, Fang Liu, Nanxuan Zhao and Rynson WH Lau. Diff-plugin: Revitalizing details for diffusion-basee low-level tasks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4197-4208, 2024.
Yun-Fu Liu, Da- Wei Jaw, Shih-Chia Huang, and Jenq-Neng Hwang. Desnownet: Context-aware deep network for snow removal. IEEE Transactions on Image Processing, 27(6):0064 3073, 2018. doi: 10.1 10.1109/TIP.201 2806202.
12

---

Published as a conference paper at ICLR 2026
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:171..0S011 2017.
Ziwei Luo, Fredrik K Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B Schön. Controlling vision-language models for universal image restoration. arXiv preprint urXiv:2310001018, 3(8), 2023.
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3883-3891, 2017.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Vaishnav Potlapalli, Syed Waqas Zamir, Salman H Khan, and Fahad Shahbaz Khan. Promptir: Prompting for all-in-one image restoration. Advances in Neural Information Processing Systems, 36, 2024.
Yunpeng Qu Kun Yuan, Jinhua Hao, Kai Zhao, Qizhi Xie, Ming Sun, and Chao Zhou. Visual autoregressive modeling for image super-resolution. arXiv preprint a7X19:2501.18993, 2025.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019a.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 10199,
Sudarshan Rajagopalan and Vishal M Patel. Awracle: All-weather image restoration using visual in-context learning. arXiv preprint arXiv:2409.0020 00266, 2024.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pp. 8821-8831. Pmlr, 2021.
Sucheng Ren, Yaodong Yu, Nataniel Ruiz, Feng Wang, Alan Yuille, and Cihang Xie. M-var: Decoupled scale-wise autoregressive modeling for high-quality image generation. arXiv preprint arXiv:2411.10433 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelenn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv: 05 2017.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv22010.02502. 2020.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputiig, 568: 127063, 2024.
Haotian Tang. Yecheng Wu, Shang Yang. Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid autoregressive transformer. arXiv preprint Xrv::22410. 10812, 2024.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839-84866 2024.
Xiangpeng Tian, Xiangyu Liao, Xiao Liu, Meng Li, and Chao Ren. Degradation-awarr feature perturbation for all-in-one image restoration. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 28165-28175, 2025.
13

---

Published as a conference paper at ICLR 2026
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
Jose Valanarasu, R. Yasarla, and V. M. Patel. Transweather: Transformer- based restoration of images degraded by adverse weather conditions. In 2022 IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pp. 2343-2353, 2022.
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelenn decoders. Advances in neural information processing systems, 29, 2016.
Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks In International conference on machine learning, pp. 1747-1756. PMLR, 2016.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Cunzheng Wang. Ziyuan Guo, Yuxuan Duan, Huaxia Li, Nemo Chen, Xu Tang, and Yao Hu. Target- driven distillation: Consistency distillation with target timestep selection and decoupled guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 7619-7627, 2025.
Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 2555-2563, 2023.
Siyang Wang and Feng Zhao. Varformer: Adapting var's generative prior for image restoration. arXiv preprint arXiv:2412.21065 2024.
Tianyu Wang. Xin Yang. Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson W.H. Lau. Spatial attentive single-image deraining with a high quality real rain dataset. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1262-11277, 2019. doi: 10.1109/CVPP 2019.01255.
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv: 04560, 2018.
Rajeev Yasarla and Vishal M Patel. Uncertainty guided multi-scale residual dearning-using a cycle spinning for single image de-raining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8405-8414, 2019.
Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. Video probabilistic diffusion models in projected latent space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18456-18466, 2023.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Ming-Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. In CVPR, 2021.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022.
He Zhang, Vishwanath Sindagi, and Vishal M. Patel. Joint transmission map estimation and dehazing using deep networks. IEEE Transactions on Circuits and Systems for Video Technology, 30(7): 1975-1986, 2020. doi: 10.11091CSSV.20099299...4
Kaihao Zhang, Rongqing Li, Yanjiang Yu, Wenhan Luo, and Changsheng Li. Deep dense multi- scale network for snow removal using semantic and depth priors. IEEE Transactions on Image Processing. 01:7419-7431, 2021a. doi: T0.1009/1..2221.31100
Xinyi Zhang, Hang Dong, Jinshan Pan, Chao Zhu, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Fei Wang. Learning to restore hazy video: A new real-world dataset and a new method. In CVPR, pp. 9239-9248, 2021b.
14

---

Published as a conference paper at ICLR 2026
Dian Zheng, Xiao-Ming Wu, Shuzhou Yang, Jian Zhang, Jian-Fang Hu, and Wei-shi Zheng. Selective hourglass mapping for universal image restoration based on diffusion model. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
Shangchen Zhou, Chongyi Li, and Chen Change Loy. Lednet: Joint low-light enhancement and deblurring in the dark. In European conference on computer vision, pp. 573-589. Springer, 2022.
Yuqian Zhou, David Ren, Neil Emerton, Sehoon Lim, and Timothy Large. Image restoration for under-display camera. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pp. 9179-9188, 2021.
Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang, Wenze Liu, Xiangyang Zhu, Fu- Yun Wang, Zhanyu Ma, et al. Lumina-next: Making lumina-12x stronger and faster with 2024.
next-dit, Advances in Neural Information Processing Systems,
15

---

Published as a conference paper at ICLR 2026
A OVERVIEW OF SUPPLEMENTARY
In this supplementary, we first present detailed computational complexity comparisons of Restore and LDM-based AiOR approaches. We then provide a theoretical analysis comparing the time complexities of VAR and LDMs. Subsequently, we analyze the effect of using Absolute Positional Embeddings (APE) versus Rotary Positional Embeddings (RoPE) (Su et al.) 2024) when scaling the resolution from 256 x 256 to 512 X 512. Next, we present more architectural details of Restore followed by breakdown of the runtime and parameter count for each component Restore VAR. We then provide additional visual results, which include qualitative results for the continuous vs. discrete conditioning ablation, visual comparisons of VAE decoders, refiner ablation, and more qualitative comparisons with other methods. Subsequently, structure correction module of AutoDIR (Jiang et al. 2023 behaves as an independent non-generative restoration network. We then discuss the limitations of approach and scope for future work. Finally, we mention the usage of LLMs in the paper. To summarize, the supplementary discusses the following:
provide experiments to show that the
1. Detailed computation complexity comparisons (Sec.
2. Theoretical complexity comparison with LDM (Sec.
3. Additional Ablations (Sec. D
(a) Performance analysis: APE vs. RoPE (Sec. D.I
(b) Additional VAE loss ablations (Sec. D.2
(c) Additional scale-space analysis (Sec.
D.3
(d) Additional LRT analysis (Sec. D.4)
4. Additional Quantitative Results (Sec.
(a) User Study (Sec.
E.
(b) Generalization Metrics (Sec.
(c) More Perceptual Quality Evaluations (Sec.
E.3
5. Additional Architectural Details (Sec.
F
6. Runtime and Parameter Breakdown (Sec. G
7. Additional Visual Results (Sec.
(a) Continuous vs. Discrete Conditioning (Sec. H.l
(b) Qualitative comparisons of VAE decoders (Sec.
H.2
(c) Visual Comparison of Refiner Variants (Sec. H.3
(d) Additional Qualitative Comparisons (Sec.
H.4
8. More details about AutoDIR comparison (Sec.
9. Limitations and scope for future work (Sec.)
10. LLM Usage (Sec. K
B DETAILED COMPUTATION COMPLEXITY COMPARISONS
Restore VAR achieves substantial performance improvements over LDMMbased AiOR ap- proaches a fraction of their computational cost. To show this, we compared Restore VAR with Diff-Plugin (Liu et 2024), AutoDIR (Jiang et al. 2023), and Pix W Wizard (Lin et al. 2024) in terms of inference steps, runtime, TeraFLOPs, and total parameter count. As shown in Ta- ble [] RestoreVAR achieves a 10x speed-up up Diff-Plugin and 16x reduction in TFLOPs. Compared to AutoDIR and Pix Wizard, Restore over 40x faster in inference.
Table 5: Comparison of the computational com-
plexity of RAs with LDM-based AiOR approaches.
Method
Steps Time (s) TFLOPs Params (M)
Diff-Plugin
20
2.04
16.08
859.50
AutoDIR
100
8.477
67.80
859.50
Pix Wizard
60
8.247
19.27
2011.40
RestoreVAR
10
0.201
1.05
296.95
16

---

Published as a conference paper at ICLR 2026
PSNR vs DOWS staps
PSNR vs Infernnce time
923
PanlorreNn 123,57 13- 125)
samp rig XAGP
Brbromne Utres (uecrrch)
(a) PSNR DDIM steps
(b) PSNR vs inference time
Figure 10: Comparison of LDM-based methods accelerated using DDIM sampling and Restore
RAR achieves best results.
Additionally, conducted experiment speed up LDM sampling using DDIM Song al. (2020) sampling and compared the performance with Restore Specifically, we varied the number of DDIM sampling steps 50, 40, 30, 20, 10, 5 and 2. Figs. T0 a) and (b) shows the variation of mean (across the datasets from Table 1) PSNR (dB) scores with inference time (seconds) and DDIM sampling steps. Diffusion models match the inference time of RA. only when using around sampling steps, but show over 3dB decrease in PSNR compared to Restore Even at higher step counts with DDIM sampling, diffusion models lag by over 1dB, highlighting Restore VAR's s clear advantages in both speed and performance over LDM based methods.
Consistency models and pectified-flow approaches are promising recent developments in accelerating diffusion models. However, both require substantial modifications to the training pipeline. Consis- tency models demand fine-tuning pre-trained diffusion models with a new consistency objective that enables few-step inference. However, using very few steps can cause over-smoothed outputs that are undesirable for image restoration (Wang et al., 22025). Rectified-flow methods learn a deterministic velocity field along straight-line path between noise and data, using flow-matching objective.
Although these frameworks speed up inference, they still lack the scale-space (Sec. 3.2) insight offered by VAR, which elegantly fits the objective of all-in-one image restoration.
Table 6: Comparison of the computational complex- ity of Restore VAR with non- generative approaches.
We also provide computational complexity comparisons with non- generative methods in Table 66, Non-generative models are generally faster since RestoreVAN like other generative approaches, requires multiple inference steps. Nevertheless, Restore maintains com- petitive inference speed to methods such as PromptIR, AWRaCLe, and DFPIR, even though it contains more parameters and requires a multi-step generation process. While is slower than methods such as DCPT and InstructIR, the substantially improved generalization and superior perceptual quality offered by RestoreVAN make a more robust choice.
Method
Time (s) Params (M)
PrompüR
0.162
35.59
InstructR
0,023
15.84
AWRaCLe
0.188
186.68
DCPT
0.024
67.88
DFPIR
0.117
182.38
VAR
0.201
296.95
C THEORETICAL COMPLEXITY COMPARISON WITH LDM
We now provide theoretical comparison of the run-time complexities of VAR and diffusion transform- ers (DiT), offering fundamental insights into their efficiency differences. The VAR time complexity derivation closely follows that in the original VAR paper.
Let a > 1 be the geometric factor of the vector quantized (VQ) scale pyramid and let the largest scale have dimensions h n. Let the number of scales be K log, n + 1 so that the side length
17

---

Published as a conference paper at ICLR 2026
at scale i is a'-1 and the largest scale is n.K a-k Assume standard self-attention transformer as in VAR with time complexity ((T²) for T tokens.
VAR. At the generation of the k-th scale the total number of tokens across the current and previous scales (fi,-.., rkk
k
2
Σ
n
2(
a²
Hence the cost of generation of the k-th scale is
2k
-1
2
a -
Summing over all K scales gives
VAR
Σ
K
a 2k
1
(ath -2 + +).
(a² - 1) (a2 - 1)2 k=
km
Substituting K = loga + 1 (so that azk
a²n²² and aak = ann²)) yields
a (n 1) 2a 20 (a²n² - 1) + (log, n + 11)
VAR
(a² - 1)
a
The asymptotic time complexity is governed by the dominant term:
a
VAR
(a 1)(a² 1)2 n²
O
Diffusion. Assume self-attention DiT where each diffusion step the fixed largest resolution X n., i.e. n² tokens. So, single step costs
Op = (n²)2 = n
With the same number of forward steps as VAR, namely K = loga n + 1, we get
K
Diff
(log, n + 1)n* = Onn log n). n
1=1
Comparison.
From the above,
CDiff
(log, 12) n²
a
[a 1a-=-)(a-11) nd (a 1) 1)2 log, O(logn).
CVAR
That is, with the same number of forward passes, VAR totals O(n²) complexity while diffusion totals On log n), yielding a Ollog n) speedup for VAR.
D ADDITIONAL ABLATIONS
In this section, we provide additional ablations and analysis of Restore and its components.
D.1 PERFORMANCE ANALYSIS: APE vs. ROPE
As discussed Sec. 3.3 we replace the absolute position embeddings (APE) used in VAR Tian et al.) (2244) with Rotary Positional Embeddings (Su et al.] 2024) (RoPE). We found that us- ing RoPE yields better performance when scaling the resolution from 256 256 to 512 x 512.
18

---

Published as a conference paper at ICLR 2026
Input image
L.1 only
I. + SSIM
w/o Disc
W Disc
Figure 12: Image reconstructed by VAE decoders fine-tuned on continuous latents with various objective functions.
To demonstrate this, we conducted an ablation where both APE and RoPE-based variants were fine-tuned at 512 512 resolution for 10 epochs. As shown in Fig. [1] the RoPF-based model achieves higher validation accuracy compared to the APE-based model, indicating its effective- ness.
AR
Valication Accuracy (163
D.2 ADDITIONAL VAE LOSS ABLATIONS
We now provide ablations on fine-tuning the VAE decoder without SSIM and perceptual losses. Fig. 12 illustrates that using only LI loss produces blurry results, adding the SSIM losses enhances structural details and incorpo- rating the perceptual loss improves sharpness while preserving the structure (w/o Disc col- umn). Incorporating the discriminator enhances sharpness even further.
Epoch
Figure 11: Validation accuracy comparison of APE and RoPE-based fine-tuning 512 x 512 resolu- tion. RoPE demonstrates better performance.
D.3 ADDITIONAL. SCALE-SPACE ANALYSIS
We provide an analysis similar Fig. 2 for real-world degradations using the POLED and REVIDE datasets. The results in Fig. T3 indicate that the scale-space decom- position of VAR tends to capture real-world degradations
coarse scales while pre-
serving scene-level details in finer scales. This suggests that the scale-space behav-
ior observed in the synthetic examples of Fig.
2 also extends to real-world degradations.
Degraded
GT coorse
GT+
GT
D.4 ADDITIONAL LRT ANALYSIS
REVIDE POLED
In this section, we analyze the LRT module can perform restoration without relying on the VAR transformer. Toward this aim, we conduct two experiments:
Reomt
ydg
+ LRT ddg quants 2),
Figure 13: Our analysis that VAR captures degra- dations in early scales (coarse) and scene-level de- tails in later scales (fine) also tends to hold for real degradations from POLED and REVIDE datasets. GT+coarse replaces early GT scales with degraded ones, while GT fine replaces the late GT scales.
quant
and
foorr pdog + LRT edeg
(
2).
dog where qquant and edgg denote the quantized and continuous VAE latents obtained directly from the degraded input. Note that as input to the LRT since the LRT is trained to utilize to predict a meaningful residual, Zeroing giving incorrect guidance to the LRT. Fig. 14 provides
passing
out leads to significant artifacts
19

---

Published as a conference paper at ICLR 2026
Input
ply quant
L
quant:
Seom
LRT
[F
conts
GT
Figure 14: Qualitative analysis of the LRT module operating directly on the degraded quantized and continuous latents. The prediction of the LRT remains hazy, indicating that the LRT cannot restore images without the VAR transformer.
the results for these experiments. Despite having access to the restored latent's representation 2, the LRT is unable recover the clean latent (and so is the VAE decoder). This demonstrates that the LRT and the VAE decoder cannot independently perform restoration. These experiments reaffirm that restoration is carried out mainly by the VAR transformer, while the LRT and decoder serve to enhance fidelity by operating on VAR's restored latent representation.
E ADDITIONAL QUANTITATIVE RESULTS
In this section, provide additional quantita- tive analysis of the results in Table Table 2 and T
Welcome survey!
question, google link provided when you open the lins, would find a set images tach image conslets
imoges stitched side side. The imgge on the left ls that of scene degraded roin, haze or ow-light, etc (irput image). The image on the right is the somee scere with dagraduttinss removed (restored imagge). Blased un the vieual cifference, please ossees the restored image on 3 metrics:
E. USER STUDY
We now provide rigorous statistical analysis for user study, by computing paired differences between RestoreVAR and each method across all 36 responses (A Restore VAR Method), and report 95% confidence intervals (CT), paired t-tests, and Wilcoxon signed-rank tests. The results, summarized in Table show that Re- storeVAR significantly outperforms all methods: every mean improvement A is positive and the 95% CIs do not cross zero. Furthermore, the t-tests and Wilcoxon signed-rank tests report p 10-4 These results demonstrate that the advantages of Restore VAR over re-geenerttive methods are statistically significant across user study responses.
I. Scone conelstency: How similar le the soone content (tine detcills such text) between the input ond restored imoges or in on not istent, very consistent)
Restoration performance: much degrodkttio
soale (1: very degrodation removed, 5: of diegracction removadd)
3. Overall quality: How good le the viauci quality of the restorad on scala on 1-5 (t poor quality, 8: high cuality)?
Every image in the drive folder has number at the beginning of it. Theee questions need to be onawered by entering the score for each matric for oach image number (given the question)..
Figure 15: Screenshot of the user study instruction page.
We also provide additional details for the user study methodology. We selected 50 images for evaluation and collected 36 independent re- sponses. Participants evaluated each result on five-point scale for (i) restoration performance,
20

---

Published as a conference paper at ICLR 2026
Please fill tabie each with number 'X' the beginning (separated by where 'X' the first column the below table.
Soe araterrsy
Resicatoonpetorreee
Owndd qpadity
Figure 16: Screenshot of the user study data collection page.
Table Statistical analysis of user study results using 95% confidence intervals (Cls), paired t-test and Wilcoxon signed-rank test. A denotes the mean paired difference between Restore VAR and each method.
Method
95%
p (I-test)
p (Wilcoxon)
AutoDIR
0.694
+0.257
6.49 x 10
3.06 x 10-3
AWRaCLe
2.028
10.245
7.15 x 10
2.91 x 10-11
PromptR
2.250
10.221
1.12 x 10
2.91 10-11
DCPT
1.944
10.262
2.15 x 10
2.91 x 10-11
InstructIR
1.417
+0.293
3.28 x 10-11
1.66 10-6
(ii) overall quality, and (iii) scene consistency. The survey conducted using Qualtries (Provo, UT). Screenshots of the survey are given in Figs. and [6] Fig. 15 shows the introduction page of the user study. The user would be provided with randomly sampled google drive link consisting of restored outputs of different methods, where each method is assigned a number. The user needs to evaluate each of these images on the afore-mentioned criteria and enter the scores in the table shown in Fig. 16
E.2 GENERALIZATION METRICS
We additionally perform statistical significance analysis for quantitative results in Table 2 Specifically, we compute per-image paired differences between Restore VAR and each non non-generative method on MUSIQ and CLIPIQA and apply dataset-weighte statistical tests to ensure that each dataset contributes equally to the global statistic. As shown Table Restore VAR consistently achieves positive improvements with tight CIs and extremely small p values. Once again, these results confirm that Restore VAR achieves statistically significant gains in perceptual image-quality metrics non-generative methods.
E.3 MORE PERCEPTUAL QUALITY EVALUATIONS
We provide no-reference metrics MUSIQ and CLIPIQA which gauge perceptual quality for the test sets Table where there significant gap between non-generative methods and Restore VAR on pixel-level metrics. From the MUSIQ/CLIPIQA scores provided in Table 9 it can be observed that the perceptual quality of Restore VAR on-par or even higher than those non-generative methods, indicating perceptually comparable results. However, due lower pixel-level fidelity than non-generative approaches, Restore VAR scores lower on pixel-level metrics.
F ADDITIONAL ARCHITECTURAL DETAILS
We now provide additional architectural details for the Restore framework. We first describe the details for the RAstoreVAR transformer, followed by the Latent Refiner Transformer (LRT).
Restore VAR Transformer. We adopted the VAR model with a transformer depth of 16, i.c., the architecture consists of 16 transformer blocks. The structure of each block is illustrated in
21

---

Published as a conference paper at ICLR 2026
Table 8: Statistical analysis for results from Table Jusing using 95% confidence intervals (CIs), paired t-test and Wilcoxon signed-rank test. A denotes the weighted mean paired difference between Restore VAR and each method.
MUSIQ
CLIPIQA
Method
A
95% CI
p-value
A
95% CI
p-value
AWRaCT.e
5.084
+0.554
2.5 x 10
0.0536
+0.0080
8.5 x 10
DCPT
5.206
10.613
10
0.0166
10.0079
3.9 10
DFPIR
8.276
-0.757
2.5 x 10-71
0.0733
#0.0093
3.6 x 10
InstructIR
6.984
-0.902
1.3
10-42
0.0495
+0.0065
1.8 x 10-41
PrompiIR
7.002
=0.648
3.3 x 10
0.0395
+0.0068
8.4 x 10-27
Table 9: Comparison perceptual quality of Restore and non-generative methods on RESIDE, Snow 100k, Rain 13K, LOLvI and GoPro datasets using MUSIQ and CLIPIQA metrics.
Method
GoPro
Rain13K
Average
MUSIQU
CLIPIQAT
MUSIQ1
CLIPIQA!
MUSIQU
CLIPIQA"
MUSIQU
CLIPIQA1
MUSIQ
CLIPIQA!
MUSIQT
CLIPIQA!
PrompiR
35.87
50.24
0.30%
66.54
0.434
64.63
63.29
0,417
0:357
53.06
AWRaCLe
44.953
0.173
19%
0.459
61.938
0.346
64.683
0.598
62.334
0,512
59.421
0:418
DCPT
46.12
0.180
67.73
D:404
66.46
0.432
62.63
0.437
60.73
0.363
DEPIR
48.03
0.190
66.77
0.409
0.423
62.28
0.432
0.364
RestoreVAR
55.45
71.47
0.396
66.45
0.481
63.83
01487
Fig. a7)..
a). The embedding dimension was set 1024, and the number of attention heads used was 16. Furthermore, the transformer predicted discrete latents at the following spatial resolutions in the latent space:
2, 3 x 3, 4 x 4, 6 6, 9 9. 13 13, 18 18, 24 24, and 32 x 32. The start-of-sequence (SOS) token is constructed by augmenting the class embedding with the mean value (along spatial dimensions) of the features obtained after learnable projection applied on fee Specifically.
SOS classemb + Isos X Mean(Proj (oe SOS R
Here, classemb is the class token embedding and Isos is initialized as 0 for gradual incorporation of degradation conditioning. Other notations follow Sec. 3.
Latent Refinr Transformer. The LRT follows a similar structure for the blocks as the Restore VAR transformer, as shown Fig. b7).). It was configured with a depth of 12, six attention heads, and an embedding dimension of 384.
fooet
Lagen!
Quiput
Imput
Qurput
Sef-Vitetlin
Cos-Allerice
Figure 17: Illustration of a transformer block in (a) Restore VAR transformer and (b) Latent Refiner Transformer.
G RUNTIME AND PARAMETER BREAKDOWN
In this section, we provide a breakdown of the run- time and parameter count for the following com- ponents of the RestoreVAR framework: the VAE, the RestoreVAR VAR transformer, and the Latent Refiner Transformer (LRT). This analysis provides insights into the distribution of the computational cost across the pipeline. As shown in Table [0] the majority of inference time is taken by the autoregressive Restore- VAR transformer.
Component
VAE Transformer Refiner
Time (s)
0.0086
0.1863
0.0061
Parameters (M) 108.95
273.98
22.97
Table 10: Compute time and parameter count breakdown for each component of Restore- VAR. VAE time includes both encoding and decoding.
22

---

Published as a conference paper at ICLR 2026
H ADDITIONAL VISUAL RESULTS
We now present additional visualizations for some of the ablations discussed in Sec. 4.4 along with more qualitative comparisons across methods.
H.1 CONTINUOUS vs. DISCRETE CONDITIONING
As shown in Sec. 4.4 conditioning RestoreVAR on the continuous latent of the degraded image yields significantly better performance compared using the quantized discrete latent. Fig. [8]ttherr illustrates this using visual comparisons between the model trained with discrete and continuous conditioning. The model trained with discrete conditioning exhibits noticeably more hallucinations than the one trained with continuous conditioning.
(a) Input
(b) Discrete
(c) Continuous
(d) GT
Figure 18: Qualitative comparisons Restore under discrete vs. continuous conditioning.
VAR with discrete conditioning exhibits more ballucinations than the variant with continuous conditioning.
H.2 QUALITATIVE COMPARISONS OF VAE DECODERS
our fine-tuned VQ-VAE decoder achieves superior reconstruction performance compared to the decoders of VAR and HART. Fig. 19 provides qualitative results to illustrate the same. Our de- coder produces the best reconstruc- tion.
As mentioned in Sec.
(a) Input
(b) VAR
(c) HART
(d) Ours
Figure 19: Qualitative comparisons of the input reconstructed using VAR (Tian et al.) 2024). HART (Tang et al. 2024 and Our VAE decoder. Our result has minimal distortions.
H.3 VISUAL COMPARISON OF REFINER VARIANTS
In Sec. 4.4 we demonstrated that our proposed Latent Refiner Transformer (LRT) achieves the best performance compared using refiner, refiner without last-block conditioning, and HART (Tang) et al. 2024)> 's diffusion-be based refiner. Quantitative results, reported in Table 4 included mean PSNR and SSIM scores the RESIDE (Li et al. 2019) test set. Fig. 20 presents qualitative comparisons for these configurations. can be observed that our LRT preserves fine details more effectively than the other variants.
23

---

Published as a conference paper at ICLR 2026
(a) Input
(b) No refiner (c) w/o last block (d) HART
(c) Our LRT
(f) GT
Figure 20: Qualitative results for the ablation latent refiner configurations. Our proposed LRT preserves fine details better than the other configurations.
H.4 ADDITIONAL QUALITATIVE COMPARISONS
In this section, provide additional qualitative comparisons of Restore with state-of-the-ae
LDM-based and non non-generative all-in-one image restoration (AiOR) methods. Fig. 21 presents results from Restore VAR alongside Diff-Plugin(Liu et al.) 2024), AutoDIR (Jiang et al. (023).
and Wizard (Lin et al. 2024) on the RESIDE (Li et al. 2019), Snow100k (Liu et al. 2018).
Rain 13K (Zamir al., 2021), LOLvI (Wei et al., 2018), and GoPro (Nah et al. 2017) datasets.
Restore VAR consistently produces outputs that are more semantically aligned with the ground truth (see zoomed-in patches).
Fig. 22 provides comparisons with non-generative methods-PPompplle Promptlo (Potlapali et al., 2024).
Instruct (Conde et al., 2025), AWRaCLe (Rajagopalan & Patel (024). DCPT (Hu et al. 2025)
and DFPIR (Tian et al. 2025 for real-world mixed and unseen degradation generalization
LHP (Guo et al. (023)) REVIDE (Zhang et al. (021b). TOLED (Zhou et al. 202T), POLED (Zhou al. 2021), CDD (Guo al. 2024) and LOLBlur (Zhou et al. 2022) datasets. Restore generates
sharper, more realistic outputs with fewer artifacts than non non-generative models. For instance, for the
TOLED and POLED cases, RestoreVAR outputs are visibly sharper than non-generative methods.
Similarly, the results of Restore are superior in the case of mixed degradations.
I MORE DETAILS ABOUT AUTODIR COMPARISON
As mentioned in the main paper, comparisons with AutoDIR (Jiang et al. 2023) were conducted without its structure correction module (SCM). AutoDIR consists of a latent diffusion model (LDM) for initial restoration, followed by SCM which is non-generative post-processing network. The intuition behind this approach is that the SCM predicts a residual based on the degraded input image and the restored output of the LDM, to correct the VAE-induced distortions. In short,
I Tesult Tss F (TTad Taog)),
where Laa is the restored output from the LDM, laeg the original degraded input image, and F(-) denotes the SCM which operates on the concatenated inputs [Td.- /ace) However, we found that instead of slightly modulating the structural details in Lgg, the SCM behaves like separate non- generative restoration model which directly restores Lieg. To show this, we evaluated AutoDIR with the SCM on the RESIDE (Li et al., 2019) dataset for two cases: (1) using Tu as the actual L.D output and (2) setting Taa 0, effectively removing any structural information from the LDM. If the SCM were functioning as a corrective module, performance in the second case should deteriorate significantly. However, we found that the SCM was able to independently restore the degraded input in the second case, as shown in Fig. 24 This suggests that the SCM largely ignores the LDM output and instead performs direct restoration Iaeg, thereby behaving as a non-generative restoration network. Therefore, to ensure a fair comparison with other generative models, we evaluated only the LDM output of AutoDIR.
24

---

Published as a conference paper at ICLR 2026 Input
Dift- Dift-Plugin
AutoDIR
PixWizard
Restore
GT
Rain13K Snow 100K Snow 100k RESIDE
YaBoo
YaBoo
YABOOo
YABOO
YAHlo!
YAHoO
LOLvI
LOLyI
GoPro
Figure 21: Additional qualitative comparisons of Restore VAR with LDM-based AiOR approaches. RestoreVAR consistently preserves fine details more effectively than the LDM-based methods.
J LIMITATIONS AND SCOPE FOR FUTURE WORK
Despite the strengths of Restore there remains scope for improvement. First, its performance inherently constrained by the latent refiner transformer (LRT) and the VAE decoder. While the LRT significantly improves results over using no refiner, it does not reach the upper bound by directly decoding from ground-truth continuous latents. Exploring improved VQVAE and refiner architectures could help address this. Another promising direction is to employ our non- generative LRT in fully generative VAR models, given its strong performance for AiOR. Finally, future work can investigate how the performance of Restore VAR scales with larger VAR backbones. Additionally, we found that Restore VAR does not perform well when the input degradation is extremely severe. Fig. 25 shows the result for very dark low-light sample from the SID (Chen eet al. 2018a) dataset. The restored output looks inferior to the GT due the very severe nature of the degradation.
Input
Restore VAR
GT
SID
Figure 25: Restore VAR does not perform well when the input degradation is very severe.
K LLM USAGE
LLM was used only for polishing writing in parts of the main paper and supplementary.
25

---

Published as a conference paper at ICLR 2026
LHP
REVIDE
TOLED
POLED
CDD + R)
Imput
AWRaCLe InstructIR PrompiIR
DCPT
GT Restore DFPIR
Figure 22: Additional qualitative comparisons of RestoreVAR with non non-genene son-generative methods on real-world, unseen and mixed degradations. ReS achieves better results, highlighting its superior generalization.
Input
PromptIR
InstructIR
AWRaCLe
DCPT
DFPIR
Restore VAR
LOLBlur Bl
LOLBIur
Figure 23: Qualitative comparisons with non-generative methods on samples from the real mixed- degradation dataset LOLBlur.
26

---

Published as a conference paper at ICLR 2026
Input
La (from LDM)
La = 0
GT
Figure 24: Illustration of the behavior of AutoDIR's (Jing et al. 2023) structure correction module (SCM). The second column shows outputs when the SCM is applied to the LDM output Isa, while the third column shows results when Isa is set zero. Despite no structural information (third column), the SCM still restores the image, indicating that it functions as a separate non-generative restoration model.
27


Reviews:
[{"review_id": "IEJFpqLnq0", "reviewer": "Reviewer_EKiz", "summary": "The paper introduces RestoreVAR, a method for image restoration based on Vision Autoregressive Modelling (VAR) framework. Compared to diffusion-based restoration methods, VAR-based image restoration offers rapid inference speed. The paper details the architecture modifications such as cross attention over the latents of degraded image, an additional latent refinement module for predicting the residual after quantization, and a fine-tuning procedure for the VAE decoder. Combining these changes, the paper shows RestoreVAR can obtain superior performance while while achieving over 10x faster inference.", "questions": "1. Could the author include some discussions on methods that speeds up inference for diffusion and how it compares with RestoreVAR? For example, consistency models and rectified flow.\n\n2. In section 4.4, the paper provides an ablation on adversarial and pixel level loss. How about SSIM and perceptual loss?\n\n3. In table 4, the paper reports a significant degradation in performance when the refinement network is trained without the last block output. Could you provide an explanation? Do you see degradation if removing the quantized prediction instead?\n\n4. Have the authors tried inference tricks that boost performance for generation, such as classifier-free guidance? It seems like the degraded image can be treated as a conditioning signal?\n\n5. The model seems to be reliant on conditioning signal after the fine-tuning. Does the model still retain any ability to generation image at all? What does $g_i$ (line 261) look like after the training?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The paper provides a nice visualization on on what is encoded in each scale and finds that VAR captures degradations predominantly in coarse scales and scene-level details in fine scales.\n\nThe paper argues that generative model offers strong generalization and convincingly demonstrates it with performance on real-world degradation as well as human preference.\n\nOverall, it's a well executed and presented paper.", "weaknesses": "Fine-tuning VAE decoder and adding a Latent Refiner Transformer (LRT) are critical to the final performance of the model. In the meantime, these techniques seem to be transferrable to other methods. So it's unclear whether we can attribute the success solely to VAR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ttYRBv0zu0", "reviewer": "Reviewer_NtbG", "summary": "The paper extends VAR (Tian et al., 2024) for All-in-one Image Restoration where a single model is trained to handle multiple image degradation types. By adopting VAR as the backbone, the proposed method claims to be much faster at inference time when compared to latent diffusion model-based methods. The method incorporates cross attention with the degraded input latents to reduce the risk of hallucination. It also includes a latent refinement transformer (LRT) to further improve the final output quality. The paper describes a series of experiments to benchmark against existing generative and non-generative methods. Further ablations are presented to justify the design choices such as the LRT and use of continuous latents for conditioning and decoding.\n\n**References:**  \nTian, K., Jiang, Y., Yuan, Z., Peng, B., & Wang, L. (2024). Visual autoregressive modeling: Scalable image generation via next-scale prediction. *Advances in neural information processing systems*, *37*, 84839-84865.", "questions": "* Compare computational complexity of proposed method against non-generative baselines.  \n* For the datasets in Table 2 that have clean/GT images (e.g. REVIDE, POLED, TOLED), is it possible to report PSNR, SSIM, LPIPS?  \n* How do the different scales contribute to the overall restoration? An ablation study in the style of section 3.2 with {restored, restored+coarse\\_deg, restored+fine\\_deg, deg} would be helpful to validate that restoring the coarse scales contribute most to overall the restoration performance.  \n* \\[Off tangent\\]What are the relative contributions of the VAR backbone and LRT to the overall restoration. Could an LRT \\+ VAE decoder acting directly on the output of the VAE encoder outputs (continuous or quantized) produce reasonable results? (i.e. $\\\\hat{f}\\_{cont} \\= f^{deg}\\_{cont} \\+ LRM(f^{deg}\\_{cont}, 0)$})  \n* Can the authors clarify the results presented in Table 1\\. Is RestoreVar trained specifically for each of the tasks? If so, is the comparison with Table 1 actually fair, as noted in line 374, non task-specific restoration models are not compared. If not, are the compared models trained with the same datasets (RESIDE, Snow100k, Rain13k, LOLv1, GoPro)?  \n* What is the performance for other generative methods (Diff-plugin, Auto-DIR, PixWizard) on real-world unseen degradations (Table 2)? Why were they excluded?  \n* For Table 2, RestoreVar is trained over what datasets?  \n* What about other VAR restoration methods? VarSR and VarFormer – their exclusion from the experiments makes it hard for this reviewer to gauge the contribution of this proposed method.  \n* The key claim of the paper’s approach in line 214 is that the degradations are separable from the fine details at various scales. Fig. 2’s illustration are images with synthetic degradation added → how far is this claim valid for real world degradations where the degradations from sensor and environment are mixed in a more complicated manner. Can the authors show the same results for the data used in Table 2?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "* The paper extends VAR for image restoration tasks, and discusses how restoration tasks can be framed within this ‘next-scale’ paradigm through the scale space analysis (Sec 3.2).  \n* The proposed method performs better than other generative methods in terms of PSNR, SSIM and LPIPS (Table 1\\) and is also significantly faster than these prior generative methods.  \n* Experimental comparisons across numerous state of the art generative and non generative methods over standard datasets.", "weaknesses": "* In terms of PSNR, SSIM and LPIPS, the proposed method is poorer than existing non-generative baselines. The authors suggest that the generative nature of the proposed method provides better generalisation capabilities and attempt to demonstrate this with comparison on other datasets using referenceless IQA metrics. Given the differences in the datasets, methods benchmarked against (generative and non-generative in Table 1 vs only non-generative in Table 2), and metrics, it is hard to place where this proposed method fits into the wider literature. The paper discusses three different dimensions: i) Multi-task restoration performance, ii) generalizability to unseen degradation types, and iii) computational complexity. A concise discussion of the proposed method in comparison to prior generative and non-generative methods along these three dimensions could be useful.  \n* The paper claims its strength for generalizability from the priors it has learnt from the VAE (line 84). It claims too that it is the first method to train a VAR directly for AiOR (line 147). Can the authors justify this claim when they have to finetune the VAE decoder and add in a latent refiner transformer? How is this different from HART (which is not compared).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O0OWGNc98E", "reviewer": "Reviewer_DSe5", "summary": "This paper introduces RestoreVAR, an all-in-one image restoration (AiOR) method based on visual autoregression (VAR). The authors improve VAR, originally designed for image generation, to enable it to handle multiple degradation types (dehazing, rain removal, snow removal, deblurring, and low-light enhancement) in a single model. Key technical contributions include: (1) a cross-attention mechanism to inject semantic information from degraded images; (2) a lightweight latent refinement transformer (LRT) to preserve fine details; and (3) a VAE decoder fine-tuned on a continuous latent vector rather than discrete labels. RestoreVAR achieves state-of-the-art performance among LDM-based generative methods, with inference speed 10 times faster than diffusion models, but significantly underperforms non-generative methods on some metrics.", "questions": "1.According to the ablation study, continuous latent fine-tuning improves performance by around 20%. Could the authors elaborate on why the original discrete VAR struggles so significantly in restoration tasks?\n\n2.How does the method perform on real-world mixtures of degradations that do not strictly fall into the five predefined categories?\n\n3.What is the impact of varying the value of K (number of scales) on the trade-off between speed and restoration quality?\n\n4.Can the authors provide a rigorous perceptual quality evaluation to justify that the gap on certain quantitative metrics between RestoreVAR and non-generative methods is acceptable in practice?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.RestoreVAR is the first work to apply Visual Autoregressive Modeling (VAR) to image restoration tasks in a generative setting, demonstrating the potential of autoregressive models beyond traditional generation tasks.\n\n2.Across five degradation types, RestoreVAR achieves state-of-the-art performance among current LDM-based generative approaches, while also providing a 10× faster inference speed.\n\n3.The ablation studies are well-designed and verify the effectiveness of each component in the proposed framework.", "weaknesses": "1.The paper compares RestoreVAR with both non-generative AiOR models and LDM-based generative methods. However, on many standard metrics, non-generative methods still outperform RestoreVAR. Although the paper acknowledges this and attributes the limitation to the VAE, it could further clarify when and why a generative approach is preferable in practice and quantify the trade-offs.\n\n2.The fine-tuning of the VAE decoder on continuous latent variables seems like a heuristic workaround, but the paper does not sufficiently justify the correctness or theoretical soundness of this approach.\n\n3.The user study reports average scores from 36 participants but does not provide confidence intervals or statistical tests. The evaluation lacks statistical significance analysis, confidence reporting, and detailed methodology, which are essential to support claims of perceptual quality superiority over non-generative methods.\n\n4.The paper briefly mentions limitations but does not provide detailed failure case analysis to illustrate when and why the method may fail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K3oPhhjgR6", "reviewer": "Reviewer_HSQV", "summary": "This paper proposes RestoreVAR, an All-in-One Image Restoration (AiOR) model that is built on a pre-trained Visual Auto-Regressive (VAR) model. This method leverages the multi-scale prior knowledge in the VAR model and is thus well-suited for the AiOR task. To achieve this, the authors introduce a cross-attention mechanism for conditioning the pre-trained VAR model on the degradation images. Finally, to mitigate the information loss due to the vector quantization operations, a transformer-based post-processing model is introduced to refine the generated tokens. Similarly, the VAE decoder is fine-tuned to further improve the reconstruction quality. Together, RestoreVAR achieves state-of-the-art performances as the diffusion-based generative-IR models while being 10 times faster at inference.", "questions": "- To help us better understand Weakness 1, could authors provide a set of ablation studies on the performance gains on the predicted higher resolution index maps? Specifically, a quantitative comparison between images with high-res index maps generated from ground-truth low-res index maps and from scratch generation. In other words, comparisons between [GT-low-res, Gen-high-res (from GT-low-res)] and [GT-low-res, Gen-high-res (from scratch)]? Additionally you could also provide another set of [Gen-low-res, Gen-high-res (from GT-low-res)] and [Gen-low-res, Gen-high-res (from scratch)], to better demonstrate the difference.\n- Given the limitations in Weakness 1 and 2, could authors discuss more on the evidence that how to justify RestoreVAR to be claimed as state-of-the-art, instead of another point on the perception-distortion plane?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The method’s design is intuitive and reasonable. The proposed method is tailored for the VAR, utilizing its efficient architecture while offering good sample quality. \n- The motivation is well presented and analyzed. In Section 3.1 and 3.2, the authors presented their observations, where the multi-scale structure of the VAR model effectively simplifies the AiOR task to the generation of low resolution features.\n- The manuscript is overall well-structured and easy reading.", "weaknesses": "- As mentioned in Section 3, the generative-IR methods are known to suffer from hallucinations that introduce artifacts in the results and affect metrics like PSNR and SSIM. Meanwhile, the generative nature of these methods offer improved image quality than their non-generative counterparts. This balance is also known as perception-distortion tradeoff. While RestoreVAR demonstrates advantages in PSNR and SSIM over other generative methods, they are not quantitatively compared on the image quality metrics like MUSIQ and CLIP-IQA. Specifically, Table 3 only presents a comparison in such metrics between RestoreVAR, a generative method, and other non-generative methods. I believe more discussions on the perception-distortion tradeoff will better demonstrate the advantages of RestoreVAR over other generative-IR methods.\n- Although the efficiency advantage of RestoreVAR is well established, the advantage in terms of generation quality of a VAR backbone is unclear. I believe Table 4 shall also include image quality metrics. Without LRT, the performance of RestoreVAR on PSNR and SSIM metrics appears to be the same level as other generative-IR models (21.71/0.690 v.s. 21.28/0.738). If LRT compromised image quality, the improvements in generation quality of RestoreVAR will be limited, especially considering other generative-IR methods were not evaluated on a fine-tuned VAE decoder.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)