InstaRevive: One-Step Image Enhancement via Dynamic Score Matching
Authors: Yixuan Zhu
Haolin Wang
Ao Li
Wenliang Zhao
Yansong Tang
Jingxuan Niu
Lei Chen
Jie Zhou
Jiwen Lu
Venue: ICLR 2025 Poster (2025)
Keywords: Image Enhancement
Diffusion model
Score Distillation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Image enhancement finds wide-ranging applications in real-world scenarios due to complex environments and the inherent limitations of imaging devices. Recent diffusion-based methods yield promising outcomes but necessitate prolonged and computationally intensive iterative sampling. In response, we propose InstaRevive, a straightforward yet powerful image enhancement framework that employs score-based diffusion distillation to harness potent generative capability and minimize the sampling steps. To fully exploit the potential of the pre-trained diffusion model, we devise a practical and effective diffusion distillation pipeline using dynamic noise control to address inaccuracies in updating direction during score matching. Our noise control strategy enables a dynamic diffusing scope, facilitating precise learning of denoising trajectories within the diffusion model and ensuring accurate distribution matching gradients during training. Additionally, to enrich guidance for the generative power, we incorporate textual prompts via image captioning as auxiliary conditions, fostering further exploration of the diffusion model. Extensive experiments substantiate the efficacy of our framework across a diverse array of challenging tasks and datasets, unveiling the compelling efficacy and efficiency of InstaRevive in delivering high-quality and visually appealing results.

Published as a conference paper at ICLR 2025
INSTAREVIVE: ONE-STEP IMAGE ENHANCEMENT VIA DYNAMIC SCORE MATCHING
Yixuan Zhu'; Haolin Wang': Ao Li", Wenliang Zhao', Jingxuan Niu?, Yansong Tang", Lei Chen Jie Zhou', Jiwen Lu!
'Department of Automation, Tsinghua University Tiinguua Shenzhen International Graduate School, Tsinghua University
ABSTRACT
Image enhancement finds wide-ranging applications in real-world scenarios due to complex environments and the inherent limitations of imaging devices. Recent diffusion-bant methods yield promising outcomes but necessitate prolonged and computationally intensive iterative sampling. In response, we propose InstaRevive, straightforward yet powerful image enhancement framework that employs score- based diffusion distillation to harness potent generative capability and minimize the sampling steps. To fully exploit the potential of the pre- pre-trained diffusion model, we devise a practical and effective diffusion distillation pipeline using dynamic control address inaccuracies in updating direction during score matching. Our control strategy enables a dynamic diffusing scope, facilitating precise learning of denoising trajectories within the diffusion model and ensuring accurate distri- bution matching gradients during training. Additionally, to enrich guidance for the generative power, we incorporate textual prompts via image captioning as auxiliary conditions, fostering further exploration of the diffusion model. Extensive experiments substantiate the efficacy of our framework across a diverse array of challenging tasks and datasets, unveiling the compelling efficacy and efficiency of InstaRevive in delivering high-quality and visually appealing results. Code available at htttt:: / / hub. cww./cco///ieeeeee Et wtt.//eeernaaa....... wwweivaannnnnnttta InstaRevive
INTRODUCTION
Image enhancement seeks improve the visual quality since images captured in wild scenarios always suffer from various degradations, like noise, blur, downsampling and compression due to the limitations of current imaging devices and complex environments. In recent years, great development
has been achieved in the image enhancement field using deep learning methods (Dong et al. 2014
Zhang et al. 2017 Liang al., 2021 Chen et al., 202 Wang al. 2022b Zamir et al. 2022
Chen et al. 2023b). While these methods yield commendable outcomes under specific, well-definel
degradations. they often fall short when faced with the complex conditions of real-world scenarios. Thus, our basic goal is to construct an effective and robust image enhancement framework capable of addressing various degradation conditions. This framework aims to deliver high-quality, visually appealing results within limited computational budget, making it more practical for real-world use.
Image enhancement ill- ill-posed due to unknown degradation processes, allowing for various possible high-quality (HQ) results from low-quality (LQ) inputs. To address this problem, researchers have explored a wide range of deep learning methods, which can be generally categorized into three
classes: predictive (Huang et al. 2020 Gu et al.) 2019; Zhang et al. (018a),, GAN-based (Wang ctt al. 2021al Yuaa et al. 2018 Fritsche et al.) 2019 Zhang et al. 2021] Wang et al. 2021c) and diffuston- based (Kawar al. [022 Wang al. 2022a Fei al. 2023] Lin et al. 2023b] Yue et al.
2023). Predictive methods use convolutional networks to estimate blurring kernels and recover images but struggle with real-world conditions. To better handle real-world challenges, some approaches leverage Generative Adversarial Networks (GANs) (Goodfellow et al. 2004) to jointly learn the data distribution of the images and various degradation types. GAN-based methods significantly enhance
" Equual contribution. Corresponding author.

---

Published as a conference paper at ICLR 2025
"A blue loke surrounded mountains and forest"
woman with brown and blue jockett
"A fred hair) and Eolossesi"
InstaRevive
InstaRevive
(n) Blind image super-resolution
(b) Blind face restoration
(c) Face carloonization
Figure 1: Our InstaRevive showcases remarkable image enhancement capabilities across diverse tasks. Leveraging highly effective dynamic score matching with textual prompts, our framework adeptly harnesses the rich knowledge within the pre-trained diffusion model for (a) blind image super- resolution and (b) blind face restoration using only one single step. Furthermore, we demonstrate that InstaRevive seamlessly generalizes additional related tasks such as (c) face cartoonization.
image quality but require careful tuning of sensitive hyper- parameters during training. Recently, diffusion models (DMs) (Ho et al. 2020: Rombach et al. 2022) have shown impressive visual generation capacity for image synthesis tasks. Some methods (Kawar et al. 2022] Wang al.) 2022a Lin et al.] 2023bb Yue et al. 2023) adopt the pre- vre-trained diffusion models and restore the images during the denoising sampling. Despite their improvements in visual quality, these methods require lengthy and computationally intensive iterative inference.
To address the aforementioned challenges, we propose a novel diffusion distillation framework tailored for image enhancement. To maintain the image quality of diffusion models, we leverage the score-based distillation to align the HQ and generated data distributions. To accurately learn denoising trajectories, we introduce dynamic control strategy for the diffusion noise scope, enabling the computation of precise scores and pseudo-GTs. This method not only refines distribution matching gradients but also overcomes the deficiency of conventional score matching, depicted in Figure After the dynamic score matching, our framework establishes a one-step mapping from low-quality inputs high-quality results. This mapping is carefully optimized ensure that the output images closely resemble the real-world HQ data distribution while distinctly diverging from undesirable distributions characterized by visible artifacts. To further exploit the generative priors embedded within the pre-trained text-to-image diffusion model, we use image captioning to extract natural language prompts, incorporating these as conditioning inputs for our framework.
We evaluate our framework on two representative image enhancement tasks: 1) blind face restoration (BFR) and 2) blind image super-resolution (BSR). Experimental results underscore the proficiency of our InstaRevive across these tasks. For BFR, InstaRevive achieves 22.3259 PSNR and 19.78 FID on the synthetic CelebA-Test. Additionally. the real-world LFW-Test. For BSR, we attain 0.4501 and 0..722 MANIQA on the RealSet65 and RealSR datasets, demonstrating both high enhancement quality and efficiency in single step. To further explore the potential of InstaRevive, we extend the applications of our framework to face cartoonization. InstaRevive consistently delivers visually appealing and plausible enhancements, underscoring the versatility and effectiveness of our framework, illustrated in Figure I
sets new benchmarks with a FID 38.73 on
RELATED WORKS
Image enhancement. Image enhancement involves tasks like denoising, deraining, and super- resolution, etc. Conventional works (Dong et al. 2014 Huang et al. 2220 Zhang al.) 2018a Dong al. 2015) utilize predictive models to estimate blur kernels and restore HQ images. With the rise of vision transformers (Dosovisskyy et al. 2020 Liu et al. 202I), some methods (Chen et al. 2021) incorporate the attention mechanism into basic architectures, yielding high-quality results. However, these models struggle with two main approaches in image enhancement, achieving significant success in complex blind image
degradations. The advent of generative models has introduced
2

---

Published as a conference paper at ICLR 2025
groundzuth (HO))
accuratc &
X
high noise
dynamic control
noise
inakcunte S
31 Xya
x
gemeraleed 4
4
qe
perturb
9:
perturb
closer
1Q Pa
Pt
Pt
occunilc estimution
R
(6) Existing Score-basod Disiillation
Dynamic Marching
Figure 2: Comparison with existing score-based matching. (a) Existing score-based distillation uses a broad range of perturbations, causing large noise to shift the generated result far from the GT. This results in inaccurate score estimations (depicted by low-quality pseudo-GT 2b)) and impedes the distillation. (b) Our dynamic score matching adjusts dimm to control the perturbation scale, ensuring more accurate score estimations and aligning distributions more closely.
restoration tasks. One approach is GAN-based methods (Wang al. 2021a Yuan et al. 2018 Fritsche et al.] Zhang et al. 2021] Wang et al.] 2021c bJ Yang et al. 2021 Zhou et al. 2022). which process images in the latent space to handle tasks like BSR and BFR. However, GAN -based methods require meticulous hyper-parameter tuning and they are often tailored to specific tasks, limiting their versatility. The other approach involves diffusion models (Ho et al. 2020 Rombach
et al. 2022), known for their impressive image generation capabilities. Methods like ( (Wang et al.
2023c Yue & Loy| 2022; Yue et al.) 2023) Kawar
2022 Fei et al.] 2023] Lin al.) [203b]] Yu
et 2024) design specific denoising structures to transfer image generation framework to image enhancement task. However, the sampling of diffusion models
consuminn To address this, some methods like (Xie et al., 2024 Wang al.) 2024 Zhu et al. 2024 Wu et al. (0224) employ distillation frameworks to process images in less steps. Nevertheless, distillation results often suffer from over-smoothing and reduced diversity, especially when facing complex degeneration.
Diffusion distillation. Diffusion distillation has gained attention for accelerating inference. Tradi- tional diffusion acceleration methods such (Zhao et al. 2023 Lu et al. 2022a (b reduce sampling steps from 1000 to around 50 by solving ordinary differential equations (ODHs), significantly cutting inference time. Researchers have since introduced distillation techniques reduce it just few even one step. Methods like (Huang et al. 2023 Luhman & Luhman 2021 use regression loss in pixel space for knowledge distillation. However, directly distilling the student model challenging due to the complexity of predicting noise at each time step and the extensive training required on large-scale datasets. Progressive distillation (Meng al. 2023] Salimans & Ho 2022) address this by iteratively reducing inference steps. Inspired by these approaches, some methods (Liu et al.) 20233
2022, Luo et al. 2023; Song al. 2023) aim to find a straighter inference path that shortens the sampling time to less than 5 steps. For one-step inference, (Yin et al.) 2023) Nguyen & Tran (024) propose one-step frameworks using score-based distillation and produce commendable image quality. However, directly applying the score-based methods to image enhancement can result in inaccuracies and over-smoothing. To overcome this, our InstaRevive introduces dynamic control, which precisely leams the denoising trajectories and significantly improves enhancement results.
3 METHODOLOGY
In this section, present the core concepts and detailed design of our InstaRevive. We will start with a brief introduction to diffusion models and score-based distillation. Subsequently, we will elucidate the motivation behind and the methodology of our dynamic score matching approach. Following this, we discuss the additional guidance with natural language prompts and describe the implementation of the framework. The overall pipeline of InstaRevive is depicted in Figure
3.1 PRELIMINARIES
Diffusion models. Diffusion models are a family of generative models that can reconstruct the distribution of data by learning the reverse process of diffusion process. In this process, noise
3

---

Published as a conference paper at ICLR 2025
Backpropagation
prompty
restorer
captioner
"A iubly out wilh stripedd coul"
Eyeg
a
Edsm
Trees or (1ed(((ng:))
Generater Go
z~N(
X.,, of1),
Tmax
Flog
Floga(x,y))
Trainable
d(xng. x)
Dynamic
Dynamic
perturb
loss
contro]
XyQ
The overall framework of InstaRevive. InstaRevive utilizes a score-based diffusion distillation framework for image enhancement. During training. we employ two score estimators to calculate the gradients of the KL divergence. To improve estimation accuracy, we devise dynamic control strategy to regulate the diffusing scope and adjust loss function weights. During inference, our generator can yield high-quality and visually appealing results in single step.
Figure 3:
incrementally added to data, while the reverse process predicts the distribution of progressively denoised data points, Given random noise ~ NO, I). the diffusion process is defined as:
3t = 0, + 03e,
where æo and æ; are the clean and noisy image, and {(a,, ot) | is the noise schedule. Conversely, the reverse process starts from pure noise #T ~ (O, I), and denoising model é iteratively predicts the noise, transitioning from #++1 to Ht Specifically, text-to-image diffusion models, the text prompt y serves as guidance during each denoising step. The training objective is to minimize:
CDM Ex,,e.t ||c Ca y)
After training, the diffusion model learns the gradient of data density V æ, log (e(2xly) via the noise prediction network Ev. We denote this gradient as the score function of pa and can be approximated by Vz, log Pe (az, v) (x t, y)/ot-
Score-based distillation. The score- based distillation (Poole et al. 2022 Wang et al. 2023a
al. 2023a) was firstly introduced in text-to-3D generation. Given 3D representation 8 (e.g. NeRF) and a rendering function g(', c), the rendered image can be obtained by zg g(8, c) with a specific camera position The objective optimize this representation 0 such that its 2D-rendered images align with the pre-trained diffusion model's 's outputs given a text prompt. However, directly addressing this optimization problem challenging due to the complexity of the diffusion model's distribution, often resulting in issues such as over-saturation or over-smoothing. To tackle this problem, (Wang 2023d) proposes Variational Score Distillation (VSD) by introducing an additional score function estimated by another model es (æt, y, c), which can be finetuned with LoRA (Hu et al. The final optimization
(02T)) during optimization with diffusion loss as described in Equation 2 objective of VSD is derived as:
min Er,c
(æ.le, ()ll)()))))))
(3)
where LT(minn Tmax), w(t) is a weighting function, yf a view-specifie prompt, and u(0ly) is the probabilistic density of given the prompt.
3.2 DYNAMIC SCORE MATCHING FOR IMAGE ENHANCEMENT
Simplify the objective of image enhancement. The primary objective in optimizing image enhance- ment tasks typically involves regression term that penalizes the distance between the generated result and the ground truth. However, relying solely on the regression term Ereg to train the generator Go often yields unsatisfactory results. This is due the complex and often irreversible degradation
4

---

Published as a conference paper at ICLR 2025
process LQQ ((zHQ) encountered in real-world scenarios, which significantly impairs image information. Under such challenging degradation conditions, it is difficult to construct a one-to-one mapping between HQ and LQ images with only simple regression approach. To ease this challenge. we relax the optimization with another term that penalizes the distance between the distributions of generated results and HQ images. Inspired by the score-based distillation, formulated this term as the KL divergence, and the optimization objective is revised as follows:
min Exra (Ga((1Q)) - THQ + ADKL (pollgo).
where X' represents the set of LQ images, Po and go denote the distribution of HQ images and generated results, and À is the weight factor. By minimizing these two terms concurrently, we alleviate the learning difficulty and improve result diversity. The generated result is not strictly forced to match æHQ, but rather to fall within the HQ distribution, allowing for various plausible results that reduce penalties. This additional term also acts as a regularization, preventing overfitting and bolstering generator robustness. Unlike GANs, which train a discriminator and only provide the probability of realness, our approach aims to expand the gradient of this term as score tensors that explicitly point out the updating direction. Given the robust gradient estimation and extensive knowledge of HQ images inherent in diffusion models, we leverage these models to compute the gradient of these two distributions and optimize our generator through diffusion distillation.
Score matching distillation framework. To solve the optimization problem in Equation 4 we aim to diminish the KL divergence term (Kt.(golleg))) Given the prompts y. we condition the probability distributions on y. However, Po(alyy) vanishes when æ is far away from HQ images, which makes the training difficult to converge effectively. In response, Score-SDE (Song & Ermon 2019 [ong et al. 2020) introduces& a method that diffuses the original distributions with varying scales of noise indexed by l. and optimizes series of KI. divergences between these diffused distributions, DRL. (p: (qc), the gradient of which is:
aG, (ZLQ y)
VoDKL= = Bt,e
(t) (V., logpe(<<(l)) Vae, log Or (2x (b))
80
(5)
aGo y)
= BE,,,
(ew(t) (Ew( y) cg( 4))
00
where 2t is the noisy image obtained by Equation T and de are the diffused distributions, and w(t) is the weight related to timestep f. (For detailed derivations, please refer A.8.I) The scores of the diffused distributions can be estimated using two diffusion models, Ey. and Eg. as mentioned earlier. With increasing & from to T, the overlap between the diffused distributions grows, making the scores well-defined and facilitating the calculation via the diffusion models. During training, we backpropagate the gradient in Equation to update the generator. Simultaneously, we update the diffusion model 6s to align with the generated distribution using the diffusion loss in Equation 2
Dynamic noise control for effective matching. The accuracy of Equation depridd on the precise estimation of gradients by the two score functions, Va, log (xxly) and Va, log 4: (x;; (y). In an ideal scenario, the diffusion models accurately compute these scores. However, during actual training, we observe that the estimates from the diffusion models might deviate, leading to potential inaccuracies, especially as nears Tmax- As shown in Figure 2 this discrepancy can yield incorrect gradients for the KL divergence, impeding effective parameter updates and hindering training progress. To overcome this challenge, we propose the dynamic noise control strategy. Unlike the typical application scenarios like text-to-image generation, where the generated image distribution diverges significantly from the target distribution initially, in image enhancement, the generated distribution closely approximates the high-quality distribution from the outset. Hence, we employ relatively subtle Gaussian noise perturbations to the data distributions, ensuring the rationality of the two score functions. Furthermore, this controlled noise diminishes the gap between a; and EGT, contributing to accurate score estimates. Specifically, we regulate the added Gaussian noise level as follows:
Trax =o- o d daHQq x)], 1 ~~ (0.022 max; Tmax]
(6)
d(zHQ. x)) V (3) 2 1l/BB
where o -1 is the inverse function of or, R is the control factor and B is the batch size. By adjusting T Thax, we assure that Var(a,, .x) S Var(aT x) of max I, controlling the distance between
5

---

Published as a conference paper at ICLR 2025
Dy and THQ via the triangle inequality (elaborated in Equation 15 score computation at zt. Additionally, our score matching strategy Is dynamic, allowing the generator to focus on varying noise levels during different training stages. We denote the score matching loss governed by this dynamic scheme as Ludsm- Specifically, Equation results in different T max with respect the current discrepancy between HQ and generated distribution. As training advances and these two distribution data become more aligned, smaller I mmax guides the generator to concentrate mainly on refining detailed artifacts in generated images.
which ensures more accurate
Dynamic loss control for stable training. Next, we compare the regression loss and dynamic score matching loss. Regression loss, commonly used in image restoration, penalizes pixel-level errors but may miss perceptual quality. In contrast, dynamic score matching loss evaluates distribution similarity between real and generated data, enhancing realism and texture, but lacking pixel-level guidance. We employ two diffusion models E and cg. to estimate the gradients of the real and generated data distribution, respectively. Both are initialized from pre-trained image generation diffusion models. While E. accurately estimates the gradient of the real data distribution, the gradient estimated by es significantly less accurate. Consequently, Equation fails to provide proper optimization directions for minimizing the KL divergence, potentially hindering generator training even introducing negative effects. To balance these loss functions, we propose a dynamic control strategy. Early in training, the regression loss drives effective optimization, as dynamic score matching is biased. As the generator improves, regression loss becomes less informative, and score matching focuses on refining finer details. In deployment, we use control the ratio between the two loss functions, which is obtained by linearly mapping max to the interval [o, 1].
[
3.3 GUIDANCE WITH NATURAL LANGUAGE PROMPTS
Leveraging the pre-trained diffusion model's knowledge of the relationship between textual and visual information, we introduce additional textual guidance to further improve image processing capabilities. Unlike implicit visual features from LQ images, textual prompts provide explicit and detailed descriptions, significantly aiding in our image enhancement. This is especially valuable for severely degraded images, where visual perception alone may be ambiguous. To enable the captioner to handle extremely low-quality images, we employ pre-trained image restorer for coarse restoration. Compared the model backbone, the restorer holds much fewer parameters and exerts minimal impact on inference speed. By incorporating proper textual prompts, can set clear goals for the enhancement result and generate reasonable outcomes from chaotic LQ images. This approach also allows for creative and editable results, offering high controllability through human guidance, enhancing restoration performance and extending the framework's ' flexibility.
3.4 IMPLEMENTATION
We maintain consistent framework structure across all tasks, employing the pre-trained diffusion model to initialize the student and teacher models. This approach exploit the rich pre-trained knowledge of image understanding and vision-language relationships acquired during the generation task. Specifically, for the generator Gg and two score estimators, 6y and cg, we initialize them with identical weights from the pre-trained text-to-image diffusion model. To estimate the generated distribution, we unfreeze the parameters of the score estimator e... For the restorer, we employ the lightweight module in (Liang et al. (02T) following (Lin al. 2023b) perform coarse restoration. To gather textual prompts, we utilize the multi-moda BLIP
(Li et al. 2022a) as the image captioner, extracting semantic contents in the images. To enhance the diversity, we concatenate æLQ with random noise and use a convolution layer to align the channel dimension. To approximate the degradation conditions in BFR and BSR, we produce the synthetic data from HQ image by TLQ [[k æHQ) Ir +2l which consists of blur, noise, resize and JPEG compression.
EXPERIMENTS
4.1 EXPERIMENT SETUPS
Datasets. For blind face restoration (BFR), we utilize the Flickr-Faces-HO (FFHQ) (Karras et al.) 2019), which encompasses 70,000 high-resolution images. We resize them 512 512 to match

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative comparisons for BFR on the synthetic and real-world datasets. We highlight the best and second best performance in bold and underline, respectively. We categorize the methods into conventional (up), diffusion- based (middle) and distilled (bottom). Our InstaRevive shows very competitive results compared with existing methods. We obtain remarkable image quality and identity consistency with the leading FID and PSNR scores. Our framework also exhibits much faster inference than the diffusion-basee method.
Synthetic Dataset CelebA
Wild Datasets
Method
LFW
WIDER
FPS1
PSNRT
SSIMT
LPIPS
FIDL
IDS
HID.
FIDL
GPEN Yang et al. 2021
21.3995
0.5742
0.4687
23.92
0.48
51.97
46.35
7.278
GCFSK He al 2022
21.8791
0,6072
0.4577
35.49
0.44
52.20
40.86
9.243
GFPGAN Wang ef
b021b
21.69G3
0.6090
0.4304
21.09
0.49
52.11
41.70
8.152
VQFR (Gu et al. 2022
21.3014
0.6132
0.4116
20.30
0.48
49.888
37.87
3.837
RestoreF Tonner Wang cl al
2022c
21.0025
0.5283
0.4789
43.77
0.56
48.43
49.79
4,964
DMDNet Li
21.6617
0.6000
0.4828
64.79
0.67
43.36
40.51
3.454
CodeFoommrr Zhou
2022
22.1519
0.5948
0.4055
22.19
0.47
52.37
38.78
5.188
DifFace 100 Yoe & Loy 2022
22.1483
0.6057
0.4129
19.95
0.61
46.17
37.42
0.225
ResShift-4 Yue et 2025
21.6858
0.5829
0.4082
20.03
0.59
50.09
37.21
3.623
InstaRevive (Ours)
22.3259
0.6109
0.4025
19.78
0.65
38.73
35,29
7.967
the input scale of the diffusion model. For evaluation, we leverage the widely used CelebA-Test dataset (Liu et al. 2015), which consists of 3,000 synthetic HQ-LQ pairs. Additionally, to further validate the effectiveness on world data, we employ 2 wild face datasets, LFW-Test (Wang eet al. 2021b) and WIDER- Test (Zhou al. (0222 which contain face images with varying degrees of degradation. For blind image super-resolution, we train our framework using the large-scale ImageNet dataset (Deng et al. 2009) and evaluate on the RealSR (Cai at. | 2019) and RealSet65 YYue) et al. (003).. Specifically, RealSR contains 100 LQ images captured by two different cameras in diverse scenarios, while RealSet65 comprises 65 LQ images sourced from widely used datasets and the Internet. For face cartoonization, we create a stylized dataset using the iterative sampling of ControlNet (Zhang et al., 2023). This dataset includes 130,000 512 x 512 images based on FFHQ.
Training details. We unfreeze two models in our framework during training: the generator Gg and the score estimator e. for the generated distribution. Both models are optimized with a batch size of 32 and learning rate of le-6 using two AdamW optimizers with weight decay of 1e-2. We initialize the generator and two score estimators by replicating the denoising transformer blocks in (Chen et al. 2023a). For BFR and BSR, we employ the high-order degradation model in (Wang) ctt al.) 2021c), training for 25K and 35K steps with 4 Nvidia A800 GPUs, respectively. For face cartoonization, we finetune our pre-trained BFR model for an additional 10K steps. We set the KL term weight to 1.0 and the control factor to 1.5 for optimal performance.
To evaluate InstaRevive's performance BFR, calculate three traditional metrics, including PSNR, SSIM and LPIPS (Zhang et al., 2018b), on the CelebA-Test dataset. However, these metrics have their limitations in assessing visual quality as they often penalize high-frequence details in our generated images, e.g., hair texture. Therefore, we also include the widely-used FID (Heusel al. 2017) score to measure overall image quality. Additionally, we compute the identity similarity (IDS) using a pre-trained face perception network (Deng et al. 2019). For BSR, we leverage four non-reference metrics MANIQA (Yang et al., 2022). MUSIQ (Ke al. 2021), CLIPIQA (Wang et al. 2023b) and NIQE to assess image quality. To demonstrate the efficiency of InstaRevive, we also compare the throughput (FPS) of our framework with other methods.
Metrics.
4.2 MAIN RESULTS
Blind face restoration. Blind face restoration necessitates a convincing mapping from LQ to the desired face image with high-quality details. We evaluate InstaRevive on both synthetic CelebA-
Test (Liu et al. 2015), with in-the-wild LFW-Test (Wang et al. 2021b) and WIDER- (Zhou et al.
2022). Our comparative analysis involves recent state-of-the-an art methods, including GPEN Yang et al. 2021), GCFSR (He et al. (022), GFPGAN (Wang et al. 2021b). VQFR (Gu et al. 2022),
RestoreFormer WWang al. 2022c), DMDNet (Li al. 2022b). CodeFormer (Zhou al. 2022),
2023) (4 steps). As depicted in Table T our Instakevive achieves notable performance metrics, including 19.78 FID, 22.3258 PSNR
DifFace (Yue & Loy| 2022) (100 steps) and ResShift (Yue et al.
7

---

Published as a conference paper at ICLR 2025
Real Input
GPEN
RestereFormer
DMDNet
CodeFormer
DifFace- 100
ResShift-4
InstaRevive
Figure 4: Qualitative comparisons on the real-world faces. Our method demonstrates impressive enhancement capabilities real-world faces, producing high-fidelity and visually appealing faces. Compared to other methods, InstaRevive exhibits robustness when handling challenging cases, Table 2: Quantitative comparisons for BSR on real-world datasets. Our framework achieves high-quality enhancement and outperforms existing methods in various image quality metrics.
Type
Method
RealSen65
RealSR
MANIQAT
MUSIQU
NIQE!
CLIPIQAT
MANIQAT
MUSIQI
NIQEL
CLIPIQAI
RealSR JPEG di al 2020
0,2923
50.537
4. 4.8042
0.5280
0,1738
36,071
6.9528
0.3614
GAN
RealESRGAN
BSRGAN Zhakz & Mle 2021
0.3064
0.3876
65.578
5,5852
4,8909
0.6160
0,3705
63.584
7.7273
4,6606
0.2363
0.5434
SwinlR- (Liing oral
20211
0,2699
44.975
8.0458
0.4225
0.2727
43.219
7,7362
0,3637
StableSR Wing 2023c
0.3643
$9.670
4,8932
0.5633
0.3807
61.362
4.6778
0.5530
Diffusion
ResSihn-s
GDP Poi a
al
M124
0.3958
0.2757
55
61..30
6.8496
5.9425
0.6537
0,2865
0,3640
59.873
59.
7,0729
5,9820
0.6533
0.5958
LDM-15 RRmbbuch
2022
0,2760
47.600
6.2490
0,4203
0.2909
50.926
5,9172
0,3932
Distille
SinSR Wang al.
InstaRevive (Ours)
0.457I
65.849
4.1995
0.6755
0.4722
64.535
4.2781
0.6572
and 0.4025 LPIPS, outperforming other methods. We also obtain comparable SSIM and IDS and establish a higher upper bound of the diffusion-based methods. Furthermore, we achieve 38.73 73 FID on the LFW-Test, demonstrating high restoration quality for real-works images. We also attain competitive performance on WIDER-Test. Besides, our InstaRevive demonstrates rapid processing with 7.967 FPS, significantly outpacing the ceffussionbbsed methods. The qualitative results in Figure 4 further illustrate that our method effectively restores face images degraded by real-world conditions, consistently delivering visually appealing outcomes with single-step inference.
Blind image super-resolution. Blind image super-resolution involves general image prior and low-level structural knowledge. We evaluate our InstaRevive RealSR (Cai al. 2019) and Re-
alSet65 (Yue et al. 2023), comparing it with cutting-edge methods, including GAN-based RealESR- GAN (Wang et 2021c). BSRGAN (Zhang et al., 2021). SwinIR-GAr -AAN (Liang al. 2021). and diffusion-based GDP (Fei et al. 2023), ResShift Yue et al. 2023), LDM (Rombach et al.) 2022) and StableSR (Wang et al., 2023c). As shown in Table 2 our InstaRevive achieves outstanding image quality with the highest MANIQA, MUSIQ and NIQE on both datasets. Note that our one- step generator outperforms other methods that use iterative sampling by a significant margin on NIQE. Additionally, we attain competitive CLIPIQA scores compared with the most recent method employing diffusion distillation. We also provide qualitative comparison in Figure 5 with recent multi-step models like DifiBIR (Lin et al. 2023b). As illustrated, enhancements and plausible details, despite utilizing only single inference step. demonstrating an excellent balance between efficiency and performance. Further analysis of the model parameters and inference time is discussed in Section AA.4, Moreover, on the synthetic ImageNet- dataset, our method achieves impressive PSNR and SSIM metrics, as reported in Table in the appendix.
generator produces high-quality
4.3 ANALYSIS
Effective distillation of diffusion priors. The major challenge in adapting diffusion distillation to image enhancement is converting the generative capacity into restoration power within a single-step model. To demonstrate our framework enables effective diffusion distillation, we perform direct distillation using only regression loss to supervise the generator's 's outputs. Results in Table and Figure 6 indicate that this approach (w/o score) yields blurry outcomes with visible artifacts. In contrast, our score matching approach, by minimizing the KL divergence, updates the student model in "softer" and more effective way, thereby reducing the distance between the HQ image and the
8

---

Published as a conference paper at ICLR 2025
(b)
(c)
(C)
(4) Input
(g) RodStina
(b) ResShon-15
(6 DiORER-50
() SinRR-
(3) IndtReevvvsl
BSRGAN
RealESRGAN
(6) Swinll-GAN
(e) Rea15R-PPGG
LDM-100
Enpui
ResShif--4
ResSbitt-15
DiRISIR-50 -50
SinSR-1
(K) EnstaRevive-
Figure 5: Qualitative comparisons real-world datasets. Our InstaRevive delivers exceptional details with just one-step inference. The numbers following each method indicate the corresponding inference steps. More results and comparisons can be found in Figure. and Figure. TO
LQ Input
w/o score
w/o dynamic
wil prompi
InstaRevive
GT
Figure 6: Visual results of the ablations. Our dynamic score matching and prompt guidance significantly enhance both image quality and controllability in the generated outputs.
current generated results. Unlike the one-to-one mapping, the KL divergence term allows for range of plausible outcomes, imparting more detailed and comprehensive knowledge to the student model.
Dynamic control for accurate learning. To evaluate the dynamic noise and loss control strategy, conducted an ablation study by removing it while keeping other hyper-parameters fixed. The results (w/o dynamic), presented in Table and Figure 66 demonstrate that the dynamic control strategy greatly improves the quality of the generated images. Furthermore, the loss curves in Figure Jindicate that this approach accelerates training and shortens convergence time significantly. This highlights the importance of maintaining the noise level within an appropriate range during the diffusion process and dynamically adjusting the loss weights. This dynamic control strategy is particularly well-suited for enhancement tasks, where the distributions of the generated and real images initially overlap, necessitating a focus on noisy image not too far away from THQ thus reducing the training difficulty and enhancing the model performance.
Additional guidance with textual prompts. Textual prompts offer clear and explicit guidance for the enhancement target. To evaluate their significance, perform ablation study using null textual input for training and evaluation. As shown in Table and Figure [ the absence textual prompts (w/o prompt) results in drop in performance, especially in more challenging scenarios. This suggests that textual prompts are crucial for the teacher model Gs to generate the correct scores.
Limitations. Despite InstaRevive's 's promising results within one-step paradigm, it may produce suboptimal results when encountering extreme degradation or complex content, as shown in Figure [9] How utilize the generative power to address these challenges remains area for future exploration.

---

Published as a conference paper at ICLR 2025
Regression Loss
Table 3: Ablation studies. We perform ablations on BFR and BSR to validate the effectiveness In- staRevive's components. The results demonstrate that score matching leads to effective distillation and that dynamic control and textual prompts are beneficial for overall performance.
2.00
1.00
0.75
0.50
Method
FIDJ
MANIQAT
CelebA
LFW
RealSet65
RealSR
0.25
M/ Score loss control
We nose centrol
Ours
w/o score
26.45
$1.03
0.4085
0.4259
0.00
OK
15
20
25
30
w/o dynamic
45.39
0.4463
w/o prompt
19,90
39.38
0.4374
0.4541
Figure 7: Loss of the ablations. Our design promotes faster convergence.
InstaRevive (Ours)
19.78
38.73
0,4571
0,4722
Real Inpur
style
oil painting style
glasses"
"gnews clip
rarrings"
"ancMonn closed
"sult yelinw giawee"
"pink hot brand""
"Ied hoir sunglosses"
"long howkeemingg
Figure 8: Qualitative results on face cartoonization. InstaRevive delivers high-quality results (Col.2 and 5). Furthermore, we can edit the details with textual prompts (Col.3, 4, and 7), showcasing high controllability. More comparisons can be found in Figure [3]
4.4 EXTENSIONS
To showcase InstaRevive's versatility and controllability, extend its application the complex task of controllable face cartoonization. This task entails not only altering the global style of the images but also editing their detailed content based on the natural language prompts. After fine-tuning our framework on the stylized face dataset, InstaRevive learns to edit the style and the semantic regions from the pre-trained diffusion model and textual prompts. As illustrated in Figure & our generator produces high-qualin results with remarkable controllability. These results also affirm InstaRevive's 's's potential for broader applications in image editing and enhancement tasks. To ensure high identity consistency during image-to-image transitions, we utilize the IP-Adapter (Ye et al. (2023) as our teacher model. The details and results are presented Section. A.5] We further explore more tasks including low-light enhancement and face inpainting, with findings detailed in Section. A.3
CONCLUSION
We propose InstaRevive, a one-step image enhancement framework that excels in efficiency and effectiveness across various tasks. Empowered by the dynamic control strategy for score matching distillation and additional guidance with textual prompts, InstaRevive enables accurate computation of gradients for both HQ and generated data distributions, thereby significantly accelerating the training process while improving the result quality in only one-step inference. We also extend InstaRevive to face cartoonization, showcasing its strong generalization. We hope our attempt can inspire future work to further exploit diffusion priors for image enhancement.
Acknowledgment: This work was supported in part by the National Key Research and Development Program of China under Grant 2023 and in part by the National Natural Science Foundation of China under Grant Grant 62125603, Grant 62321005, Grant 62336004 and Grant
62306031.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. CVPR, pp. 18392-18402, 2023.
Jianrui Cai, Hui Zeng, Hongwei Yong. Zisheng Cao, and Lei Zhang. Toward real-world single image super-resolution: new benchmark and a new model. In ICCV, pp. 3086-3095, 2019.
Hanting Chen, Yunhe Wang. Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu. Chao Xu, and Wen Gao. Pre-trained image processing transformer. In CVPR, pp. 12299-12310, 2021.
Junsong Chen, YU Jincheng, GE Chongjian, Lewei Yao, Enze Xie, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2023a.
Xiangyu Chen, Xintao Wang, Jiantao Zhou, Yu Qiao, and Chao Dong. Activating more pixels in image super-resolution transformer. In CVPR, pp. 22367-22377, 2023b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR. pp. 248-255, Icee, 2009.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In CVPR, pp. 16690-4699 2019.
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Learning deep convolutional network for image super-resolution. In ECCV, pp. 199-199. Springer, 2014.
Chao Dong, Chen Change Loy. Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. TPAMI, 38(2):295-307, 2015.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.
Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. Generative diffusion prior for unified image restoration and enhancement. In CVPR, pp. 9935-9946, 2023.
Manuel Fritsche, Shuhang Gu, and Radu Timofte. Frequency separation for real-world super- resolution. In ICCVW, pp. 3599-3608. IEEE, 2019.
Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. Removing rain from single images via deep detail network. In CVPR, pp. 3855-3863, 2017.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 27, 2014.
Jinjin Gu. Hannan Lu, Wangmeng Zuo, and Chao Dong. Blind super-resolution with iterative kernel correction. In CVPR, pp. 1604-1613, 2019.
Yuchao Gu, Xintao Wang, Liangbin Xie, Chao Dong, Gen Li, Ying Shan, and Ming-Ming Cheng. Vgfr: Blind face restoration with vector-quantized dictionary and parallel decoder. In ECCV, pp. 126-143. Springer, 2022.
Jingwen He, Wu Shi, Kai Chen, Lean Fu, and Chao Dong. Gefsr: a generative and controllable face super resolution method without facial and gan priors. In CVPR, pp. 1889-1898, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. NewrIPS, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33: 6840-6851, 2020.
11

---

Published as a conference paper at ICLR 2025
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint 1eXiv:2106009685,, 2021.
Tao Huang, Yuan Zhang, Mingkai Zheng, Shan You, Fei Wang, Chen Qian, and Chang Xu. Knowl- edge diffusion for distillation. NeurIPS, 36, 2023.
Yan Huang, Shang Li, Liang Wang. Tieniu Tan, et Unfolding the alternating optimization for blind super resolution. NeurlPS, 33:5632-5644 -643,, 2020.
Xiaozhong Ji, Yun Cao, Ying Tai, Chengjie Wang, Jilin Li, and Feiyue Huang. Real-world super- resolution via kernel estimation and noise injection. In CVPRW, June 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. CVPR, pp. 4401-4410, 2019.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. NeurIPS, 35:23593-23606, 2022.
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. ICCV, pp. 5148-5157, 2021.
Gihyun Kwon and Jong Chul Ye. Clipstyler: Image style transfer with a single text condition. In CVPR, pp. 18062-18071, 2022.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-iman image pre- training for unified vision-language understanding and generation. In ICMI., pp. 1288-. PMLR, 2022a.
Xiaoming Li, Shiguang Zhang, Shangchen Zhou, Lei Zhang, and Wangmeng Zuo. Learning dual memory dictionaries for blind face restoration. TPAMI, 45(5):5904-5911 2022b.
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In ICCV, pp. 1833-1844, 2021.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung-Yi Yi Lin. Magic3d: High-resolution text-to-3d content creation. In CVPR, pp. 300-309, 2023a.
Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Ben Fei, Bo Dai, Wanli Ouyang, Yu Qiao, and Chao Dong. Diffbir: Towards blind image restoration with generative diffusion prior. arXiv preprint arXiv:2308. 2500:23000..0000, 2023b.
Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2022.
Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step enough for high-quality diffusion-basee based text-to-image generation. In 2023.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pp. 10012-10022, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV. pp. 3730-3738, 2015.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. NeurIPS, 35:5775-5787, 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint 22X1v:2211.0109) 2022b.
12

---

Published as a conference paper at ICLR 2025
Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.22388, 2021.
Calvin Luo. Understanding diffusion models: A unified perspective. arXiv preprint urXiv:2208.11977 2022.
Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference, arXiv preprint arXiv:3310.04378, 2023.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In ICLR, 2021.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In CVPR, pp. 14297-14306, 2023.
Thuan Hoang Nguyen and Anh Tran. Swiftbrush: One-step text-to-image diffusion model with variational score distillation. CVPR, 2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. ICLR, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pp. 10684-10695, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. NeurlPS, 32, 2019.
Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2020.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML. pp. 32211-32252. PMLR, 2023.
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Ych, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In CVPR, pp. 12619-12629, 2023a.
Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. AAAI, volume 37, pp. 2555-2563, 2023b.
Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image super-resolution. In arXiv preprint 0oXiv:2305.00015 2023c.
Longguang Wang, Yingqian Wang, Xiaoyu Dong, Qingyu Xu, Jungang Yang, Wei An, and Yulan Guo. Unsupervised degradation representation learning for blind super-resolution. In CVPR, pp- 10581-10590, 2021a.
Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. Towards real-workh blind face restoration with generative facial prior. In CVPR, pp. 9168-9178, 2021b.
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In ICCV. pp. 1905-1914, 2021c.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In ICLR, 2022a
Yufei Wang, Wenhan Yang. Xinyuan Chen, Yaohui Wang, Lanqing Guo, Lap-Pui Chau, Ziwei Liu, Yu Qiao, Alex C Kot, and Bihan Wen. Sinsr: Diffusion-basee -ased image resouuienolution in a single step. CVPR, 2024.
13

---

Published as a conference paper at ICLR 2025
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: general u-shaped transformer for image restoration. In CVPR, pp. 17683-17693, 2022b.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In NeurlPS, 2023d.
Zhouxia Wang, Jiawei Zhang, Runjian Chen, Wenping Wang, and Ping Luo. Restoreformer: High- quality blind face restoration from undegraded key-value pairs. In CVPR, pp. 17512-17521, 2022c.
Chen Wei, Wenjing Wang, Wenhan Yang. and Jiaying Liu. Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv: 180:804600 2018.
Rongyuan Wu, Lingchen Sun, Zhiyuan Ma. and Lei Zhang. One-step effective diffusion network for real-world image super-resolution. NeurIPS, 2024.
Rui Xie, Ying Tai, Kai Zhang, Zhenyu Zhang, Jun Zhou, and Jian Yang. Addsr: Accelerating diffusion- based blind super-resolution with adversarial diffusion distillation. arXiv preprint arXiv:2404.01717, 2024.
Serin Yang, Hyunmin Hwang, and Jong Chul Ye. Zero-shot contrastive loss for text-guided diffusion image style transfer. In ICCV, pp. 22873-22882, 2023.
Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi-dimension attention network for no-reference image quality assessment. In CVPR, pp. 1191-1200 2022.
Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang. Gan prior embedded network for blind face restoration in the wild. In CVPR, pp. 672-681, 2021.
Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. Deep joint rain detection and removal from a single image. In CVPR, pp. 1357-1366, 2017.
Wenhan Yang, Robby T Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, and Jiaying Liu. Joint rain detection and removal from a single image with contextualized deep networks. TPAMI, 42(6): 1377-1393, 2019.
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint 20X.1.2308.06721, 2023.
Tianwei Yin, Michaël Gharbi, Richard Zhang. Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. arXiv preprint arXiv:2311.1882 2023.
Fanghua Yu, Jinjin Gu, Zheyuan Li, Jinfan Hu, Xiangtao Kong, Xintao Wang, Jingwen He, Yu Qiao, and Chao Dong. Scaling up excellence: Practicing model scaling for photo-realistic image restoration in the wild. CVPR, 2024.
Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. 1d0k diverse driving dataset for heterogeneous multitask learning. In CVPR, pp. 2636-2645, 2020.
Yuan Yuan, Siyuan Liu, Jiawei Zhang, Yongbing Zhang, Chao Dong, and Liang Lin, Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. In CVPRW, pp. 701-710, 2018
Zongsheng Yue and Chen Change Loy. Difface: Blind face restoration with diffused error contraction. arXiv preprint 2rXiv2212..06112. 2022.
Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Resshift: Efficient diffusion model for image super-resolution by residual shifting. NeurlPS, 36, 2023.
14

---

Published as a conference paper at ICLR 2025
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, pp. 2022.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep enn for image denoising. TIP, 26(7):3142-3155, 2017.
Kai Zhang, Wangmeng Zuo, and Lei Zhang. Learning single convolutional super-resolution network for multiple degradations. CVPR, pp. 3262-33 1002-,27,222 201 Sa.
Kai Zhang, Jingyun Liang. Lue Van Gool, and Radu Timofte. Designing a practical degradation model for deep blind image super-resolution. In ICCV, pp. 4791-4800, 2021.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In ICCV, pp. 3836-3847, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, pp. 586-595, 201
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipe: A unified predictor- corrector framework for fast sampling of diffusion models. NeurlPS, 36, 2023.
Shangchen Zhou, Kelvin Chan, Chongyi Li, and Chen Change Loy. Towards robust blind face restoration with codebook lookup transformer. NeurlPS, 35:30599-30611, 2022.
Yixuan Zhu, Wenliang Zhao, Ao Li, Yansong Tang, Jie Zhou, and Jiwen Lu. Flowie: Efficient image enhancement via rectified flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13-22, 2024.
15

---

Published as a conference paper at ICLR 2025
Figure 9: More qualitative results on real-world images with larger resolution. InstaRevive excels enhancing real-world images with high resolutions, such as 2K. This figure contrasts the input (left) with InstaRevive's output (right), highlighting its impressive visual performance.
A APPENDIX
A.1 MORE QUALITATIVE RESULTS
We extend our framework to accommodate nishertnesolution images with diverse aspect ratios. As illustrated in Figure 9 we provide more qualitative results on larger-scale images, demonstrating model's ability to manage high-resolution images and restore pixel-level details. For further
16

---

Published as a conference paper at ICLR 2025
DSRGAN
RalESROAN
(4) SwinIR-GAN
ReaISR-PPGG
LDM-1100
(4) Inpus
( LUM-15
(5) ResStnn-15
O ReaShilt-4
SMISR-1
(13 InstaRevivee
BSRGAN
RealESRGAN
SWEERRGGAAN
RealS-JPEGG
LQ Input
LDM-1S
ResShin-15
ResSain-4
SinSR-1
(k) InstRevive-
TY KETS
TY
TY
TY
TY
high meets
QUALITY
TY
KETS
KETS
KETS
KETS
icest
SUPERRIA SEERS
BSRGAN
ReaESSRGAN
SwinIR-GAA
ReaISR-PPE
LDM-100
TY
TY
TY
TY
TY
KETS
KETS
KETS
KETS
KETS
Inpur
(2) LDM-15
(i) ResStiib-15
(i) ResShilt-4
SiiSR-1
(k) InstaReviive
Figure 10: More qualitative comparisons on real-world datasets. InstaRevive demonstrates high- performance blind image super-resolution on real-world images, consistently producing clear and detailed results with only single inference step.
evaluation, we compare method with recent state-of-the-ard approaches, including OSEDiff (Wu al. 2024) and SUPIR (Yu et al. 2024), Figure The results reveal that our framework delivers highly competitive performance. Moreover,
showcase more comparisons Figure I0 highlighting InstaRevive's 's versatility in enhancing styles ranging from comic images to real photos. We also provide more qualitative results of BFR tasks in Figure II Our InstaRevive successfully restores real-world images with challenging degradations, demonstrating the efficacy and robustness of our framework. For face cartoonization,
the BSR task in
compare InstaRevive with current diffusion-based methods like SDEdit (Meng et al.) 2021) (40 steps) and InstuctPix2Pix (Brooks et al. 2023) (100 steps). As shown in Figure [3] our framework yields visually appealing outcomes with finer details. Note that our framework only requires one step for inference.
A.2 QUANTITATIVE RESULT ON SYNTHETIC DATASET
To further assess the consistency of results in the BSR task, we conduct evaluations on the synthetic ImageNet-Test dataset, as proposed in ResShift (Yue et al., 2023). This dataset comprises 3,000 images randomly selected from the ImageNet validation set based on the widely-used degradation model. The results, presented in Table 4 indicate that our method achieves highly competitive consistency metrics, such PSNR and SSIM. Additionally,
report the performance of SwinIR- GAN (Liang et al. 202I). which serves as the restorer in our framework. The results reveal that the restorer struggles handle complex degradation scenarios, resulting in subpar performance.
17

---

Published as a conference paper at ICLR 2025
Real Input
GPEN
RestoreFormer
DMDNet
CodeFormer
DifFace-100
ResShitt-4
InstaRevive
Figure 11: More qualitative comparisons on the real-world faces. Our method performs plausible enhancement real-world faces, producing high-fidelity and visually satisfactory faces. Compared to other methods, InstaRevive enjoys robustness in front of challenging cases.
Real Input
OSEDETE
SL PUR-5550
InstaRevive-1
Real Inpex
OSEDit-11
SUPIR-50
InstaRevivel
Figure 12: Qualitative comparisons with recent methods. We present visual comparisons with additional methods, demonstrating that our model consistently delivers satisfactory results.
This highlights the strength of the generator in our framework, further validating its effectiveness in addressing challenging degradations.
Table 4: Quantitative comparison on ImageNet-Test. Our InstaRevive demonstrates strong
performance across various consistency metrics.
Method
PSNR
SSIM
LPIPS J
BSRGAN (Zhang et al. 2021
24.42
0.659
0.259
SwinlR-GAs GAN LLang et al 2021
23.97
0.667
0,239
ResShift Yue et al. 2023
24.90
0.673
0.228
SinSR /Wang et al. 2024
24.56
0.657
0.221
InstaRevive (Ours)
25.77
0.721
0.232
A.3 MORE EXTENDED TASKS
Our framework is versatile and not specifically designed for any single task. demonstrate its broader applicability, we extend our experiments to include (1) face inpainting, (2) low-light image enhancement, (3) denoising, (4) deblurring, (5) image-to-image transition, (6) deraining. We also
compare part of our performance with GDP (Fei et al.) 2023). Notably, our generator performs one-step inference while GDP requires 1,000 steps for sampling.
Face inpainting. For face inpainting, irregular polyline masks on face images as our inputs and fine-tune our BFR model for 10K iteration. During inference, resize the mask to the latent map's shape and use it to maintain the visible area on the inputs. As shown in Figure 14 our InstaRevive successfully reconstructs the challenging cases and seamlessly completes them with coherent content.
employ the script from GPEN Yang et al. 2021) to draw
Low-light image enhancement. For low-light image enhancement, we tne-tune the BSR model with 10K iterations using the Lol. dataset (Wei al., 2018). Our results, shown in Figure 14 demonstrate
18

---

Published as a conference paper at ICLR 2025
3D styli
oil painting style
Real Input
SDEdit
InstructPix2Pix
InstaRevive
SDEdit
InstructPix2Pix
InstaRevive
Figure 13: More qualitative comparisons face cartoonization. Our InstaRevive produces high-quality results compared with other diffusion- methods, underscoring its exceptional generalization capability for various enhancement tasks.
Masked GDP-10000
Ours-I
Low-light
GDP-1000
Ours-1
Figure 14: Face inpainting and low-light enhancement of InstaRevive. Our framework demon- strates the capability to produce satisfactory results across range of challenging tasks.
promising outcomes with less noise compared GDP. These findings illustrate that method can effectively generalize to these additional challenges.
Denoising. To evaluate the denoising capabilities of our approach, focus addressing salt-and- pepper noise, a common artifact arising during imaging or data transmission. Our one-step generator trained over 8K iterations using randomly generated salt-and-pepper noise with probabilities varying between 0.02 and 0.1. As illustrated in Row of Figure [5] the proposed method effectively removes this noise while preserving the fidelity of the input content.
The trained BSR model is inherently capable of performing deblurring tasks, as it leverages the degradation model from Real- -eal-ESRGANN Wang al. (2021c). which incorporates variety of blur types. As demonstrated in Row 2 of Figure 15 our proposed InstaRevive effectively restores clarity blurred images, delivering sharper details and well-defined edges.
Deblurring.
Image-to-image transition. Beyond face cartoonization, more complex challenge lies in transition- ing between real-world image domains. investigate this, we utilize the BDD100K dataset Yu et al. (2020) for training, specifically focusing on the dlay-to-night" transition. Our generator is trained for 10K iterations to achieve this task. As shown Figure [5] the proposed model generates visually realistic and temporally consistent night-time images from daytime inputs.
Deraining. For training, we employ RainTrainH Yang et al. (2017), RainTrainL. Yang al. (2017), and Rain I 12000 Fu et al. (2017) datasets, and evaluate framework the Rain-1001 dataset Yang et al. (2019). As illustrated in Figure [5] our approach effectively climinates rainy regions, producing clean and visually appealing results.
A.4 PARAMETERS AND INFERENCE TIME
As noted in limitations, one-step generator cannot surpass larger models employing multi-step inference. However, it achieves a favorable balance between efficiency and performance. To provide clear comparison of model parameters and inference time (evaluated Nvidia 3090 GPU),
19

---

Published as a conference paper at ICLR 2025
Input Image
Ours-1
Input Image
Ours-1 I
Denoising
Deblurring
Transition
Deraining
Figure 15: More extended tasks InstaRevive. Our framework demonstrates strong performance across more tasks, including denoising, deblurring, image-to-iman transitions, and deraining.
present Ta which includes state-of-the-art methods on RealSet65. Additionally, we fine-tune generator based on the denoising U-Net from Stable Diffusion 2.1 and include its results for reference. As demonstrated in Table our DiT-based generator achieves image quality comparable to SUPIR (Yu al., 2024) and DiffBIR (Lin et al.) (023b), while maintaining the inference speed advantages of one-step methods. The U-Net variant also delivers competitive results, although it exhibits a minor decrease in efficiency compared to DiT-based model. Furthermore, our method achieves comparable performance to OSEDiff (Wu et al. 2024), which incorporates additional tuning of the VAE encoder within the diffusion model.
Table 5: Comparison on parameter and inference time. Our generator strikes effective balance between computational efficiency and performance.
Method
Params.
Inference Time (s)
MANIQAT
MUSIQU
ResShift- 15
Yue et al.
2023)
121M
1.13
0.3958
61.33
SinSR-1
Wang et al. 2024)
119M
0.12
0.4374
62.64
OSEDiff-
Wu et al.
2024
866M
0.18
0.4573
65.68
SUPIR-50
Yu al. 2024)
3.86B
14.88
0.4735
66.79
D5
Lin el al.
(222bb
1.22B
10.27
0.4612
66.24
InstaRevive-1 (U-Net)
865M
0.18
0.4547
65.48
I (DiT)
611M
0.14
0.4571
65.85
20

---

Published as a conference paper at ICLR 2025
"3D style"
"oil painting style"
Input
SDEdit
ZeCon
Clipstyler
Ours
SDEdit
ZeCon
Clipstyler
Ours
Figure 16: Qualitative comparison with style transfer methods. Please zoom in for the best view.
Real Input
"wearing black glosses"
Real Input
"blonde hair and red hot"
Figure 17: Image-to-image transition with identity consistency. The generated image maintains similar identity to that of the input image.
A.5 IMAGE-TO-IMAGE TRANSITION IDENTITY CONSISTENCY
In tasks like image cartoonization, maintaining the source image's 's identity is not essential. However, in many other applications, such face swapping and style transfer, identity consistency is crucial. Compared with style transfer methods like Clipstyler (Kwon & Ye 2022) and ZeCon (Yang et al. 2023), our face cartoonization model falls behind in identity similarity, shown in Figure 16 To address this, we can utilize a teacher model equipped with Ahentity-preservatee
capabilities. This enables our generator to learn image-to to-ima transitions while retaining identity. our experiment, we employed the -d-Aateer (Ye et al. 2023). a diffusion based model with an image prompt adapter, as our teacher model ca- After training with InstaRevive's ' framework, our generator demonstrated a proficient capacity for identity preservation, as illustrated in Figure 17
A.6 ABOUT DIVERSITY AND CONSISTENCY
Consistency is our primary concern in the BFR and BSR tasks. We employ regression loss to ensure consistency and prevent unnecessary hallucinations during training. However, it is important to stress that, as discussed in works like BSRGAN (Zhang et al. 202I), these tasks are inherently ill-posed. The degradation processes real-world scenarios are often complex and unknown, leading to irreversible damage to images. This allows various plausible results from the low-quality input. It's notable that the allowance for multiple results does not necessarily cause inconsistency as long as they align with the input. To demonstrate the diversity of the output, provide visual results in Figure 18] showing five diverse outputs from different initial noises. These outputs include various shapes of the hat and slight differences in face identity, but all share consistency with the input.
A.7 FAILURE CASES
InstaRevive encounters challenges when dealing with extremely challenging scenarios. As illus- trated in Figure 19 our model faces difficulties with severely blurred inputs (left half) and images containing intensive objects and complex content (right half). Compared to GAN-based methods like BSRGAN (Zhang et al., 2021) which introduce many artifacts and blur, InstaRevive generates cleaner images. However, the final results may still exhibit unrealistic regions due to the challenging scenarios. Recently, some multi-step methods like SUPIR YYu et al. 2024) attempt to handle these limitations with large-scale scale models that generate finer details but at the cost of significant computa- tional resources and extended inference times. Despite these advancements, they also fall short in completely resolving these issues, sometimes leading artificial and unrealistic regions.
21

---

Published as a conference paper at ICLR 2025
Real Input
Diverse Outputs
Figure 18: Diversity of InstaRevive. Our framework can produce a range of results based on different initial noise inputs. While all outcomes are plausible, they exhibit slight variations in detail.
LQ Input
BSRGAN
Ours
LQ Input
BSRGAN
Ours
Figure 19: Failure cases. Our framework may give unsatisfactory results when confronted with severe degradation or complex content.
A.8 THEORY OF DYNAMIC SCORE MATCHING
A.8.1 DETAILS ABOUT SCORE MATCHING DISTILLATION
We start by expanding the KL divergence term DKL Po) as:
(ax (azly)
DKL(qullpo)
E log
E
(- (loogg((x)))) log (o(go))))))
(8)
Po (o
zm-ggo
(
where the LQ image dataset. To find the optimal point, we calculate the gradient as follows:
[v logPu(=l)) V log (x)g)()))))
ÖGo(#LQ. y)
VoDKL
(9)
FLO
00
==Go(LL
where denote the first two gradient terms V log Po(x)y) and V log (o y) as the scores of HQ images and generated images, respectively. These scores, akin to gradients of data density, suggest the use of diffusion models for computation. Ideally, the above optimization will match the distributions of HQ images and generated results. However, the score can easily diverge when the probability term is small-specifically, po (ar yy) vanishes when æ is far away from HQ images. Another issue that the score estimator, the diffusion model, performs best with noisy images obtained through the
diffusion process. Score-SDE (Song Ermon 2019 Song al. (0220) introduces a method that diffuses the original distributions with varying scales of noise indexed by and optimizes series of KL divergences between these diffused distributions, DKL (qellp/) l(pt).
Assuming that the characteristic functions of distribution go and po are on (8) and on (s). The diffusion process satisfying that 2; = a,æ + dge, where e ~ (O, Considering the property of characteristic function, obtain that:
0/s² Op. (s) = Ono (ars)) exp(- Jopo (ars)
(10)
222 - jo (ars). Therefore, we conclude that
the same way, we can get oy (s) = expp
DKL(gell::)
q: = p: + dg. Op. # dgp pp
go Po
(11)
For each t, DKL (gel and DKu (po) reach their minimum values simultancously. So it is equivalent to minimize D (g.ll).) Similar to Equation equation divergence is:
9
the gradient this KL
22

---

Published as a conference paper at ICLR 2025
| (Væ, log (c(x)l)) Væ, log (2)
@G y)
VoDKL= æ1,
(12)
00
Additionally, we adopt the same time-dependent scalar weight (()) for better training dynamics. We utilize pre-trained diffusion model 6ø to estimate Vn, log (#t y) and train another diffusion
model En to predict Væ, log (ozl)))). Therefore, the total gradient of the KL divergence becomes:
VOL = Ei,,, [u(()ox (Ew(<rly) - (0( @Go(oLQ.
(13)
06
A.8.2 DETAILS ABOUT DYNAMIC NOISE CONTROL
Predicted noise E PS æp and score 8.. We first clarify the relationship between these three important predicted targets in diffusion models. As detailed in (Luo. 2022). actually have Zo = (a, orEw)/ //a,, and sy = Et. Furthermore, we have:
Varaz of
|| (2r, t, y) - Vz, logp(x:lll) = Ev col
1 (zt, t) xo11
(14)
Where 2g is GT, and Eo the GT noise. Therefore, we conclude that the accuracy of predicted noise Cs. pseudo-Gi GT dgg and score 8g is consistent, allowing to determine the accuracy of the other two based on the accuracy of any one of them.
The motivation and rationale for controlling Tmax. As figured out in [ the estimates of gradients for real data distribution could be inaccurate or overly smoothed when x, is far away from THQ so we hope limit their mean distance. Considering the diffusion process that 0, ~ No0 I) and using triangle inequality, we obtain that
Egl,,, æhollz N Eg æ - 2112 + æ - x 0||2
E. ( 1)x + o 2 + I æ112
(15)
< a2 - 1 ||||=||| + E, + || 2112
Given that E,||e||2: [ - æ112 are all constants, the upper bound of E, 1|, zholle is indeed controlled by noise schedule or, d₁. When
a₁ and a, 0. As increases, or approaches O, while o tends to 1. Therefore, the upper bound of IE, || æhQ 22 will monotonically increase with t. By utilizing limited maxx, we can reduce the expected distance between o; and æHQ, thus ensuring better score estimates.
A.9 MORE IMPLEMENTATION DETAILS
Datasets, For BFR, we use FFHQ (CC BY-NC-SA 4.0) (Karas et al.) 2019) for training, which contains 70,000 face images in 1024 1024 resolution. For evaluation, utilize synthetic dataset CelebA (custom, research-only. non-commercial) (Liu et al. 2015) with 3,000 HQ-LQ pairs. We also employ LFW-Test (custom, rescarch-only) (Wang al. 2021b) and WIDER-Test (custom research-only) (Zhou et al. 2022) for in-the-wild evaluation. For BSR task, we use large-scale ImageNet (custom, research, non-commercial) (Deng et al. 2009 for training, and leverage RealSR (repository link) (Cai et al. 2019) and RealSet65 (NTU
S- License 1.0) Yuue et al. 2023
as benchmarks.
Training details. The training process is efficient with our dynamic score matching strategy, which focuses on refining detailed content in LQQ images. To train this framework, we utilize Nvidia A800 GPUs with approximately 2.5 days. As demonstrated in Section. 3 we concurrently update the parameters of the generator Gg and the score estimator eg- To achieve this, we adopt a two- stage training pipeline. Firstly, we calculate the regression loss and the KL divergence using the generated result æ. For the regression loss, we employ the Learned Perceptual Image Patch Similarity (LPIPS) (Zhang et al. 2018b) for better quality. The KL divergence is backpropagated equation Following this, we update the score estimator Eg to align the generated distribution qn- We detach æ and compute the diffusion loss according to equation This step ensures that es accurately learns
23

---

Published as a conference paper at ICLR 2025
the distribution of the generated images and provides precise scores. Note that two optimizers for these two models, respectively. In our framework, we mainly manipulate images in a latent space established by trained VQGAN, which comprises encoder & and a decoder D, facilitating seamless conversion between pixel space and latent space. To compute the LPIPS, we need to convert the latent codes pixel space with the decoder D.
A.10 CODE
We have included the complete source code for our method in the code folder. This folder contains all the necessary files and instructions to reproduce our experiments with the Revia framework.
24


Reviews:
[{"review_id": "6G9ujlUFed", "reviewer": "Reviewer_mTLA", "summary": "This paper propose a one-step diffusion framework called InstaRevive which employs score-based diffusion distillation with dynamic noise control and dynamic loss control. Dynamic noise control is to control the max noise level during training according to the difference of LQ and GT, and the dynamic loss control is to adjust the loss weights of fidelity and distribution. Then, this paper verifies the effectiveness of the framework on several enhancement tasks and generally performs well.", "questions": "Please see weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes a dynamic control method to constrain the optimization process according to the image distribution, which effectively improves the performance of the present method.\n2. The writing of the article is good, making the whole article clear and easy to understand\n3. This paper verifies the effectiveness of the method on some different enhancement tasks.", "weaknesses": "1. The paper does not mention some important pre-work (e.g., img2img-turbo and OSEDiff), which may exaggerate some contributions of this work. In fact, the methods (Reduce the steps by distillation) have been used before, and the proposed innovation core of this paper should be the dymic control.\n2. Due to the lack of reference to some previous work, in the comparison of other methods in the article are not new enough. For example, the visual effects and quantitative results of OSEDiff, SUPIR should be included.\n3. The ablation of dynamic control in the experiment shown in Figure 7 is not detailed enough. Because the main contribution is dynamic control, I suggest to draw two lines for the dynamic control of noise and the dynamic control of loss, instead of coupling them together, so that we can see which method is actually taking effect.\n\nThe article has more advantages than disadvantages on the whole, but I think the above weaknesses still need to be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HFQlWXGqwj", "reviewer": "Reviewer_LCnj", "summary": "This method improves image enhancement by utilizing a distillation approach, which significantly reduces time and computational resources. It introduces a dynamic score matching framework that effectively guides models to generate realistic (using KL divergence) and well-resolved (using regression) images from low-quality inputs. Additionally, the method is adaptable to various applications beyond BFR and BSR, including style transfer, low-light enhancement, inpainting, and more.", "questions": "1. Why is it necessary to use a restorer-fused method rather than zero-shot diffusion-based restoration methods, such as DPS, PSLD, or GDP?\n\n2. If possible, please provide additional comparisons with the restorer using traditional metrics.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. \"Introducing one-step restoration using diffusion models\" is a compelling topic for researchers in the field of image processing.\n\n2. The experimental results strongly support the authors' claims and demonstrate impressive restoration outcomes for real degraded images.\n\n3. Dynamic score matching can be valuable for various distillation studies to address potential inaccuracies at large timesteps.", "weaknesses": "1. While the method demonstrates impressive results, my main concern is that recent diffusion-based image restoration methods primarily focus on a zero-shot approach. My concerns are as follows:\n\n* The proposed method initially relies on a restorer, such as SwinIR-GAN, to enhance low-quality images. This approach implies that degradation types vary, requiring the model to use different restoration modules depending on the specific degradation family.\n* Additionally, the model is trained in a supervised manner using paired low-quality and high-quality datasets.\n\n\n2. More comparisons with the original restorer are needed.\n\n* From my perspective, the initial restorer recovers most of the information, while the distilled model further refines it to appear more realistic. If this is the case, it is crucial to compare the method with the initial restorer, especially using traditional metrics like PSNR, SSIM, and LPIPS. However, most experimental results do not provide this comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4U9JKMGLZv", "reviewer": "Reviewer_ZCNg", "summary": "InstaRevive presents a novel and efficient approach to image enhancement, leveraging score-based diffusion models for impressive results. The framework introduces dynamic score matching to refine the training process and incorporates textual prompts for enhanced guidance. Experimental results demonstrate superior performance across various tasks, including blind face restoration and blind image super-resolution.", "questions": "- How might InstaRevive need to be adapted to handle deblurring or dehazing tasks? Are there any particular challenges you foresee in extending the framework to these degradation types?\n\n- Have you considered applying InstaRevive to style transfer tasks between real images? What modifications, if any, would be necessary to adapt the framework for tasks like day-to-night image conversion or season transfer?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* **Clear and concise presentation:** The paper is well-structured and easy to follow, making the proposed method accessible to a wide audience.\n* **Strong performance:** InstaRevive consistently outperforms baseline methods on challenging image enhancement tasks, highlighting its effectiveness.\n* **Innovative approach:** The dynamic score matching technique and incorporation of textual prompts offer a unique and promising direction for image enhancement.", "weaknesses": "* While the paper demonstrates excellent results on face restoration and image super-resolution, it would benefit from exploring its applicability to other degradation types, such as deraining, dehazing, deblurring, and salt and pepper noise.\n* The paper's focus on face cartoonization as an image-to-image translation task could be expanded to include other types of image-to-image translations, such as real image to real image translation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)