DeblurDiff: Real-Word Image Deblurring with Generative Diffusion Models
Authors: Lingshun Kong
jiawei zhang
Dongqing Zou
Fu Lee Wang
Jimmy Ren
Xiaohe Wu
Jiangxin Dong
Jinshan Pan
Venue: NeurIPS 2025 poster ()
Keywords: Image deblurring
Primary area: applications

Abstract: Diffusion models have achieved significant progress in image generation and the pre-trained Stable Diffusion (SD) models are helpful for image deblurring by providing clear image priors. However, directly using a blurry image or a pre-deblurred one as a conditional control for SD will either hinder accurate structure extraction or make the results overly dependent on the deblurring network. In this work, we propose a Latent Kernel Prediction Network (LKPN) to achieve robust real-world image deblurring. Specifically, we co-train the LKPN in the latent space with conditional diffusion. The LKPN learns a spatially variant kernel to guide the restoration of sharp images in the latent space. By applying element-wise adaptive convolution (EAC), the learned kernel is utilized to adaptively process the blurry feature, effectively preserving the information of the blurry input. This process thereby more effectively guides the generative process of SD, enhancing both the deblurring efficacy and the quality of detail reconstruction. Moreover, the results at each diffusion step are utilized to iteratively estimate the kernels in LKPN to better restore the sharp latent by EAC in the subsequent step. This iterative refinement enhances the accuracy and robustness of the deblurring process. Extensive experimental results demonstrate that the proposed method outperforms state-of-the-art image deblurring methods on both benchmark and real-world images.

DeblurDiff: Real-World Image Deblurring with Generative Diffusion Models
Lingshun Kongl:", Jiawei Zhang", Dongqing Zou", Fu Lee Wang", Jimmy S. Ren Xiaohe Wu Jiangxin Dong' JJishan Pan Nannjnng University of Science and Technology "Sense Research RPBVR Hong Kong Metropolitan University Harrbn Institute of Technology
Abstract
Diffusion models have achieved significant progress in image generation and the pre-trained Stable Diffusion (SD) models are helpful for image deblurring by pro- viding clear image priors. However, directly using blurry image pre-deblurred one as a conditional control for SD will either hinder accurate structure extraction make the results overly dependent the deblurring network. In this work, we propose a Latent Kernel Prediction Network (LKPN) to achieve robust real- co-train the LKPN in latent space with
world image deblurring. Specifically, conditional diffusion. The I.PNN learns a spatially variant kernel to guide the restoration of sharp images in the latent space. By applying element- wise adaptive convolution (EAC), the learned kernel utilized to adaptively process the blurry feature, effectively preserving the information of the blurry input. This process thereby more effectively guides the generative process of SD, enhancing both the deblurring efficacy and the quality detail reconstruction. Moreover, the results each diffusion step are utilized to iteratively estimate the kernels in LKPN to better restore the sharp latent by EAC in the subsequent step. This iterative refinement enhances the accuracy and robustness of the deblurring process. Extensive experi- mental results demonstrate that the proposed method outperforms state-of-the-art image deblurring methods on both benchmark and real-world images.
Introduction
Image deblurring aims to recover sharp image from blurry observation. Blurring be caused by various factors, such as camera shake and high-speed movement of the photographed objects. This task is challenging as only the blurry images are available and the blur might be non-uniform. Traditional deblurring methods [17, 16] have made significant progress by utilizing hand-crafted features and priors. However, these methods often struggle handle complex blur patterns and may produce unsatisfactory results.
Numerous Learning-based approaches [25, 31, 1] have been inclined to employ variety of convolutional neural network (CNN) architectures. Compared with traditional algorithms, CNNs have demonstrated remarkable success. However, the convolution operation is spatially invariant local operation, which cannot effectively model the spatially variant characteristics of image content and the global contexts for image deblurring. To address limitations of CNNs, Transformers [32, 27, 12, 10] have been increasingly applied to image deblurring and have achieved commendable performance.
"This project is done during the internship at SenseTima Research. Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) Blurred patch
(b) DBGAN
(c) FFTformer
(d) HI-Diff
Blurred image from DVD dataset
(c) ControlNet
(f) PASD
(g) DiABBIR
(h) Ours
Figure 1: Visual comparison with state-of-the-art image deblurring methods. The results of GAN- based method (b) and diffusion-bas based method without pretraining (d) still contain significant blur effects. Directly using the blurry image the conditional input (e) presents significant challenges in effectively extracting structural information. (1) is a method based on pre-trained SD that performs pre- deblurring the input features, which alters the original information, leading erratic generation. For (g), it uses the result of the pre-trained FFTformer (c) as the condition. (g) is influenced by the erroneous structures in (c), resulting in generated outputs that retain erroneous artifacts and erroneous structures. In contrast, our approach, guided by the clear structural information provided by LKPN, generates more distinct and artifact-free image.
Recently, diffusion models have demonstrated outstanding performance in image generation [7, S]. Some rescarchers have attempted to utilize Denoising Diffusion Probabilistic Models (DDPMs) for image restoration [18, 30], aiming to leverage their ability to capture complex data distributions and generate detailed images. However, due to the lack of large-scale pre- treinirining, these methods have not demonstrated satisfactory results, particularly when applied to
data, often leading to inferior performance and visually unpleasant artifacts. With the surprising performance of large-scale pre-trained models like Stable Diffusion (SD) in image generation, it has been developed for image restoration [13, 29]. SD models have strong priors on the structure and details of high- quality (HQ) images. However, directly utilizing blurry images as conditional inputs can hinder the extraction of effective structural information, especially in cases of severe blur, ultimately resulting in inaccurately generated structures (Figure (e)). While recent methods attempt to mitigate these limitations, their technical trajectories introduce new issues. These methods [13, 29] require training an additional Degradation Removal Model (DRM) to first restore the clear images, and then enhance the details using a ControlNet [36]. This suggests that the restoration results of the entire method are notably influenced by the results of the degradation removal model. When the DRM (implemented via FFTformer [10]) yields inaccurate results (Figure (c)), it may adversely affect the diffusion process, potentially resulting in suboptimal performance (Figure (g)). Moreover, due the poor generalization of existing degradation removal models across different datasets, these methods also tend to perform poorly in real-world scenarios.
In this paper, we investigate in-depth the problem of how to leverage pre-trained SD models to assist in real-world image deblurring while reconstructing realistic details and textures. This approach circumvents the direct use of blurry images as conditions, as severe blur can hinder the extraction of accurate structural information, leading to suboptimal final generated structures. Unlike pre- training an additional degradation removal model, which uses restored images as conditions and can be problematic due to poor restoration quality introducing incorrect structures and resulting in erroneous generation, we jointly train Latent Kernel Prediction Network (LKPN) with the diffusion model. The LKPN, together with Element- wise Adaptive ConvolutionEEAC), designed to guide the conditional generation at each step of the diffusion process.
The effectiveness of the LKPN lies in its ability to predict dedicated convolution kernels for each latent pixel, dynamically adjusting the kernel weights based on local content (e.g.. edges, textures, and flat regions). These pixel-specific kernels are then applied to the latent blurry image by the EAC, enabling better restoration of clear structures by adaptively addressing distinct blur characteristics at each latent pixel location. This adaptive mechanism not only preserves the necessary information in the input image but also avoids the destruction of structural details, making it particularly advantageous for recovering fine structures and textures. By integrating the LKPN, the diffusion model uses the clear structures from the LKPN as conditional inputs to guide its generation process at each step. This produces more accurate results and enhances both deblurring and detail reconstruction. Furthermore, the intermediate results obtained at each step of the diffusion process are utilized to refine the output of the LKPN,, progressively improving the accuracy of the deblurred results by EAC. This iterative refinement generates increasingly clearer conditional inputs, which in turn guide the generation

---

(a) Input
(c) ControlNet
Deblurred image by LKPN and EAC
(b) FFTformer
(d) DiMIBI
(I) DeblurDiff
Figure 2: Iterative results of the diffusion model. The arrow represents the iterative diffusion process. To visualize this process, we decode the results of each step of the diffusion model through the VAE decoder to the image space. Using the blurry image directly as conditional input (c) makes the diffusion model struggle to recover clear structures and fine details in (a). For (d), it uses the result of the pre-trained FFfformer (b) as the condition. However, when the results in (b) fail remove the blur, the generated results will also contain blur (as indicated by the blue box). In contrast, the proposed LKPN can preserve the input information and restore the structure (c) by EAC, thereby guiding the diffusion model to generate better results (f), where the leaves are generated more clearly.
process of the Stable Diffusion (SD) model achieve iterative improvements. Additionally, the LKPN continuously benefits from the strong prior information provided by SD, enabling it to estimate better kernels to remove blur in the latent space.
The main contributions are summarized as follows: First, we design an architecture based on pre- trained SD models to achieve effective deblurring while reconstructing realistic details and textures. Second, we propose an LKPN architecture that estimates a spatially variant kernel, which is then utilized by the EAC to progressively generate clear structures and preserve the necessary information the input image throughout the diffusion process. This guides SD produce more accurate details and structures, thereby enhancing the overall deblurring performance. Finally, quantitatively and qualitatively evaluate the proposed method on benchmark datasets and real-world images and show that our method outperforms state-of- the-art methods.
Related Work
Image deblurring. Due to the fact that image deblurring is an ill-posed problem, traditional methods [11, 17, 16] often develop various effective priors to constrain the solution space. These manually designed priors help remove blur. However, they do not fully utilize the characteristics of clear image data, which leads to a struggle in handling complex blur patterns and may produce unsatisfactory results
With the development of deep learning, many lcarning-basse methods have tended to use various CNN architectures for image deblurring. SRN [25] proposes multi-scale structure that performs image deblurring from coarse to fine. MIMOUnet [3] redesigns the coarse-to- structure, significantly reducing the computational cost. NAFNet [1] analyzes the baseline module and simplifies it by removing the activation function, which better facilitates image restoration.
Due to the excellent performance of Transformers in global context exploration and their great potential in many visual tasks, some methods have applied it to image deblurring. Restormer [32] simplifies the baseline module by estimating self-attention in the channel dimension, reducing the computational cost of self-attention Uformer [27] proposes a general U-shaped Transformer model, computing self-attention within local windows address the image deblurring. FFTformer [10] proposes a frequency-done -domann based Transformer model and achieves state-of ofethhh-rtt results.
Although these methods have achieved good deblurring effects, these regression-based methods tend to predict smooth results, with limited ability depict details.
Diffusion model. Denoising Diffusion Probabilistic Models (DDPM) [7] have shown remarkable capabilities in generating high-quality natural images. Some methods [30, 18, 2] have attempted to
directly train a diffusion model for image restoration. Rombach et al. [21] extended the DDPM struc- ture to the latent space and conducted large-scale pre-training, demonstrating impressive generative

---

capabilities. Recently, some researchers have utilized powerful pre- treinrained generative models, such as SD [21], to address image restoration problems. DifBBIR [13] proposes two-stage approach. first restoring the degraded image and then using SD to generate details. PASD [29] restores clear images through a Degradation Removal module to provide clear conditional inputs for SD. However, these methods require training an additional image restoration model and then enhance the details through SD. This means that the final results of the SD largely depend on the outcomes of the restoration model. When the degradation removal model produces erroneous results, it may lead to poor performance in the diffusion process.
3 Method
Our goal is to leverage the powerful priors of pre-trained Stable Diffusion models to achieve robust image deblurring while simultancously reconstructing realistic details and textures. Our method iteratively provides clearer guidance to the diffusion model during the diffusion process, thereby progressively restoring clear structures and enhancing details. Figure shows the overview of the proposed method.
3.1 Motivation and Preliminaries
Motivation. Directly using the blurry image as the condition for SD often results in suboptimal deblurring, as it lacks sufficient structural information to effectively guide the generation process. This makes it difficult for diffusion models to recover clear structures and details, as shown in Figure 2(c). Therefore, providing SD with clear guidance images is necessary. However, directly predicting clear structures from the blurry image not an easy task and often leads to inaccurate results (Figure 2(b)), which can lead to erroneous structures in the final output of the diffusion model (Figure 2(d). However, directly predicting a sharp structure from a blurry image is a highly ill-posed problem and often results in inaccurate estimations (Figure 2(b)), which can further lead to erroneous structures in the final output of the diffusion model (Figure 2(d)).
Our work is inspired by existing methods [24, 22, 16], which iteratively estimate the blur kernel and restore the blurred image. Since directly estimating an accurate blur kernel from the blurred image often leads to suboptimal results with residual blur, they first utilize the blurred image along with various priors (e.g., patch priors [24] and dark channel priors [16]) to first estimate an intermediate deblurring result with clear structures. Based on this intermediate result, they then estimate the corresponding blur kemnl,, which is subsequently used to refine the intermediate deblurring result. This iterative process alternates between kerel estimation and refinement of the intermediate deblurred result, progressively improving the overall deblurring performance. Building upon these ideas, we propose method that integrates deblur kernel estimation with the clear priors inherent in SD. Specifically, our method uses the priors provided by SD and the blurry input image to first estimate deblur kernel in the latent space. This kernel is then applied to the blurry image in the latent space to produce a clearer intermediate result, which guides the diffusion model for conditional generation (Figure 2(e)). During the diffusion process, the results generated by the diffusion model are combined with the original blurry image to iteratively estimate a more accurate deblur kernel and progressively refine clearer guidance image. As the iterations proceed, the enhanced guidance enables SD to progressively recover finer details while preserving the structural integrity of the input. As result, the diffusion model can generate outputs that are not only sharper and more detailed but also better preserve the original content (Figure 2())).
Denoising Diffusion Probabilistic Models (DDPM) learn data distributions by progressively de- noising a normally distributed variable. The training process involves a forward noise-adding process and reverse denoising process. At the t-th step of the forward diffusion process, a noisy image x; is generated from the clear image Ty by:
St / + v1 044,,
where denotes the noise sampled from the standard normal distribution //, I), and ā controls the amount of noise added at each step 4, which is typically a predefined value. In the reverse denoising process, a denoising neural network en =0(x 1) is used to predict the noise in s4. After training, the denoising network can iteratively run for T steps starting from noise sampled from standard normal distribution to generate clear image. To enhance efficiency, the diffusion process is often moved to

---

predicted kernel
Latent Kernel Prediction Network
Element-wise
Adaptive Convolution
Blurred image
kky
Ky
1g
Encuderr
Concat
A
-
Zo
Element-wise Addition
T-]
2
VAE Decoder
Fixed
Lincar
Z4
H
Resblock
Atteation
Linear
Trainable
ig
K
Zero
Convolution
(a) Latent Kerne Prediction Network
(b) Flement-wise Adaptive Convolution
Restored image
Figure 3: Overall architecture the proposed DeblurDiff. integrates Latent Kernel Prediction Network (LKPN) with generative diffusion model to address the challenges of real-world image deblurring. The LKPN leverages the priors from SD and the blurred image to estimate pixel-specific deblurring kernels during the diffusion process. These kernels applied via Element-wise Adaptive Convolution (EAC) to progressively recover clear structures from blurred images. The refined clear 2" is used as condition to guide the diffusion process, enabling the model to effectively preserve the input information and structural integrity.
latent space via pre-trained Variational Autoencoder (VAE). This latent space is more suitable for likelihood-based generative models as it focuses on the important and semantic parts of the data, operating -iver-dimensional and computationally efficient space [21]. Specifically, a pre-trained VAE is utilized to transform the clear image zo into a latent representation by 20 {(zo,, and the forward noising and reverse denoising processes of DDPM are performed this latent space. The corresponding optimization objective can be simplified to minimizing the denoising loss:
||e ea ez (2(.,)))1
Lennose E,,,c-
(2)
where 2, is the noisy latent representation at step t.
3.2 LKPN for clear structure guidance.
In the context blind deblurring, directly using blurry images as conditions can impede the extraction of effective structural information, especially when dealing with significant blur, which can lead to inaccurate final generated structures. To address this challenge, we propose a method that incorporates an LKPN trained simultaneously with the diffusion model. The LKPN dynamically estimates pixel- specific deblurring kernels at each step of the diffusion pipeline, by jointly leveraging the blurred input and intermediate image priors generated during the diffusion process. These estimated kernels are fed into the element- wise adaptive convolution (EAC), which applies content-awar convolutions based local image structures, allowing the model to effectively address spatially varying blur patterns in the latent space.
Inspired by STFAN [38] and DWNet [6], which predict pixel-wise kernels from single adjacent frames, we adopt similar philosophy spatially adaptive convolution. However, while these methods perform a single, static estimation in the input space, we adapt this idea to the latent space of pre-trained diffusion model. Our LKPN re-estimates the blur kernel at every denoising step by conditioning the current, progressively cleaner latent produced by the diffusion backbone, and
5

---

Table 1: Quantitative evaluations of the proposed method against state-of-the-ar ones both synthetic and real-world benchmarks. The models marked with an asterisk indicate that we retrain them our training set. The best and second performances are marked in red and blue, respectively. For the RWBI and Real Blurry Images datasets, which lack ground truth (GT) data, we evaluate the performance using only no-reference metries. We provide comparison results with more methods in the supplementary material.
Dataset
Metries
FFTYormer
DBGAN
ResShift
HI-Diff
CootrolNet"
PASD
DiABIR
Ours
PSNR
26.86
31.18
29.03
33.33
22.31
22.82
23.86
24.32
0,9182
0.6559
0.7375
LPIPS
0,1538
0.1120
0.0780
0.0820
0.3292
0.2984
0,2772
0.2191
GoPro
FID
13,8896
10.7629
8.8820
8.1553
37,2749
39,0057
27.0576
17.6948
NIQE
4,1200
5,1988
4,8367
4,6119
3,5305
2,6567
3,4193
3.1769
MUSIQ
42,0985
MANIQA
0.5454
0.4976
0.5419
0.5119
0.5746
0.5904
0,5464
0.6134
CLIP-IAA
0.4360
0.3788
0.4229
0.4841
0.5869
0.5758
0,5260
0.5966
PSNR
27.07
27.78
27.78
30.31
22.03
22.23
23.49
23.74
SSIM
0,8534
0,8356
0.8420
0.8972
0,6409
0,6440
0.7114
0.7055
0,2126
0.2501
DVD
NIQE
FID
3,8562
6.7968
14.9420
4,8188
7.1787
4,6422
5,1858
5,7370
24.1213
3,4037
22.3637
3,2504
20.2467
3.1357
12.5545
2,7822
MUSIO
60.1091
40.478.
52.7551
45,5395
65.3657
68,3299
61.6415
67, 67.2447
MANIQA
0,5548
0.6480
CLIP IQA
0.5271
0.4346
0.4831
0.4065
0.6606
0.6528
0.5829
0.6686
PSNR
26.94
23.91
26.30
30.18
23.77
02
25
25.71
SSIM
0,8580
0.7434
0,8140
0.9049
0.6787
0.7642
0.7724
0.7705
LPIPS
0,1411
0.2945
0.1249
0.0868
0.2565
0,2075
0.1951
0.1693
Realhlur
NIQE
4,3473
99.5935
5,3238
5,2628
5.1437
4.6525
3.9192
4,3053
22.7713
4,2666
MUSIQ T
61,5808
38,4866
49.3209
57,1640
66.5174
61.1488
58,7450
65 0557
MANIQA
0,6374
0,43222
0.5373
0.6218
0.6452
0,5994
0,5869
0.6538
0,3469
0.6087
NIQE
4,4631
$,2905
5,4446
5,3373
5,0331
4,1973
4,2742
4,5171
RWBI
MANIQA
MUSIQ
59,6223
0.5425
42.7631
0.4852
51,0359
0.4953
0.5082
1820
62,5079
0.5758
0.5645
1680
61 8865
0.5618
66.
0.6260
CLIP-IQAT
0,5413
0,3645
0.5032
0,3907
0,6199
0,5820
0.6042
0.68-49
NIQE
3,8520
4,9338
5,4704
4, 4,7018
4,0978
4,4460
3,7964
3,6628
MUSIQ
$2,9290
32,0568
48.8154
43,8702
51,5191
61.6320
53,6088
52,9263
Images
MANIQA 0.5170
0,4488
0,5345
0,5722
0.5544
0.5937
0.5564
0.5963
CLIP-IQA
0.5026
0.3501
0.4767
0.4545
0.5254
0.5919
0.5384
0.5496
immediately applies it via EAC. This forms a closed iprrior-sstimatt-restor loop that turns the conventional static kernel prediction into dynamic, iterative refinement process, requiring no extra frames yet fully exploiting the generative priors of the diffusion model.
Specifically, we first employ a pre- pre-trained and frozen VAE encoder initialized from SD to encode the blurry image and the clear one into the latent space, obtaining their corresponding latent representa- tions Bty and zg- We follow the (1) to add noise 2g, obtaining 2,. Then the LKPN, which is U-Net architecture, predicts a spatially variant kernel in latent space at step t:
k LKPN(2t: Zig t),
(3)
where hy is the predicted kernel time step L. Then is used refine the blurry image in the latent space by:
= EAC(Ziq, kt),
where EAC is the clement-wiss adaptive convolution in Figure 3 (b) and 2 is the deblurred latent result. In the supplementary material, we provide detailed explanation of the LKPN and EAC architecture.
During training, the LKPN trained simultaneously with the diffusion model. The LKPN follows the framework of DDPM, imposing constraints simultaneously in both the latent space and the pixel space to progressively optimize the kernel estimation. Specifically, our objective is to minimize the following objective:
LLKPN = Llatent Lpisel,
Liatent =
E Ezo.zig &e
[120 DEACCEAAI kg 61)1
(5)
Lpizel = Ezo |1((z)) - D(EAC(Z) Aie kxx))

---

(a) Blurred patch
(b) DBGAN
(c) FFTformer
(d) HI-Diff
Blurred image from DVD dataset
(c) ResShift
(f) PASD"
(g) DifiBIR*
(h) Ours
Figure 4: Deblurred results on the DVD dataset [23]. Existing methods struggle to effectively restore
clear images. In contrast, our approach not only removes blur but also recovers sharp structures and fine details.
(a) Blurred patch
(b) DBGAN
(c) FFTformer"
(d) HI-Diff
Blurred image from RWBI dataset
(c) Resshift
(f) PASD"
(g) DifiBIR
(h) Ours
Figure 5: Deblurred results the RWBI dataset [33]. The structures not recovered well in (b)-(g).
The proposed method generates an image with much clearer structures.
where EAC denotes the element- wise adaptive convolution (EAC) in Figure (b), and D denotes the pre-trained VAE decoder of SD.
3.3 Conditional diffusson for image deblurring.
For the conditional generation network, we follow the training methodology of ControlNet [36], given its demonstrated effectiveness in conditional image generation. Specifically, adopt the encoder the UNet in SD as a trainable conditional control branch and initialize the control network by copying weights from the pre-trained SD model. We concatenate z recovered by the EAC and Zig as the input to the ControlNet, which is initialized with the weights of SD. During training, the LKPN and ControlNet are jointly trained following the framework of DDPM [7], we minimize the following loss function:
Leenoise (2) + LLKPN (5).
3.4 Sampling process of the DeblurDiff.
During the inference stage, the LKPN first estimates the initial spatially variant kernel from random Gaussian noise and the blurred image, which is used to obtaín initial conditional zÿ. The controlled diffusion model uses zf. and Big as conditions to generate initial clear result 2y Subsequently, 2T-1 is fed back into the LKPN to help estimate more accurate kernel:
kr- LKLN (2T-122-111 21: 1),
thereby iteratively optimizing the generated results in subsequent steps. We provide more details in the supplementary material.
This synergy between the LKPN and the diffusion model creates a mutually reinforcing cycle, where clear structural guidance from the LKPN improves the diffusion process, and intermediate results from the diffusion model further refine the deblurred results.

---

(a) Blurred patch
(b) DBGAN
(c) FFTiormer
(d) HI-Diff
Blurred image from Real Images
(e) Resshift
(I) PASD"
(g) DifIBIR*
(h) Ours
Figure 6: Deblurred results on real blurry images dataset [4]. The deblurred results in (b)-(g) still contain significant blur effects. The proposed method generates clear image.
Experimental Results
4.1 Experimental Settings
Training Datasets. Current deblurring datasets are generally small in scale and low in resolution, which is insufficient for effectively training diffusion models. Therefore, do not use common deblurring datasets such as GoPro [14] as our training set. Instead, we have collected and created a large-scale dataset containing approximately 500,000 data pairs. Our training dataset consists of three parts: (1) Existing deblurring datasets (including MC-BII Blur [34] and RSBlur [20]). (2) We capture some high-definitun
video clips, generating blurred and clear data pairs using the same strategy as
REDS [15]. (3) We collect a large number of high-definition images and generate various motion blur kernels synthesize corresponding blurred images. We provide dataset the supplementary material.
details about the training
Test Datasets. We evaluate the proposed DeblurDiff on commonly used image deblurring datasets, including synthetic datasets (GoPro [14], DVD [23]) and Real Blurry Images [4], RealBlur [19], RWBI [33].
Implementation Details. We use SD2.1 the base model. We employ the Adam optimizer [9] train DeblurDiff with batch size of 128. The learning rate is set to fixed value of 5 x 10 The model trained for 100,000 iterations using 8 NVIDIA 80G-A100 GPUs.
Evaluation Metrics, We employ a range of reference- based and no-reference metrics to provide comprehensive evaluation of the deblurring performance. For fidelity and perceptual quality
assessment, we employ reference-basee metrics, including PSNR, SSIM, and LPIPS [37]. For no-reference evaluation, include NIQE [35], MANIQA [28], MUSIQ [8], and CLIPIQA [26], which assess image quality based on statistical and learning-basse approaches. This diverse set of metrics ensures thorough analysis of both fidelity and perceptual quality.
Compared Methods. We compare our DeblurDiff with several state-of-the-art image deblurring methods, which are categorized into two groups: (1) hon-diffusion-base methods, including FFT- former [10] and DBGAN [33], and (2) diffusion-based methods, including HiDiff [2], ResShift [30], ControlNet [36], PASD [29]. and DifIBIR [13]. We employ the pre-trained FFTformer the Degra- dation Removal Module (DRM) in DifIBBR and retrain DifiBIR, as the original DifiBIR framework is not designed for deblurring tasks. We also train ControlNet and PASD for image deblurring tasks. All the aforementioned retrained methods are trained and tested using the publicly released codes and models of the competing methods, ensuring fair and consistent comparison under the same dataset and training settings as our proposed method.
4.2 Comparisons with the state the arts
We evaluate our approach on the synthetic and real-world datasets. Table shows the quantitative results. Our method shows strong performance no-reference metrics, achieving higher scores compared to existing approaches. This indicates that our method excels in perceptual quality and realism, which are critical real-world applications where ground truth images are often unavailable. Our method achieves lower scores in reference-based metrics compared to HI-DIff. However, HI-Diff does not utilize a pre-trained SD model, thus focusing more on reference-basee metrics while lacking generative capabilities, which results in relatively worse performance in no-reference metrics.

---

Table 2: Effectiveness of each component in the proposed method on the GoPro dataset [14].
LKPN
EAC
prior for LKPN
PSNR
SSIM
NIQE
MUSIQ
MANIQA
CLIP-IQA
ControlNet
X
X
X
22.31
0.6547
3.5305
59,4246
0.5746
0,5869
EAC
V
X
22.57
0.6553
3,4872
58.5845
0,5834
0.5701
w/o for LKPN
X
23.14
0.6614
3.3391
59.7763
0.5917
0.5914
DeblurDiff
24.32
0.7375
3.1769
61.6369
0.6134
0,5966
Figure 4 shows visual comparisons on the synthetic dataset of DVD [23]. The GAN-based method [33] exhibits significantly inferior deblurring performance, failing to restore clear structures and fine details effecttively. Existing diffusion-based methods, such as HiDiff [2] and ResShift [30], fail to achieve satisfactory results in Figure 4(d) and (e) due to their lack of SD priors, which leads to suboptimal generation quality in terms of both structural clarity and detail fidelity. Leveraging the pre-trained FFTformer [10] to preprocess blurred images Figure 4(c) and subsequently applying diffusion- based restoration can partially remove blur. However, since FFTformer sometimes cannot eliminate blur and tends to introduce undesirable artifacts during preprocessing, the final results generated by DiffBIR often exhibit unnatural structures and are inconsistent with the input. In contrast, our method, guided by the LKPN that progressively generates clear structures, produces better results with sharper and more accurate structures.
Figure and Figure present the visual comparison results on the real-world dataset. DBGAN [33] and FFTformer [10] struggle recover clear structures from severe blur, while Hi-Diff [2] and
ResShift [30] fail to reconstruct fine details due to the lack of pre-trained image priors. PASD [29]
and DiffBIR [13], which rely on degradation removal models for pre-deblurring, often produce suboptimal results with noticeable artifacts, as these models cannot effectively handle complex blur patterns. In contrast, our method demonstrates better performance both structural recovery and detail reconstruction. The LKPN progressively generates clear structures and adaptively refines the deblurring process, enabling our approach to achieve high-quality results with minimal artifacts.
Input
ControlNet
w/o EAC
(d)w/o SD LKPN
(e)DeblurDiff
Figure 7: Effectiveness of the proposed DeblurDiff on image deblurring. ControlNet struggles to recover clear structures due the lack of explieit struetural guidance, resulting in blurred outputs. Without the Stable Diffusion (SD) priors (referred to as w/o SD for LKPN), the LKPN fails to leverage intermediate clear priors for deblurring, leading artifacts and inconsistencies in the generated results. When the LKPN directly predicts the deblurred result without using EAC (referred to as w/o EAC), the outputs tend be overly smooth, losing important details and structural information. In contrast, our method effectively recovers sharp structures and fine details while preserving the input information.
4.3 Analysis experiments.
The proposed LKPN is used to leverage intermediate clear priors generated during the diffusion process, providing clear structural guidance as conditional inputs the diffusion model. When the blurry image is directly used as the conditional input (ControlNet for short), the diffusion model struggles recover clear structures and fine details (Figure 7(b)), particularly in cases of severe blur. This is because the blurry image lacks sufficient structural information to guide the generation process effectively, leading to suboptimal deblurring performance. Table shows the quantitative evaluation results on the real blurry images dataset.
To validate the effectiveness of leveraging clear priors from the diffusion process the LKPN, we compare our method with the baseline that directly estimates the kernel from the blurry image without utilizing the intermediate results from the diffusion process (w/o SD for LKPN for short). In this baseline, the estimated kernel is applied to the blurry image to generate deblurred result, which then used as conditional input to the diffusion model. Since the LKPN lacks the guidance of intermediate clear priors, its predictions remain static and do not improve over time (Figure 8(a)). As a result, the LKPN struggles to provide accurate structural guidance, leading to structural emros and inconsistencies in the generated results, shown in Figure 7(d).

---

(a) Visualization of the LKPN results during the diffusion process in the baseline "w/o SD for LKPN".
(b) Visualization of the LKPN results during the diffusion process in the bascline "w/o EAC".
(c) Visualization of the LKPN results during the diffusion process in the DeblurDiff.
Figure 8: Iterative results of LKPN. The arrow represents the iterative diffusion process. To visualize this process, we decode the features deblurred by the LKPN to the image space using the VAE decoder in each time step.
We further investigate an alternative design which the LKPN directly predicts the deblurred latent than estimating element semeent-wise adaptive kernels to serve as the conditional input for diffusion. This variant, denoted w/o EAC, performs worse than our full method that predicts kernels and applies EAC to produce clear guidance. This approach fails to adaptively address distinct blur characteristics at each pixel location, resulting in artifacts and inconsistencies in the generated images. Additionally, the generated results of LKPN tend to be overly smooth (Figure 8(b)), and as a consequence, the deblurred images lack sharpness and fine details, leading to suboptimal visual quality and only moderate fidelity. Furthermore, it cannot effectively preserve the imput information and structural integrity, resulting in a loss important details and coherence in the deblurred results (Figure 7(c)).
In contrast, our method leverages intermediate clear priors from the diffusion process to iteratively refine the spatially variant kernels estimated by the LKPN. This iterative refinement enables the LKPN to progressively improve its predictions, generating increasingly accurate deblurring results (Figure 8(c)). The refined kernels are then applied through the Element-wise Adaptive Convolution (EAC), which adaptively addresses distinct blur characteristics each pixel location, effectively preserving the imput information and recovering sharp structures and fine details (Figure 8(e)).
Limitations. We develop an effective method that explores the properties of the pre-trained SD for image deblurring. However, we have currently only experimented with the SD2.1 model. In the future, we will extend our method to the SD3.5 model based on DiT and attempt to optimize multi-step diffusion into a single-step diffusion to accelcrate model inference.
Conclusion
In this paper, we propose DeblurDiff, framework for real-world image deblurring that integrates Latent Kernel Prediction Network (LKPN) with a generative diffusion model. Our approach addresses the limitations of existing methods by leveraging the priors of pre-trained SD models and introducing an adaptive mechanism to estimate pixel-specific kernels. These kernels are applied through Element- wise Adaptive Convolution (EAC), which adaptively adjusts to local content, enabling the model to preserve input information and structural integrity effectively. Through extensive experiments on both synthetic and real- well-wrrdd benchmarks, we demonstrated that DeblurDiff performs favorably against state-of-the thete-rrt methods in terms of structural fidelity and visual quality.
10

---

Acknowledgment. This work has been supported in part by the National Natural Science Foundation of China (Nos.62272233, U22B2049, 62332010, and 62476067).
Broader Impact. This work explores how diffusion-based deblurring module can be plugged into a pre-trained Stable Diffusion pipeline to restore sharp images on edge devices. The positive implications are immediate: clearer photos on low-cost smartphones, reduced bandwidth in video conferencing, and better visual evidence for journalists or firstrespoonerss working in the field. At the same time, the proposed model could be misused, raising concerns about privacy and forensic validity, We therefore advocate responsible disclosure: any public release of the model should be accompanied by (I) hisbbe-wwtermarkkngg of Al-restored regions, (2) cryptographic provenance metadata, and (3) clear user agreements that prohibit non-consensual enhancement of biometric imagery. Overall, we believe the societal benefit outweighs the risks when these simple safeguards are observed.
References
[1] Liangyu Chen, Xiaojic Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. In ECCV, 2022.
[2] Zheng Chen, Yulun Zhang, Liu Ding, Xia Bin, Jinjin Gu, Linghe Kong, and Xin Yuan. Hierarchical integration difffusion model for realistic image deblurring. In NeurlPS, 2023.
[3] Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. Rethinking coarse-to-inee approach single image deblurring. In ICCV, 2021.
[4] Sunghyun Cho, Jue Wang, and Seungyong Lee. Video deblurring for hand-held cameras using patch-based synthesis. ACM Transactions on Graphics (TOG), 31:1 9, 2012.
[5] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurlPS, 2021.
[6] Qi Han, Zejia Fan. Qi Dai, Lei Sun, Ming-Ming Cheng, Jiaying Liu, and Jingdong Wang. On the connection between local attention and dynamic depth-wise convolution. In International Conference on Learning Representations, 2022.
[7] Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[8] Junjie Kc, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In ICCV, 2021.
[9] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In ICLR, 2015.
[10] Lingshun Kong, Jiangxin Dong, Jianjun Ge, Minggiang Li, and Jinshan Pan. Efficient frequency domain- based transformers for high-quality image deblurring. CVPR, 2023.
[11] Dilip Krishnan, Terence Tay, and Rob Fergus. Blind deconvolution using normalized sparsity measure. In CVPR, 2011.
[12] Yawei Li, Yuchen Fan, Xiaoyu Xiang, Denis Demandolx, Rakesh Ranjan, Radu Timofte, and Lue Van Gool. Efficient and explicit modelling of image hicrarchies for image restoration. In CVPR, 2023.
[13] Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Yu Qiao, Wanli Ouyang, and Chao Dong. Diffbir: Toward blind image restoration with generative diffusion prior. In ECCV, 2024.
[14] Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, 2017.
[15] Seungjun Nah, Sungyong Baik. Seokil Hong, Gycongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. Ntire 2019 challenge video deblurring and super-resolution: Dataset and study. In CVPRW, 2019.
[16] Jinshan Pan, Deqing Sun, Hanspeter Pfister, and Ming-Hsuan Yang. Blind image deblurring using dark channel prior. In CVPR, 2016.
[17] Jinshan Pan, Zhe Hu, Zhixun Su, and Ming Ming-Hsuan Yang. lo regularized intensity and gradient prior for deblurring text images and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2):342-355 2017. doi: 10. 10.109/TAAMM22211.5555112

---

[18] Mengwei Ren. Mauricio Delbracio, Hossein Talebi, Guido Gerig, and Peyman Milanfar. Multiscale structure guided diffusion for image deblurring. In CVPR, 2023.
[19] Jaesung Rim, Haeyun Lee, Jucheal Won, and Sunghyun Cho. Real-world blur dataset for learning and benchmarking deblurring algorithms. In ECCV, 2020.
[20] Jaesung Rim. Geonung Kim, Jungeon Kim, Junyong Lee, Seungyong Lee, and Sunghyun Cho. Realistic blur synthesis for learning image deblurring. In ECCV, 2022.
[21] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent difffusion models. In CVPR, 2022.
[22] Qi Shan, Jiaya Jia, and Aseem Agarwala. High-quality motion deblurring from a single image. Acm transactions on graphics (tog), 2008.
[23] Shuochen Su, Mauricio Delbracio, Juc Wang, Guillermo Sapiro, Wolfgang Heidrich, and Oliver Wang. Deep video deblurring for hand-held cameras. In CVPR, 2017.
[24] Libin Sun, Sunghyun Cho, Jue Wang, and James Hays. Edge-hased blur kemnel estimation using patch priors. In ICCP, 2013.
[25] Xin Tao, Hongyun Gao, Xiaoyong Shen, Juc Wang, and Jiaya Jia. Scale-recurrent network for deep image deblurring. In CVPR, 2018.
[26] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In AAAI, 2023.
[27] Zhendong Wang. Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: A general u-shaped transformer for image restoration. In CVPR, 2022.
[28] Sidi Yang. Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi- multi-dimension attention network for no-reference image quality assessment. In CVPR), 2022.
[29] Tao Yang, Rongyuan Wu, Peiran Ren, Xuansong Xie, and Lei Zhang. Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization. In ECCV, 2024.
[30] Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Resshift: Efficient diffusion model for image super-resolution by residual shifting. In NeurlPS, 2024.
[31] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Husgn Yang, and Ling Shao. Multi-stage progressive image restoration. In CVPR, 2021.
[32] Syed Waqas Zamir, Aditya Arora. Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022.
[33] Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Bjöm Stenger, Wei Liu, and Hongdong Li. Deblurring by realistic blurring. In CVPR, 2020.
[34] Kaihao Zhang, Tao Wang, Wenhan Luo, Wenqi Ren. Bjöm Stenger, Wei Liu. Hongdong Li, and Ming- Hsuan Yang. Mc-blur: A comprehensive benchmark for image deblurring IEEE Transactions on Circuits and Systems for Video Technology, 34(5):3755-376, 2024.
[35] Lin Zhang, Lei Zhang, and Alan C. Bovik. A feature-enriched completely blind image quality evaluator. IEEE Transactions on Image Processing, 24(8):2579-259, 2015.
[36] Lvmin Zhang, Anyi Rao, and Mancesh Agrawala, Adding conditional control to text-to-image diffusion models. In ICCV, 2023.
[37] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metrie. In CVPR, 2018.
[38] Shangehen Zhou, Jiawci Zhang, Jinshan Pan, Haozhe Xie, Wangmeng Zuo, and Jimmy Ren. Spatio- temporal filter adaptive network for video deblurring. In ICCV, 2019.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the contributions made in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discuss the limitations of the work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information to enable replication of the key experimental results reported.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [No]
Justification: the paper is accepted, we will make all training and testing code, as well as the datasets used, publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: cenollccggiiess...
/ /nips...
sssisssin ssisssolicyyyyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify all the training and test details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper report error bars suitably and correctly.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: answerYes
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https pettpa//nneurigee cc/ppblic/tthiccciiiieeeeeeeeeee
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: have discussed the potential societal impacts in the supplemental material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [No]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators or original owners of the assets (code, data, models) used in the paper have been given proper recognition.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The new assets introduced in the paper are well documented, and we have provided detailed descriptions.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [No]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
18

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: / /ttp///erss////// theurips.cc/ooffeeee /20e 20LM/LLM for what should or should be described.
19


Reviews:
[{"review_id": "uBOKdHYtwT", "reviewer": "Reviewer_qdjB", "summary": "This paper presents DeblurDiff, a new diffusion based model for real-world image deblurring. The key contribution is alternating diffusion of two entities: latent kernel and latent image feature. The Latent Kernel Prediction Network (LKPN) predicts spatially variant deblurring kernels within the latent space and it is trained in conjunction with a conditional diffusion model. These kernels are then applied to the latent representation of the blurry image using Element-wise Adaptive Convolution (EAC), producing a cleaner conditional input for the diffusion process. This iterative process is used to refine the kernel prediction and latent image prediction for the subsequent step. The model achieves state-of-the-art results, especially in terms of perceptual quality, across both synthetic and real-world deblurring datasets.", "questions": "Operations such as attention have become the new norm and they have essentially replaced convolution operations in image restoration literature. Did the project explore any alternatives to convolution operation?\n\nI wonder if a DRM would also benefit the proposed approach. Intuitively, adding a predeblurred image (using a DRM) to the conditioning images could further boost the accuracy of the model. Since the denoiser of ControlNet is capable of accepting multiple latents as input (in concatenated format), maybe we could also supply the latents extracted from a predeblurred image This could further improve scores since exiting methods such as PASD using such predeblurred image are already doing well across benchmarks, specially on real images (Table 1)\n\nThe paper mentions that a pre-trained FFTformer is used as the DRM in DiffBIR. Kindly clarify whether or not the FFTformer used in DiffBIR is retrained on the proposed dataset. If not, the comparison with DiffBIR would not be completely fair.\n\nKindly also address the key weaknesses mentioned above.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nUsing diffusion based Kernel Prediction in the latent domain to iteratively refine the input for the diffusion based restoration model is a clever idea. It tackles a prevalent problem of diffusion based restoration models - how you provide conditioning without losing important detail or introducing artifacts.\n\nThe perceptual quality of results shown is quite impressive. Table 1 shows clear improvement on metrics like MUSIQ and CLIP-IQA. The qualitative results reflect this very well too, by achieving a tough balance between sharpness and fidelity.\n\nThe proposed large scale dataset (if released) is also an important contribution to the future development in the deblurring community. Although it is not completely new (built upon existing datasets such as DVD, RSBlur etc.), its careful curation seems effective, as exemplified in the qualitative results.\n\nThe details and insights (specially on effect of diffusion model ablations and datasets) provided in the supplementary material are quite informative!\n\n\n\nWeaknesses:\n\nContributions are lacking in terms of novel architectural components or a more profound theoretical insight. While the overall framework is novel, the individual components are not fundamentally so. The paper presents a clever engineering solution by employing relatively known ideas (such as Deep kernel prediction and Element-wise Adaptive Convolution) in a diffusion based formulation. Pixel-adaptive Filter estimation and element wise convolution (in feature domain) for motion deblurring has been proposed and successfully used in a prior research eg. “Spatio-Temporal Filter Adaptive Network for Video Deblurring (ICCV 2019)”. Similarly, joint sharp image and blur kernel estimation (in pixel domain) through diffusion has been proposed in an existing paper: “Parallel Diffusion Models of Operator and Image for Blind Inverse Problems (CVPR 2023)”. Such literature should be included in related works and experiments/analysis. \n\n\nThe paper is primarily empirical and there is a lack of theoretical analysis of why the proposed iterative refinement process should converge or why it is superior to other potential approaches.\n\n\nThe filtered blurred images (obtained by applying the estimated filters on the blurred image latent and passing it to the decoder to obtain an image) are not shown in the paper. Since learning accurate deblurring filters is at the core of the paper, it is important to demonstrate that through visual examples. \n\n\nThe proposed approach’s parameter count and FLOPs (which are crucial for practical applications) have not been discussed or compared with state-of-the-art deblurring methods. Since the proposed method involves two diffusion processes with multiple sampling steps, it would be interesting to compare its overall speed with the best existing methods.\n\n\nThe qualitative results in Figs. do not contain results of existing methods retrained on new dataset. It is difficult to disentangle the effect of model design and new dataset. It would be helpful to know which of these changes contribute more to the significant qualitative improvements over the existing methods. \n\n\nThe proposed method’s key metric scores (MUSIQ, CLIP-IQA) are inferior to the existing approach PASD on real images (shown in Table). Does this indicate overfitting of the model on training data? \n\nThe main paper does not provide enough details and experiments on the LKPN architecture. The architecture presented in Fig. 3 only shows a couple of layers while the actual LKPN (as mentioned in the supplementary material) is a UNet with 8 encoder and decoder blocks containing a ResBlock and a self-attention layer. For reproducibility and a deeper understanding of the method, it would be beneficial to know and architectural ablations, and know more details such as specific layers, number of channels, parameters, etc.\n\n\nSince paper implementation is not provided, extensive qualitative results on key benchmarks should be provided for better assessment and future comparisons.", "was_revised": true, "final_justification": "Using diffusion based Kernel Prediction in the latent domain is an effective solution to a key problem in diffusion based deblurring methods: conditioning without losing important detail or introducing artifacts. The perceptual quality of results shown is quite impressive. The proposed large scale dataset (if released) is valuable to the future development in the deblurring community. The details and insights validate the approach! The only negative aspect is lack of fundamental novelty in the proposed modules.", "rebuttal": "**1. The novelty of this paper**\n\nWhile it is true that some underlying techniques, such as kernel prediction and adaptive convolution, have been explored in prior research, our contribution lies in the novel integration and application of these concepts within a latent diffusion framework for image restoration, leading to a distinct and effective solution.\n\nSpecifically:\n\n1. Diffusion-based Formulation: The \"Spatio-Temporal Filter Adaptive Network\" (ICCV 2019) directly estimates kernels from the blurred frame and its adjacent frames in a video sequence, leveraging temporal information. In contrast, our method is specifically designed for the latent space of pre-trained diffusion models. It does not rely on temporal context but instead estimates the degradation kernel by jointly leveraging the input latent of the blurred image and the progressively refined, denoised latent features generated by the pre-trained diffusion model during the iterative process. This integration of the diffusion model's strong generative prior fundamentally differentiates our approach from non-diffusion based methods.\n\n2. Spatially-Variant Degradation Modeling: Unlike \"Parallel Diffusion Models\" (CVPR 2023), which estimates a single, global blur kernel, our method focuses on estimating a spatially-variant degradation. This is crucial for handling real-world blurs that vary across the image. Our LKPN network predicts location-specific kernels, enabling more accurate and localized restoration.\n\n3. Iterative Refinement: We introduce an iterative mechanism where the denoised output from the diffusion model at each step provides a clean prior that actively guides the refinement of the spatially-variant degradation estimate. Non-diffusion methods (as mentioned above) perform \"static, one-time\" degradation estimation and restoration, whereas our method enables \"dynamic, iterative\" estimation and restoration. It fully leverages the diffusion model's strength in generating high-quality images. This is further validated by our ablation study in Table 2: the \"w/o SD for LKPN\" variant, which removes the iterative refinement by not using the diffusion model's features, results in significantly worse performance, confirming the critical role of our iterative design.\n\nWe agree that the related work section can be strengthened, and we will include the mentioned papers in the revised manuscript to provide a more comprehensive context. \n\n**2. Why the proposed iterative refinement process should converge**\n\nThe process is designed to leverage the inherent property of diffusion models: the restoration quality improves progressively over the denoising steps. As the diffusion model generates increasingly cleaner latent features with each step, these high-quality features serve as a strong prior for our LKPN network. This enables LKPN to estimate more accurate deblurring guidance at each subsequent step.\n\nAs detailed in Section 5 of the Supp, we show that our LKPN, combined with the EAC module, generates increasingly accurate deblurring guidance over the diffusion steps, which in turn facilitates better image restoration. This dynamic, iterative interaction is a key advantage of our approach. Figure 10(c) shows that the MANIQA scores of the intermediate results progressively improve from left to right, with values of 0.43, 0.47, 0.55, and 0.58, respectively, clearly demonstrating the quality enhancement throughout the iterative process.\n\n**3. Why DeblurDiff is superior to other potential approaches**\n\nAs discussed in L40–L50 and L121-L129 of the main paper, methods like DiffBIR rely heavily on the initial pre-deblurred result. When this initial estimate is poor (Figure 1(c)), it can lead to significant structural errors in the final output (Figure 1(g)).\n\nIn contrast, our method features an iterative refinement process, where the LKPN, guided by the denoised output of the diffusion model, progressively generates more accurate degradation estimates, leading to a clearer conditioning input and a feedback loop that produces a more distinct and artifact-free image. (Figure 1(h)).\n\n**4. Visualization of the filtered blurred images**\n\nIn Figure 2 of the main paper, we show the intermediate results of the reconstructed latent clean images at different diffusion steps. These results are obtained by applying our estimated deblurring kernels in the latent space and then decoding them, effectively serving as visualizations of the filtered blurred images. Furthermore, Figure 10 in the Supp provides additional ablation studies that include these intermediate reconstructions.\n\nWe will provide a more detailed discussion and clarification on these visual results in the revised paper.\n\n**5. Comparison of Params, FLOPs, and runtime.**\n\nWe have provided runtime information for some methods in Table 4 of the Supp. We provide comprehensive comparisons of model parameters and FLOPs in the following table.\n\n| |FFTformer|ResShift|DiffBIR|PASD|Ours|\n|-|-|-|-|-|-|\n|Params (M)|16.6|119|1717|1900|1845|\n|FLOPs (G)|131|5491|24234|29125|25526|\n|Times (s)|0.8|15|55|52|46|\n\nAs discussed in L86–L92 of the Supp, FFTformer is not a diffusion-based method, which makes it faster but results in inferior visual quality. ReShift does not use a pre-trained diffusion model, giving it some advantages in runtime over methods that do; however, its visual performance is worse than diffusion-based methods such as DiffBIR, PASD, and DeblurDiff. Among all methods that leverage pre-trained diffusion models, our approach achieves both higher fidelity (as shown by better PSNR and SSIM in Table 1) and better perceptual quality. Moreover, our method has fewer parameters and lower FLOPs than PASD, and runs faster than both DiffBIR and PASD.\n\n**6.  The qualitative comparisons lack baseline methods retrained on the new dataset**\n\nThe qualitative results (e.g., in Figures 1-2 and Figures 4-6) are consistent with Table 1, where  DiffBIR and PASD were retrained on our dataset for fair comparison. In the revised manuscript, we will add an asterisk (*) to DiffBIR and PASD in the figure labels to clarify this.\n\n**7. Overfitting of the model on training data**\n\nIn fact, as stated in line 217 of our paper, our training set includes only RSBlur and MCBlur, while the test sets consist of five distinct datasets: GoPro, DVD, RealBlur, RWBI, and Real Images, ensuring a strict train/test split and avoiding data leakage. Furthermore, both RWBI and Real Images are real-world blur benchmarks without ground truth, making overfitting to GT impossible. Notably, on the RWBI dataset, our method achieves the highest scores in MUSIQ, MANIQA, and CLIP-IQA among all compared methods. This demonstrates the strong generalization capability of our model rather than overfitting.\n\n**8. More details and experiments on the LKPN architecture.**\n\nIn the revised manuscript, we will provide a comprehensive description of the LKPN architecture. Specifically, LKPN is a U-Net with 4 stages (often referred to as 4 levels), resulting in a total of 8 encoder and decoder blocks (2 blocks per stage). Each encoder block consists of a ResBlock followed by a self-attention layer, and each decoder block consists of a ResBlock followed by a nearest-neighbor upsampling layer. After each encoder block, a downsampling layer is applied to reduce the spatial resolution before feeding the features to the next stage.\n\nThe input channel is 8, and the number of hidden channels is 128 throughout the network. The total number of parameters is 144M.\n\nDue to time constraints, we were only able to conduct ablation studies on different parameter scales of the LKPN network. The results are shown in the table below, where a larger LKPN with more parameters brings moderate performance improvement, indicating a positive correlation between model capacity and restoration quality.\n\n||NIQE|MUSIQ|MANIQA|CLIPIQA|\n|-|-|-|-|-|\n| LKPN_C64|3.9463|51.7462|0.5861|0.5416|\n| LKPN_C128|3.6628|52.9263|0.5963|0.5496|\n\nWe commit to releasing the full code and model configuration files to ensure complete reproducibility.\n\n**9. Detailed implementation of the proposed method**\n\nWe are actively preparing the code, model weights, and corresponding test results. We commit to releasing the code to ensure reproducibility and facilitate future research.\n\n**10. Replacing convolution operations with attention**\n\nOur goal is to estimate a spatially variant kernel to guide the restoration of sharp images in the latent space, effectively preserving the information from the blurry input. We have explored replacing this dynamic filtering operation with a cross-attention mechanism. However, as shown in the table below, this alternative design resulted in inferior performance. Therefore, we find that the dynamic filtering approach is more effective for this task. This remains an interesting direction for future exploration.\n\n| |NIQE|MUSIQ|MANIQA|CLIPIQA|\n|-|-|-|-|-|\n|Cross Attention|4.2351|50.3223|0.5642|0.5257|\n|EAC|3.6628|52.9263|0.5963|0.5496|\n\n**11. Adding a pre-deblurred image as extra input**\n\nWe have experimented with this approach by retraining our model to accept additional latents extracted from a pre-deblurred image (generated by a pre-trained DRM) as input, concatenated with the original conditioning features. The experimental results are shown in the table below.\n\n| |NIQE|MUSIQ|MANIQA|CLIPIQA|\n|-|-|-|-|-|\n| DeblurDiff|3.6628|52.9263|0.5963|0.5496|\n| DeblurDiff w/ DRM|3.5472|53.8815|0.5852|0.5581|\n\nAdding the DRM input leads to improved performance in NIQE, MUSIQ, and CLIP-IQA, while MANIQA slightly decreases. This indicates that incorporating the pre-deblurred image as an additional conditioning signal can enhance the model's perceptual quality to some extent.\n\n**12. Whether or not the FFTformer used in DiffBIR is retrained on the proposed dataset.**\n\nYes, the FFTformer used in DiffBIR was retrained on our proposed dataset. We will clarify this point in the revised manuscript."}, {"review_id": "TKpF5fs8qg", "reviewer": "Reviewer_Xgdm", "summary": "Directly using a blurry image or pre-deblurred one as a conditional control for Stable Diffusion will either hinder accurate structure extraction or make the results overly dependent on the deblurring network. In this work, author proposes a Latent Kernel Prediction Network (LKPN) to achieve robust real world image deblurring.", "questions": "1.The compared methods are still too outdated — DiffBIR was released in 2023. I hope the authors can compare with more recent SOTA methods to truly demonstrate the effectiveness of their approach.\n\n2.The author's method is compared with several others in Table 1. From what I see, these methods were all released in 2023. Even so, I find that the performance of the author's method is still not SOTA. I'm also confused because the author's PSNR, SSIM, and LPIPS scores are not as good as those of Hi-Diff, MUSIQ, NIQE, CLIP-IQA, or even PASD (also from 2023). So, what justifies the claim that your method is advanced?\n\n3.Please provide a comparison of all methods in Table 1 in terms of the number of parameters and inference time.\n\n4.The NIQE and MANIQA metrics used in Table 2 are no-reference quality assessment indicators, and they exhibit large fluctuations. Furthermore, they cannot accurately evaluate visual quality. Metrics like PSNR, SSIM, and LPIPS would be more appropriate here.\n\n5.The novelty of the paper is not very strong. Degradation estimation in Latent Diffusion Models (LDM) has already been explored by many image restoration methods, such as:\n\n[1] DiffIR: Efficient diffusion model for image restoration\n\n[2] DeeDSR: Towards Real-World Image Super-Resolution via Degradation-Aware Stable Diffusion\n\n[3] Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper is well written.", "was_revised": true, "final_justification": "For A1, the publication note for TPAMI 2024 indicates that PASD was submitted in 2023, while the submission note for ECCV 2024 states diffbir was released before in February 2024. In fact, even the latest version of DiffBIR doesn't show much difference from the version released in June 2023. These papers have been around for at least a year and a half. In such a rapidly developing field, model performance improves every three months, so the methods compared by the authors are too outdated, and I believe they lack any convincing power.\n2.For A2, although the authors provided some explanations, they cannot change the fact that its performance is inconsistent. In my view, this is merely an incremental improvement and enhancement.\n\n3.For A3, it seems that the methods you compared in Table 1 are not limited to the few you’ve listed. Please provide a comparison of parameters and inference time for a few other methods, such as ControlNet, DBGAN, and HI-Diff. This would allow for a better determination of the advantages of your approach.\n\n4.For A5, after hearing your explanation, I still think it is just an incremental improvement, and the performance does not show significant progress compared to those older methods.\n\nI believe the issues mentioned above significantly undermine the credibility of this paper, and it does not meet the acceptance standards. Therefore, I maintain the decision to reject it.", "rebuttal": "**1. The compared methods are outdated**\n\nThank you for your suggestion. We would like to clarify that the version of DiffBIR we compare against is not outdated. While the initial arXiv submission was in August 2023, the method has been significantly updated and was accepted to ECCV 2024. We compare with this latest and most competitive version of DiffBIR.\n\nSimilarly, PASD was also published at ECCV 2024, and for ReShift, we use the updated results from its TPAMI 2024 publication. Therefore, our comparisons are based on the most recent and up-to-date versions of these methods, ensuring a fair and current evaluation.\n\nImportantly, the final published versions of both DiffBIR and PASD (in ECCV 2024) contain significant updates and improvements compared to their initial arXiv submissions, making them strong and contemporary baselines for comparison.\n\nFurthermore, we have evaluated the \"Efficient Visual State Space Model for Image Deblurring (EVSSM)\", which was published at CVPR 2025, on the benchmarks. The results are shown in the following table. EVSSM$^*$ indicates that we retrain EVSSM on our training set.\n\nOur method also achieves better performance on no-reference metrics compared to the latest image deblurring method from CVPR 2025 on real-world images, demonstrating better visual quality.\n\n| RWBI dataset | NIQE   | MUSIQ   | MANIQA | CLIPIQA |\n|--------------|---------|---------|--------|---------|\n| EVSSM (CVPR 2025)        | 5.0161 | 46.9781 | 0.4968 | 0.3827  |\n| EVSSM$^*$     | 4.8744 | 57.1694 | 0.5611 | 0.5686  |\n| DeblurDiff   | **4.5171** | **66.7505** | **0.6260** | **0.6849**  |\n\n| Real Images dataset | NIQE   | MUSIQ   | MANIQA | CLIPIQA |\n|--------------|--------|---------|--------|---------|\n| EVSSM (CVPR 2025)        | 4.6822 | 38.5475 | 0.5362 | 0.4141  |\n| EVSSM$^*$    | 3.8132 | 51.9340 | 0.5517 | 0.5176  |\n| DeblurDiff   |  **3.6628** | **52.9263** | **0.5963** | **0.5496**  |\n\n**2. The proposed method is not the best on some metrics.**\n\nOur goal is to develop a diffusion-based framework that leverages the powerful generative prior of pre-trained models to produce high-fidelity and perceptually realistic results for image restoration. While pixel-level metrics like PSNR are important, our primary focus is on achieving a superior balance between fidelity and perceptual quality.\n\nIt is important to note that Hi-Diff is fundamentally a regression-based method built on a transformer architecture, which only incorporates features from diffusion models as guidance, rather than using a diffusion process to generate the final image. In contrast, our method is a fully diffusion-based framework.\n\nAmong all methods that leverage pre-trained diffusion models (ControlNet, DiffBIR, PASD, and DeblurDiff), our method achieves the highest PSNR, SSIM, and LPIPS scores on synthetic datasets (GoPro, DVD, RealBlur), as shown in Table 1 of the main paper, demonstrating better fidelity. This indicates that our method is more effective at preserving the information from the input image.\n\nFor no-reference metrics, Table 1 of the main paper shows that our method achieves the best MANIQA and CLIP-IQA scores on all four benchmarks: GoPro, DVD, RealBlur, and RWBI. On the RealImages dataset, our method also achieves the highest NIQE and MANIQA. \n\nWe acknowledge that on the Real Images dataset, our MUSIQ and CLIP-IQA scores are slightly lower than those of PASD. However, our overall state-of-the-art performance across the majority of metrics and datasets demonstrates the effectiveness and advancement of our approach.\n\n\n**3. Please provide a comparison of all methods in Table 1 in terms of the number of parameters and inference time.**\n\nWe have provided runtime information for some methods in Table 4 of the Supplemental Material.\n\nWe provide a more comprehensive comparison of other methods, including the number of parameters and inference time, in the following table.\n\n|           | FFTformer | ResShift | DiffBIR | PASD  | Ours  |\n|-----------|-----------|----------|---------|-------|-------|\n| Params (M) | 16.6      | 119      | 1717    | 1900  | 1845  |\n| FLOPs (G)  | 131       | 5491     | 24234   | 29125 | 25526 |\n| Times (s)  | 0.8       | 15       | 55      | 52    | 46    |\n\nAs discussed in L86–L92 of the Supplemental Material, FFTformer is not a diffusion-based method, which makes it faster but results in inferior visual quality compared to diffusion-based approaches. ReShift does not use a pre-trained diffusion model, giving it some advantages in runtime over methods that do; however, its visual performance is worse than diffusion-based methods such as DiffBIR, PASD, and DeblurDiff. Among all methods that leverage pre-trained diffusion models, our approach achieves both higher fidelity (as shown by better PSNR and SSIM in Table 1) and better perceptual quality. Moreover, our method has fewer parameters and lower FLOPs than PASD, and runs faster than both DiffBIR and PASD.\n\n**4. The NIQE and MANIQA metrics used in Table 2 are no-reference quality assessment indicators, and they exhibit large fluctuations. Furthermore, they cannot accurately evaluate visual quality. Metrics like PSNR, SSIM, and LPIPS would be more appropriate here.**\n\nSince Table 2 in the main paper evaluates results on real-world images where ground truth (GT) is unavailable, metrics like PSNR and SSIM cannot be computed. To provide a comprehensive comparison, we also report PSNR, SSIM, and LPIPS on synthetic datasets (GoPro) where GT is available, as shown in the following table.\n\n|                 | PSNR  | SSIM   | LPIPS  | FID     |\n|-----------------|-------|--------|--------|---------|\n| ControlNet      | 22.31 | 0.6547 | 0.3292 | 37.2749 |\n| w/o EAC         | 22.57 | 0.6553 | 0.2914 | 38.7419 |\n| w/o SD for LKPN | 23.14 | 0.6614 | 0.2651 | 25.5312 |\n| DeblurDiff      | **24.32** | **0.7375** | **0.2191** | **17.6948** |\n\nWhen the EAC module is removed (w/o EAC), the fidelity of DeblurDiff decreases, demonstrating that EAC effectively preserves information from the input image and contributes to improved restoration accuracy. When the features from the pre-trained diffusion model (SD) are not used in LKPN (w/o SD for LKPN), the model lacks the benefit of the iterative refinement guided by the denoised prior, making it difficult for LKPN to provide precise structural guidance. This highlights the importance of the interaction between the degradation estimator and the diffusion model.\n\n**5. The novelty of the paper is not very strong. Degradation estimation in Latent Diffusion Models (LDM) has already been explored by many image restoration methods**\n\nMethods like DiffIR operate directly in the pixel space, while DeeDSR, S3Diff, and PASD, though performing estimation in the latent space, directly estimate degradation from the corrupted input. This makes their results highly dependent on the accuracy of the initial degradation estimation.\n\nIn contrast, our method introduces two key innovations:\n\n1. Iterative Refinement with a Clean Prior: Methods like PASD, DeeDSR, and S3Diff perform a single, static degradation estimation at the beginning of the process, typically based solely on the corrupted latent input. This makes their entire restoration pipeline critically dependent on the accuracy of this initial, one-off estimate. If the initial estimation is poor, the error propagates irreversibly through the diffusion process, leading to suboptimal results.\nIn contrast, our method introduces a dynamic, iterative refinement loop. We do not estimate the degradation once, but re-estimate it at every diffusion step. Crucially, our estimation leverages not just the corrupted input, but also the progressively denoised and cleaner latent features generated by the diffusion model itself. This denoised output serves as a powerful \"clean prior,\" providing rich, high-quality structural information that guides a more accurate degradation estimate for the next step. This closed-loop feedback mechanism allows the degradation estimate to evolve and improve over time, correcting early mistakes and leading to significantly more robust and accurate results.\n\n2. Deblur Kernel-Guided Reconstruction: Existing methods employ the estimated degradation as a generic conditioning signal (e.g., via concatenation as an auxiliary input). In contrast, our approach leverages the predicted deblurring kernel in a more precise and effective manner by integrating it through an Element-wise Adaptive Convolution (EAC) module, which performs feature-adaptive modulation in latent space. This allows the model to apply spatially adaptive deblurring operations. Such a design ensures tight coupling between the restoration process and the estimated degradation, thereby enhancing reconstruction fidelity and mitigating artifact generation.\n\nThese design choices result in a more accurate estimation and higher-quality restoration, representing a meaningful advancement over prior approaches. We will cite and discuss the distinctions from these papers in the revised manuscript."}, {"review_id": "WOwNbYggXj", "reviewer": "Reviewer_34aF", "summary": "This paper presents DeblurDiff, a novel real-world image deblurring framework that integrates a Latent Kernel Prediction Network (LKPN) with pre-trained Stable Diffusion (SD) models. The LKPN predicts pixel-wise adaptive kernels to refine blurry latent representations via Element-wise Adaptive Convolution (EAC), providing improved structural guidance for the diffusion process. Extensive experiments demonstrate superior perceptual quality over state-of-the-art CNN-, Transformer-, and diffusion-based deblurring methods.", "questions": "Please refer to the weakness part.", "limitations": "Yes. The authors mention limitations regarding SD model versions and runtime efficiency.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "+ Strong design combining SD priors with an adaptive kernel estimation strategy; ablation studies (Table 2) and multiple benchmarks validate the method.\n+ The paper is generally well-written with clear motivation, figures, and architectural illustrations (Fig.3).\n+ Shows practical potential by addressing limitations of existing degradation-removal pipelines.\n\n- The improvement in reference-based metrics (PSNR, SSIM) over prior methods is modest, despite substantial complexity.\n- Limited discussion on computational overhead of iterative kernel refinement during inference.", "was_revised": true, "final_justification": "Thanks to the author's reply, I think this paper is acceptable and I maintain the score.", "rebuttal": "**1. The improvement in reference-based metrics (PSNR, SSIM) over prior methods is modest, despite substantial complexity.**\n\nCompared to the transformer-based FFTformer, our PSNR is lower, which is partly due to the inherent trade-off in diffusion models between perceptual quality and pixel-level accuracy. Our method prioritizes generating visually realistic and detailed results, especially on real-world images, over maximizing PSNR/SSIM.\n\nWhen compared to the pre-trained diffusion-based approaches (ControlNet, DiffBIR, PASD), our method achieves higher scores in both reference-based (PSNR, SSIM) and no-reference metrics. Specifically, on the GoPro, DVD, and RealBlur datasets, our method consistently outperforms ControlNet, DiffBIR, and PASD in both PSNR and SSIM (0.44dB higher in PSNR on the GoPro dataset), demonstrating better restoration fidelity. This indicates that our method is more effective at preserving the information from the input image. \n\nFurthermore, on all four benchmarks—GoPro, DVD, RealBlur, and RWBI—our method achieves the best performance in the no-reference metrics MANIQA and CLIP-IQA, indicating better perceptual quality. Our method prioritizes generating visually realistic and detailed results, especially on real-world images, over maximizing PSNR/SSIM, which aligns with the strengths of the diffusion framework.\n\n**2. Limited discussion on computational overhead of iterative kernel refinement during inference.**\n\nWe analyzed the computational overhead, including runtime and GPU memory, in L86–L92 and Table 4 of the Supplemental Material.  As shown in the following table, we provide a comprehensive comparison of other methods in terms of parameters, inference time, and FLOPs.\n\n|           | FFTformer | ResShift | DiffBIR | PASD  | Ours  |\n|-----------|-----------|----------|---------|-------|-------|\n| Params(M) | 16.6      | 119      | 1717    | 1900  | 1845  |\n| FLOPs(G)  | 131       | 5491     | 24234   | 29125 | 25526 |\n| Times(s)  | 0.8       | 15       | 55      | 52    | 46    |\n\nCompared to FFTformer, our method is slower in inference time, as FFTformer is not a diffusion-based method and thus inherently faster. ReShift, which does not use a pre-trained diffusion model, also has a runtime advantage. However, both methods achieve inferior visual quality compared to the approaches that does not use a pre-trained diffusion model.\n\nAmong all methods that leverage pre-trained diffusion models (ControlNet, DiffBIR, PASD, and ours), our method achieves a favorable trade-off between speed and performance. With an inference time of 46 seconds, our method is slightly slower than ControlNet (40 seconds) but 16.4\\% faster than DiffBIR (55 seconds). In terms of GPU memory, our method uses 7.8 GB, which is slightly higher than ControlNet (7.6 GB) but lower than DiffBIR (8.0 GB). Additionally, our method has fewer parameters and lower FLOPs than PASD.\n\nThis demonstrates that our iterative kernel refinement does not introduce significant computational overhead. More importantly, among all pre-trained diffusion-based methods, our approach achieves both higher fidelity (as shown by superior PSNR and SSIM in Table 1) and better perceptual quality. This indicates that our method delivers state-of-the-art performance without incurring excessive computational cost."}, {"review_id": "vNHMDzUgN1", "reviewer": "Reviewer_TarL", "summary": "This paper propose a novel diffusion-based image deblurring architecture that combining both supervised training and pretrained generative priors, which achieves the SOTA performance on multiple dataset with regard to the visual quality.", "questions": "1. Authors mentioned \"All the aforementioned retrained methods are trained and tested using the publicly released codes and models of the competing methods\". Did you just retrain the DiffBIR or all compared methods. If you retrained all compared methods, then why did the proposed method performs worse on synthetic data. The proposed method also used L2 supervision between ground-truth and output, same as other method.\n\n2. Some related method [https://arxiv.org/pdf/2308.02947] estimate the adaptive kernels on the original image, have you tried this idea instead of on the latent space? And advantage of estimate the kernel in the latent space?", "limitations": "1. Long inference time makes it inapplicable for real time scenarios\n2. The information loss of VAE is inevitable, which may lead to a slight blur on the final result.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The method outperforms other SOTA methods on non-reference based metrics.\n2. A novel kernel predictor to map the low-quality latent feature to high-quality, avoid the artifacts and blur on the condition.\n\nWeaknesses\n1. Lacks the quantitative evaluation of ablation study.\n2. In the Figure 13 of the supp, I think \"Latent Kernel Prediction Network\" should change a name.\n3. If all methods are retrained on the proposed dataset, then it does not make sense to me that the proposed method performs worse on PSNR and SSIM, since there is a L2 loss during the training of this method.", "was_revised": true, "final_justification": "I agree to accept this paper. for the novelty of latent kernel estimation and the solid experiments", "rebuttal": "**1. Lacks the quantitative evaluation of ablation study.**\n\nAs real blurred images lack ground-truth references, we employ no-reference metrics for quantitative evaluation. Consequently, Table 2 in the main paper already reports the ablation results with two standard no-reference indices—NIQE and MANIQA. To further strengthen the quantitative evidence, we now provide an extended set of reference-based metrics on the GoPro dataset in the table below.\n\n|                 | PSNR  | SSIM   | LPIPS  | FID     |\n|-----------------|-------|--------|--------|---------|\n| ControlNet      | 22.31 | 0.6547 | 0.3292 | 37.2749 |\n| w/o EAC         | 22.57 | 0.6553 | 0.2914 | 38.7419 |\n| w/o SD for LKPN | 23.14 | 0.6614 | 0.2651 | 25.5312 |\n| DeblurDiff      | **24.32** | **0.7375** | **0.2191** | **17.6948** |\n\nWhen the EAC module is removed (w/o EAC), the fidelity of DeblurDiff decreases, demonstrating that EAC effectively preserves information from the input image and contributes to improved restoration accuracy. When the features from the pre-trained diffusion model (SD) are not used in LKPN (w/o SD for LKPN), the model lacks the benefit of the iterative refinement guided by the denoised prior, making it difficult for LKPN to provide precise structural guidance. This highlights the importance of the interaction between the degradation estimator and the diffusion model.\n\n**2. In the Figure 13 of the supp, I think \"Latent Kernel Prediction Network\" should change a name.**\n\nThank you for this suggestion. We agree that the term “Latent Kernel Prediction Network” in the Figure 13 of the supp is inappropriate. This network should be named “Latent Image Restoration Network”. We will revise the label accordingly.\n\n**3. If all methods are retrained on the proposed dataset, then it does not make sense to me that the proposed method performs worse on PSNR and SSIM, since there is a L2 loss during the training of this method.**\n\nAs shown in Table 1 on the GoPro dataset, the PSNR of non-diffusion methods (e.g., FFTformer) is generally higher than that of diffusion-based approaches, with FFTformer outperforming all diffusion-based methods in PSNR. This is because non-diffusion models, such as CNNs or transformers, are typically optimized directly for pixel-level accuracy. In contrast, the training objective of diffusion models focuses more on generating high-quality, perceptually realistic images rather than pixel-level precise matching. Although L2 loss is included during training, the core optimization goal and the stochastic nature of the generative process may result in relatively lower performance on pixel-level metrics like PSNR and SSIM compared to methods specifically designed for these metrics. This is an inherent difference in the design objectives between generative and discriminative models.\n\n**4. Did you just retrain the DiffBIR or all compared methods. If you retrained all compared methods, then why did the proposed method performs worse on synthetic data. The proposed method also used L2 supervision between ground-truth and output, same as other method.**\n\nWe provide a comprehensive comparison of all retrained methods in Table 1 of the supplemental material\n\nThe lower PSNR/SSIM scores for our diffusion-based method, despite using L2 supervision, stem from fundamental differences:\n\n1. Optimization Focus: CNNs or Transformers (FFTformer) directly minimize the pixel-wise L2 error between the predicted output image and the ground truth, which inherently optimizes for high PSNR and SSIM. In contrast, our diffusion model is trained to predict the noise added to a latent representation at each denoising step. The L2 loss is applied between the predicted noise and the actual noise, not directly on the final reconstructed image. This indirect optimization prioritizes learning the data distribution and generating perceptually realistic details, often at the expense of perfect pixel-level accuracy. As a result, while our method produces visually superior results, its PSNR and SSIM metrics may be lower than those of CNNs that are explicitly trained to minimize pixel-level reconstruction error.\n\n2. Model Prior: Diffusion models use a powerful generative prior, which can produce more realistic textures but may sacrifice slight pixel fidelity compared to CNNs or Transformers optimized for these metrics. When compared to the pre-trained diffusion-based approaches (ControlNet, DiffBIR, PASD), our method achieves higher scores in both reference-based (PSNR, SSIM) and no-reference metrics. Specifically, on the GoPro, DVD, and RealBlur datasets, our method consistently outperforms ControlNet, DiffBIR, and PASD in both PSNR and SSIM (0.44dB higher in PSNR on the GoPro dataset), demonstrating better restoration fidelity. This indicates that our method is more effective at preserving the information from the input image.\n\n\n**5. Some related method estimate the adaptive kernels on the original image, have you tried this idea instead of on the latent space? And advantage of estimate the kernel in the latent space?**\n\nSince diffusion models operate in the latent space to significantly reduce computational cost, we design our approach to be compatible with this paradigm. Therefore, we perform kernel estimation directly in the latent space instead of on the original high-resolution image. Estimating the kernel on the original image would not only contradict this efficiency principle but also lead to dramatically higher memory and processing demands, as the resolution is 8 times larger. Additionally, it would require continuous and expensive VAE encoding/decoding during both training and inference.\n\nEstimating the kernel in the latent space is advantageous because it significantly reduces computational overhead while maintaining acceptable accuracy. The lower resolution makes the process more efficient and avoids repeated VAE operations.\n\nWe will cite and discuss the distinctions from these papers in the revised manuscript.\n\n**6. Long inference time makes it inapplicable for real time scenarios**\n\nThis is indeed a limitation of our method. As discussed in L295–L297 of the main paper, we plan to optimize the multi-step diffusion into a single-step approach to accelerate inference.\n\n**7. The information loss of VAE is inevitable, which may lead to a slight blur on the final result.**\n\nTraditional VAEs often compress an image into a single latent vector, which can lead to severe information loss and blurry reconstructions due to the reliance on pixel-wise L2 loss. In contrast, the VAEs used in pre-trained diffusion models (Stable Diffusion and FLUX) are specifically retrained and designed differently. They do not compress the image into a single vector, but rather into a spatial latent feature map of size h/8 x w/8 (where h and w are the original image dimensions), which preserves much more spatial information.\n\nFurthermore, their training process incorporates advanced loss functions, such as perceptual loss and adversarial loss, in addition to reconstruction loss. This design prioritizes perceptual quality and detail preservation, resulting in significantly clearer and more realistic reconstructions compared to traditional VAEs. While some information loss is still inherent, the VAEs in modern diffusion models are optimized to minimize this issue.\n\nWe acknowledge this limitation and believe it's an area where future research on diffusion models can focus."}]

Author rebuttal:

Meta-review: This paper introduces a latent kernel prediction strategy within Stable Diffusion for image deblurring. The submission received mixed ratings (3×Accept and 1×Weak Reject). After the rebuttal stage, most of the concerns raised by Reviewers qdjB, 34aF, and TarL have been largely addressed. However, Reviewer Xgdm maintained strong concerns regarding the evaluation validity and the incremental performance.

After carefully reading the paper, reviews, and all discussions, the AC shares concerns about the originality of the key components and the performance, as also highlighted by Reviewers qdjB and Xgdm:

- On EAC originality (qdjB): The proposed Element-wise Adaptive Convolution (EAC) layer appears nearly identical to the Filter Adaptive Convolutional (FAC) layer from *“Spatio-Temporal Filter Adaptive Network for Video Deblurring (ICCV 2019)”*. Upon examination, the AC observed that the EAC figure (Fig. 3) is essentially the same as FAC (Fig. 3 in that paper), and the equation and description in supplementary (Lines 49–53) almost replicate FAC’s formulation (Eq. 1 and its explanation). Yet, the submission does not cite or discuss this prior work. Such an omission raises concerns and may mislead reviewers into overestimating the novelty of the contribution, and therefore EAC cannot be considered a new component of this paper. Moreover, the ablation study in the authors’ response (Q4 by Xgdm) indicates that performance heavily depends on EAC, removing it (“w/o EAC”) reduces PSNR from 24.32 to 22.57. This further raises concerns about the true contribution and originality of the method. 

- In addition, Reviewer Xgdm questioned that degradation estimation has already been widely studied in image restoration. The authors argue their distinction lies in “iterative refinement.” The AC agrees this is an interesting and reasonable design to introduce into the diffusion framework, though it is not a fundamentally new concept and has been explored in earlier works such as *“Blind Super-Resolution With Iterative Kernel Correction (CVPR 2019)”*.

- On performance and efficiency (Xgdm): The most serious concern raised by Reviewer Xgdm is that the performance improvements are incremental and do not demonstrate clear progress over the state of the art. The AC also share this concern, competing methods such as Hi-Diff and ResShift surpass the proposed approach on multiple metrics (Table 1), while being significantly more efficient in terms of parameters, FLOPs, and inference time. For instance, according to the authors’ response, Hi-Diff is ~50× faster with only 1/200 FLOPs, yet still outperforms the proposed method on several benchmarks.

In summary, this paper received three positive ratings and introduces an interesting integration of iterative kernel refinement into a diffusion framework for image deblurring. At the same time, the AC notes that the novelty may be overestimated due to the omission of prior work, particularly with the key component EAC showing strong similarity to the earlier FAC design but without proper discussion or acknowledgment. While the quantitative improvements over recent state-of-the-art methods are not always substantial, the approach provides clear gains in visual quality, which is an important strength for practical deblurring applications. For the final version, the AC encourages the authors to explicitly cite and clarify the relation to prior works (e.g., FAC for EAC) and to more clearly position their contributions.
Decision: Accept (poster)