Title: Cross-Scale Self-Supervised Blind Image Deblurring via Implicit Neural Representation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Blind Image Deblurring

Self-Supervised Learning

Implicit Neural Representation
Abstract: Blind image deblurring (BID) is an important yet challenging image recovery problem. Most existing deep learning methods require supervised training with ground truth (GT) images. This paper introduces a self-supervised method for BID that does not require GT images. The key challenge is to regularize the training to prevent over-fitting due to the absence of GT images. By leveraging an exact relationship among the blurred image, latent image, and blur kernel across consecutive scales, we propose an effective cross-scale consistency loss. This is implemented by representing the image and kernel with implicit neural representations (INRs), whose resolution-free property enables consistent yet efficient computation for network training across multiple scales. Combined with a progressively coarse-to-fine training scheme, the proposed method significantly outperforms existing self-supervised methods  in extensive experiments.

Cross-Scale Self-Supervised Blind Image Deblurring via Implicit Neural Representation

Abstract

Blind image deblurring (BID) is an important yet challenging image recovery problem. Most existing deep learning methods require supervised training with ground truth (GT) images. This paper introduces a self-supervised method for BID that does not require GT images. The key challenge is to regularize the training to prevent over-fitting due to the absence of GT images. By leveraging an exact relationship among the blurred image, latent image, and blur kernel across consecutive scales, we propose an effective cross-scale consistency loss. This is implemented by representing the image and kernel with implicit neural representations (INRs), whose resolution-free property enables consistent yet efficient computation for network training across multiple scales. Combined with a progressively coarse-to-fine training scheme, the proposed method significantly outperforms existing self-supervised methods in extensive experiments.

1 Introduction

Uniform blurring, a degradation commonly encountered in optics, leads to the loss of important details within a captured image. Uniform blurring usually can be described as the convolution:

where ⊗ denotes the 2D discrete convolution operation, ( y , x ) is the pair of blurred and sharp (latent) images. k represents the blur kernel responsible for degradation, and n denotes measurement noise. For example, in digital photography, motion blur is caused by camera shake during exposure. When scene depth variation is small and camera movement is mainly translational within the image plane, motion blur can be approximated by the convolution model (1), with k representing camera motion. In fluorescence microscopy, specimens stained with fluorescent dyes and exposed to specific wavelength light often exhibit blurring due to light diffraction and optical path imperfections, such as optical component misalignment or lens aberrations. These factors obscure critical details, like micro-tubule arrangements in cells. Such blurring can also be modeled by (1). In either case, both the latent image x and blur kernel k are unknown and must be estimated from the blurred image y .

BID aims at estimating ( k , x ) , the pair of latent image and blur kernel, from the degraded image y . BID is a challenging non-linear inverse problem with many plausible solutions due to its inherent solution ambiguity. This ambiguity stems from the fact that the kernel can be decomposed into k = k 1 ⊗ k 2 , suggesting that the pair ( k 1 , k 2 ⊗ x ) are also viable solutions since y = ( k 1 ⊗ k 2 ) ⊗ x = k 1 ⊗ ( k 2 ⊗ x ) . One such example is the pair ( δ , y = k , x ) , where δ denotes the delta kernel, which gives a trivial solution saying a blurred image is the convolution of a blurred image and a delta kernel. This ambiguity indicates the ill-posed-ness of the BID problem.

1.1 Discussion on existing approaches for BID

The existing deep learning methods for BID can be roughly classified into the following categories: 38th Conference on Neural Information Processing Systems (NeurIPS 2024).

Supervised deep learning for BID with paired data : Supervised learning methods ( e.g. , [50, 78, 68, 34, 69]) trains a neural network (NN) using paired data of blurred images (or blur kernels) and GT images, to predict the latent image or/and blur kernel from an input blurred image. These methods mainly depend on a large amount of paired data for training, limiting their application to specific scenarios where such paired data is challenging to collect.

Supervised deep learning for BID with un-paired data : Some works ( e.g. [36, 61]) use generative adversarial networks (GANs) to train on unpaired blurred and latent images. Although effective for domain-specific images like faces and text, GANs perform poorly on general natural images due to domain shift. Additionally, GAN-based methods often suffer from mode collapse and training instability, resulting in suboptimal outcomes.

BID with pre-trained generative model : These methods employ pre-trained generative models, such as diffusion models [13], which are well-suited for processing images with structural consistency features like human faces. However, their reliance on pre-trained models to generate outputs that conform to the learned image distribution can lead to inauthentic results. This is particularly problematic in fields that require high fidelity, such as medical imaging and microscopy.

Self-supervised learning for BID : To circumvent the challenges of data collection and mitigate potential biases and inauthentic outcomes from generative models, a growing body of research ( e.g. , [1, 47, 32, 33, 9, 18, 77]) focuses on developing self-supervised deep learning approaches for BID that do not require training dataset or pre-trained model.

In this paper, we focus on self-supervised BID, which is challenging due to the lack of GT images but offers many practical benefits. Most existing works on self-supervised BID are based on the deep image prior (DIP) [57] of convolutional NNs (CNNs), which introduces the implicit prior from CNN for preferring structured patterns over random noise during training. The DIP-based self-supervised BID methods typically use two NN-based generators, G k and G x , to re-parameterize the blur kernel k and the latent sharp image x . The generators are then trained to maximize the likelihood of the blurred image y , minimizing the following self-supervised reconstruction loss:

For instance, SelfDeblur [47] employed a CNN for G x and a Multi-Layer Perceptron (MLP) for G k . MCEM[33] used U-Net for both G x and G k , and Zhuang et al. [77] used INR for both G x and G k .

Despite lacking access to GT images, these self-supervised BID methods perform competitively against supervised or pre-trained model based approaches, particularly with severely blurred images. However, their performance is less impressive on modestly blurred images and on real-world images. There is practical need for further studies to improve the performance of self-supervised BID across various blurring degrees and real-world images from different optics systems.

1.2 Main Idea and Contributions

Without GT images, self-supervised BID methods must address two vital questions for NN training:

Without accessing GT images, how to formulate a self-supervised loss to teach the NN-based generators to accurately predict the latent image and kernel from only the blurred image?

The non-linear structure of BID makes training NN-based generators challenging. How can we efficiently train them to ensure accurate convergence to the latent images and kernels?

Our answer to Question 1 is a cross-scale loss function that leverages the cross-scale consistency of the estimates from consecutive scales for regularization. Our answer to Question 2 is a progressive cross-scale training scheme to training the NNs, enhancing training efficiency and ensuring the convergence to GT image/kernel.

Resolution-free INR for effective cross-scale interaction in BID: Our proposed method is built on re-parametrizatrion of both latent images and kernels by INR [53, 51]. INR represents signals as continuous functions rather than discrete valuel arrays, which in our case is expressed as follows,

where ( i, j ) ∈ R 2 denotes the spatial co-ordinates of images/kernels, and Θ k , Θ x denotes the weights of NNs Φ k , Φ x for the kernel and image, respectively.

Using INR for representing images/kernels is due to its inherent resolution-free property. By enabling the model to generate the prediction with higher/lower resolutions from the same learned model, INR facilitates seamless multi-scale processing and cross-scale interaction. In contrast, DIP-based NNs that directly maps noise to an image handle cross-scale interaction clumsily, requiring manual re-scaling or interpolation, which likely introduces artifacts in the prediction.

Self-supervised cross-scale loss for BID: Without GT images, the sole constraint we have for training is y = k ⊗ x . Additional priors for regularization are necessary to alleviate over-fitting. Historically, the down-sampled version of y , denoted as y ↓ s for scale s , has often been used to initiate the blur kernel estimate. However, it is important to note that the blurred image after down-sampling cannot be modeled by a convolution between the down-sampled image and a blur kernel:

̸

In this paper, we present a cross-scale constraint that accurately characterizes the connection between ( y , x , k ) at different scales. For instance, when s = 2 , the cross-scale constraint is

where { g d } 3 d =1 denote three quadrature mirror filters (QMFs) [59] of the kernel k . This constraint can be easily implemented with INR-based resolution-free generators, and helps regularize the training of two generators to prevent likely over-fitting caused by the lack of GT images in loss function.

Progressive cross-scale learning for BID : A well-established practice in traditional alternating iterative methods of BID to avoid convergence to trivial solutions involves solving the problem in a coarse-to-fine manner [67, 71]. This means the kernel is initially estimated for the blurred image at a coarse scale, followed by propagation of the kernel estimate to finer scales. This approach has proven effective in preventing iterations from converging to trivial solutions.

Despite its effectiveness, the coarse-to-fine strategy remains under-exploited in existing selfsupervised BID methods, likely due to the resolution-fixed limitations of CNN or MLP based re-parametrization. Utilizing INR's resolution-free properties, we introduce a progressive learning strategy that begins training INR-based generators at a coarser scale and then refines training at finer scales. This scheme effectively addresses over-fitting and ensures convergence to the truth.

Main contribution: Our main contributions are summarized as follows:

Leveraging the resolution-independent properties of INR for latent images/kernels, we propose a self-supervised cross-scale loss for training the NN without requiring GT images.

We introduce a progressive multi-scale learning approach for BID, specifically designed to mitigate potential over-fitting due to the non-linear nature of the BID problem.

Extensive experiments conducted on a variety of datasets reveals that our proposed method outperforms existing self-supervised BID techniques as well as the supervised alternatives.

2 Related Works

Traditional non-learning methods for BID: Before deep learning became prevalent, regularization methods were the prominent approach for BID. These methods resolve solution ambiguities by imposing pre-defined priors on images and blur kernels, such as image gradient sparsity [8, 4, 24, 5, 67], image patch recurrence [54, 38], and Laplace priors in dark channels [42, 70]. These methods are based on some iteration scheme, and edge selection is an effective technique for better robustness and stability of the iteration [12, 65, 41, 15, 71]. Since regularization methods can be recast as Maximum-A-Posteriori (MAP) estimators in Bayesian inference, another class of BID methods is derived from variational Bayesian estimators [35, 14, 28, 63, 2], as well as variational expectation maximization [29, 71]. The success of these methods requires rigorous tuning of hyper-parameters related to priors. In contrast, deep learning based methods can automatically learn priors from data.

Supervised deep learning for BID: In recent years, there has been rapid progress in supervised deep learning methods for BID. By training over many pairs of blurred/truth images, these methods either explicitly estimate the blur kernel ( e.g. , [50, 7, 40, 34] or only estimate latent images ( e.g. ,

[66, 56, 25, 75, 20, 6, 11, 73, 72]). The former is more efficient for handling uniform blurring. The latter is more general and can handle non-uniform blurring.

Deep learning for BID with unpaired data or pre-trained model: There are also methods that are trained on unpaired dataset. Lu et al. [36] train the GAN for domain-specific deblurring. Wen et al. [61] propose a structure-aware deblurring method, and Chung et al. utilize generative priors from diffusion models to jointly estimate the blur kernel and latent images [13]. Nevertheless, these methods still require GT images in unpaired data for training or depend on pre-trained models, which can be challenging and present difficulties in adaptation. In contrast, our method goes through a self-supervised manner, addressing this limitation.

Self-supervised deep learning for BID: These methods address the issues of data collection and dataset bias in supervised methods. Built on the DIP prior for image/kernel, Ren et al. [47] introduced SelfDeblur, leveraging two NN-based generators trained through a loss with optional TV regularization. The ensemble NN [9] aggregates deblurring outcomes from multiple NNs to improve performance. Li et al. [32] proposed using Monte-Carlo methods to sample NN weights as an approximation of the MAP estimator of images and kernels. Li et al. [33] presented a self-supervised training scheme derived from the EM method. Dong et al. [18] combined the implicit prior from NN architecture and hand-crafted prior to regularize the NN training. Zhuang et al. [77] re-parametrized images and kernels by INR, and reply on the implicit prior induced by INR and early stopping for regularization. Built on resolution-free INR, we propose a cross-scale self-supervised loss function and an efficient coarse-to-fine training scheme for self-supervised BID.

Most self-supervised BID methods are limited to handling uniform blurring, with Li et al. [33] being the only one capable of addressing both uniform and non-uniform blurring caused camera shake. Additionally, self-supervised super-resolution methods[3, 55] also involve the estimation of blur kernel. While both estimate the blur kernel. However, these methods differ in their input: low-resolution images versus high-resolution images. Moreover, the blur degree in BID usually is much more severe than in super-resolution.

Coarse-to-fine estimation for BID: Coarse-to-fine schemes, which gradually refine the at different scales, have proven effective in traditional alternating iterative methods [65, 12, 52]. The traditional multi-scale methods typically use estimates from coarser scales solely as initial estimates for finer scales. In contrast, our approach for self-supervised deep BID leverages the cross-scale interaction of estimates with exact relations, rather than approximations. Coarse-to-fine scheme used in recent supervised NN-based methods for BID ( e.g., [39, 56, 74, 11, 73]) train the NN such that the estimates fit well the input image at different scales. Our multi-scale scheme is more for introducing a new scale consistency loss function specifically tailored for self-supervised BID. This allows for more precise and consistent refinement of the estimations across scales.

INR for image recovery: In the domain of images, INR [10, 37, 43] encodes images as NN weights, mapping coordinates to pixel values for a compact, continuous representation. There are also multiscale extensions [27, 49] of INR for more efficient image compression. It has been used for image restoration tasks such as image in-painting, denoising, and super-resolution [51, 21, 64]. Besides images, INR also has been used for encoding defocus blur kernels [46] and motion blur kernels [77]. Our work is the first to exploit the resolution-free properties of INR for BID, leveraging these properties for a multi-scale approach and cross-scale interaction.

3 Methodology

In this paper, we propose an INR-based progressive cross-scale self-supervised BID method. We first introduce the INR-based modeling for latent images and blur kernels, followed by the formulation of the self-supervised cross-scale loss function and the progressive learning strategy.

Double-INR model for BID: In our approach, the blur kernel k and the latent image x are reparameterized by two INR models, Φ k and Φ x , respectively. Each model maps a spatial coordinate [ i, j ] to a pixel value. Let I k , I x ⊂ Z 2 denote the sets of spatial coordinates within the feasible domain for the blur kernel k and the latent image x , respectively. Then, they can be expressed as

where Θ k , Θ x denote the NN weights of k , x . The INR-based re-parameterization (4) allows for the generation of kernel and image at any coordinates, providing a representation at arbitrary scales.

INR-based multi-scale representation: Let ↓ m denotes the standard down-sampling operator:

Then, we can form both the kernel and the image in a dyadic pyramid, from the original scale and to coarser scales: Let k (0) = k , x (0) = x . Then define

For any co-ordinate set I , we define its co-ordinate set at scale s as I ( s ) = { [ i, j ] : [2 s i, 2 s j ] ∈ I } . Then, using INR-based model (4), k ( s ) and x ( s ) at scale s can be expressed as

Such a multi-scale representation can facilitate the training of the INR-based generators for BID, as it allows for the generation of image details at arbitrary scales.

3.1 NN architecture of INR-based generators for kernel/image

Kernel generator: The kernel generator Φ k in our approach is a three-layer MLP with 128 feature nodes that takes coordinates normalized to [ -1 , 1] as input and adopts sinusoidal activation functions in every layer, following the Sinusoidal Representation Networks (SIRENs) [53]. The output layer employs a Softmax activation function to ensure the prediction satisfies the two physical constraints:

Then, a Kernel Centering layer is applied to address possible positional shifts in estimated kernels for better training stability. For a kernel ˜ k ∈ R H × W satisfying (7), calculating its centroids ( c x , c y ) by

The kernel ˜ k is then shifted by [ ⌊ H/ 2 ⌋ -c y , ⌊ W/ 2 ⌋ -c x ] to ensure it is centered geometrically.

Image generator: The image generator Φ x adopts a U-Net structure with five blocks, separated by down- or up-sampling layers and connected by skip connections. Each block integrates a sequence of Convolution, Batch Normalization, and ReLU. The NN concludes with a 1 × 1 convolution layer followed by a Sigmoid layer to ensure the output image values remain within [0 , 1] . To efficiently generate high-frequencies of images, following [51], the input spatial coordinates are first transformed into a higher dimensional space using a high-frequency function γ ( · ) (sinusoidal):

where p := ( i, j ) represents the normalized coordinate values within [ -1 , 1] , and L is an positive integer. Note this encoding operation is used only in Φ x , not in Φ k .

3.2 Self-Supervised scale consistency loss

Without GT images, the only readily available loss function to train the generators is the fitting loss:

where M f ( · ) is some distance metric. Such a fitting loss clearly is not sufficient to resolve solution ambiguities in BID. To address this, we introduce a scale consistency loss that regularize the training by enforcing cross-scale consistency of the estimation, which is based on the following proposition.

Proposition 1. For a kernel (filter) k , let g 1 , g 2 , g 3 denote its associated QMF filters [59] defined by

for any [ m,n ] ∈ I k . Then, we have the following relation between consecutive two dyadic scales:

As seen from Proposition 1, the down-sampled blurred image y ↓ 2 does not equal to the convolution of the down-sampled latent image x ↓ 2 and the down-sampled kernel k ↓ 2 , incurring additional term ∑ 3 d =1 ( x ⊗ g d ) ↓ 2 . Therefore, down-sampled blurred images are fine for some initial estimation, not for regularizing the NN to obtain accurate estimation. To address this, based on Proposition 1, we introduce a scale consistency loss across two consecutive scales: for each scale s ,

where { g ( s ) d } 3 d =1 denotes the QMF filter bank of the kernel k ( s ) defined by (10), and by (6),

The scale-consistency loss (13)-(14) enforces cross-scale consistency of the estimations at different scales, providing additional regularization for training two INR-based generators.

3.3 Progressively coarse-to-fine training for BID

To address the non-linear nature of BID and avoid convergence to trivial solutions, we introduce a progressive learning strategy that trains the INR-based generators at multiple scales. The training process consists of three stages at different scales. Define the fitting terms at different scale s by

The first stage serves as the initialization, operating at the coarsest scale with the fitting loss L ( S 0 ) fit . The second stage progressive refines the training from the scale S 0 to 0 with both the fitting loss and cross-scale consistency loss. Specifically, at scale s , the loss is L ( s ) fit + λ L ( s ) cross where λ is a weight and L ( s ) cross is the cross-scale consistency loss defined by (13)-(14). The third stage is the final tuning stage at scale 0 with L (0) fit only. The training process is summarized in Algorithm 1. IIn our implementation,

Algorithm 1: Self-supervised progressively coarse-to-fine training for BID

input

: a blurred image y

output : an estimated kernel k ∗ and latent image x ∗

1: Initializing two generator NNs Φ k , Φ x with random weights Θ k and Θ x ;

2: Initial training : at the scale S 0 , training the NNs with only the fitting loss L ( s ) fit ;

3: %%Progressively training the NNs

4: for s ← S 0 to 0 do

Training the NNs at the scale s with the loss L ( s ) fit + λ L ( s ) cross ;

5: Final tuning : training the NNs at scale 0 with only the fitting loss L (0) fit ;

Structural Similarity Index Measure (SSIM) [60] is used for M f ( · ) in (15):

For M c ( · ) in (13), note that convolution process in (13) can be efficiently computed by by transforming the convolution operation into pointwise multiplication in its discrete Fourier transform (DFT), denoted by F ( · ) . Thus, we define M c ( · ) in frequency domain with ℓ 1 -norm:

4 Experiments

Implementation details: The training consists of 5000 iterations across three stages. The first stage operates at the coarsest scale S 0 with 500 iterations. The second stage refines training from scale S 0 to scale 0, with 500 iterations per scale. The final stage is tuning at scale 0 for the remaining iterations. The NN is trained using the Adam optimizer with a batch size of 1. The initial learning rates for the image and kernel generators are set to 5 × 10 -3 and 5 × 10 -5 , respectively, decreasing to half their values every 2000 iterations. The weight λ in the loss function is set to 0.001 to keep the values of the two loss terms L cross and L fit in the same order. For comparison, we use results from the literature when available; otherwise, we use pre-trained models or train from the provided code to achieve optimal performance. The code of the proposed method is available on Github. 1 .

4.1 Evalution of motion deblurring on synthesized datasets

Two metrics, PSNR (peak signal-to-noise ratio) and SSIM, are used for performance evaluation. Following SelfDeblur [47], we compute PSNR/SSIM after finding the best shift between GT and the result to handle shift ambiguity. In the tables, Bold in blue indicates the best among all supervised methods, Bold in black indicates the best, and underline the second-best among all GT-free methods.

Synthetic dataset with uniform blurring from Lai et al. [26]: This dataset consists of 100 images categorized into five groups: Manmade, Natural, People, Saturated, and Text, and covers 4 different kernels whose size ranges from 31 × 31 to 75 × 75 . For this dataset, S 0 is set to 2 . As the focus of BID is on accurately estimating the blur kernel. Thus, a two-stage evaluation protocol, as outlined by [47], is also used in our test. Note that most supervised methods only generate clear images without kernels, and thus their evaluation is solely on their output images. The competing methods include 7 non-learning methods, 5 supervised methods, and 4 self-supervised methods. For all

supervised methods, except for [20], we use their models pre-trained on the GoPro dataset [39] with both uniform and non-uniform blurring. For two recent methods, MPRNet [73] and Restormer [72], we also include their models retrained on the BSD-D dataset [48] with only uniform blurring.

The comparison in Tab. 1 shows that our method achieved the best performance. Note that for two supervised methods, MPRNet [73] and Restormer [72], their models trained on a dataset with only uniform blur performs better on testing data with uniform blur, when compared to the ones trained on the dataset with non-uniform blurring. However, this improvement is still not enough to match the performance of our method. The main reason is that existing supervised methods target general blurring and overlook the physics prior of image formation, specifically the convolution model for uniform blurring. As a result, they underperform compared to our self-supervised method, which leverages this prior. This highlights the generalization issues inherent in supervised learning approaches, whose performance heavily depends the correlation degree between the training and testing data. Refer to Appendix F.1 for visual comparisons of different methods.

Synthetic dataset with modest non-uniform blurring from Köhler et al. [22]: The benchmark dataset Köhler [22] comprises 48 motion-blurred images which blurring is not exactly uniform. This dataset is for evaluating the robustness of BID method to handle modest non-uniform blurring. Totally 17 methods are selected for comparison, and S 0 is also set to 2 . Following the evaluation protocol in [33], we use the average PSNR and MSSIM (multi-scale SSIM) as the evaluation metrics. Tab. 2 shows that our method outperforms all others in terms of both PSNR and MSSIM, indicating its robustness in handling modest non-uniform blur. Refer to Appendix F.2 for visual comparisons.

4.2 Evaluation of motion deblurring on real-world datasets

Real-world dataset from Lai et al. 's [26]: Lai et al. 's real-world dataset [26] consists of 100 real blurred images captured in diverse scenarios using various capturing settings. As no GT images for quantitative evaluation, we present only visual comparisons of many samples. Please refer to Appendix F.3 for visual comparisons of different methods. Overall, our method generates the images with the best visual quality, consistent with its performance on the synthetic datasets.

Real-world dataset RealBlur [48] with small/modest blurring: Our proposed method is also evaluated in another real-world dataset published in [48]: RealBlur-J and RealBlur-R, both containing

980 real-world blurry images. See Tab. 3 for the results. It can be seen that, unlike the experiments on synthetic datasets, there is a gap between the proposed method and supervised methods. This is understandable, as our method cannot access GT images, whereas the supervised methods can. However, our method still outperforms all traditional methods and existing self-supervised methods. Refer to Appendix F.4 for visual comparisons of different methods.

4.3 Evaluation on microscopic deconvolution

In microscopic imaging, acquired images often suffer from blur due to optical limitations, outof-focus elements, specimen motion, and the diffraction limit of light. Following [45], the test dataset consists of 120 images, covering 24 images from the test subsets "Confocal_BPAE_B" and "TwoPhoton_MICE", and includes 3 Gaussian point spread functions (PSFs) and 2 Poisson PSFs as the blur kernels. For the compared supervised methods, Restomer [72] and INIKNet [46], we retrained their models using the microscopic dataset [45]. It can be seen from Tab. 4 that our method outperforms all other self-supervised methods and is comparable to the supervised methods trained on the microscopic dataset. Refer to Appendix F.5 for visual comparisons of different methods.

4.4 Ablation study

The ablation study is conducted on the Lai et al. dataset [26]. The results are shown in Tab. 5

Effectiveness of self-supervised cross-scale consistency loss: To evaluate the gain from the proposed cross-scale consistency loss functions, we retrain the NN using only the fitting loss L fit for each scale, i.e. , without (w/o) L cross. Tab. 5 shows an average gain of about 0.72 dB in PSNR with the cross-scale consistency loss, demonstrating its effectiveness.

Effective of progressive coarse-to-fine training: We first examine the effectiveness of multi-scale training by performing the training only at the original scale. Tab. 5 shows about a 1 dB loss in PSNR, indicating multi-scale training's contribution. To evaluate the progressive training strategy, we train the NN with the sum of all loss functions at three scales, which resulting in a 3dB loss. This clearly indicates that progressive training is critical for effectively utilizing the multi-scale scheme. The reason is that the loss function at the coarser scale emphasizes lower frequencies since the image at coarser scale retains low but loses high frequencies. Thus, a NN trained on the sum of loss functions across 3 scales focuses more on low frequencies than one trained only at the finest scale, which fits both low and high frequencies. Therefore, a scale-progressive training scheme is more effective than a joint multi-scale loss. In scale-progressive training, the estimation at the coarser scale provides

a strong initialization for the finer scales. The final result is obtained by applying the fitting term exclusively at the finest scale.

INR (coordinate NN) vs. MLP/CNN (image-to-image NN) This ablation study is to evaluate the necessity of using INR for re-parametrizatrion of image and/or kernel, instead of using the NN that maps an image to an image. In the study, we separately replace our INR-based kernel/image generator with MLP/CNN based representation adopted in [47], whose down-sampled versions of images/kernels at different scales are generated by standard down-sampling process.

Direct down-sampling vs. using multi-scale grid in INR : To verify the benefit of resolution-free property of INR, we also consider generating multi-scale versions of image/kernel by standard down-sampling using bilinear interpolation, on their high resolution version from INPs, same to a CNN-based representation. This is referred to as 'Multi-scale by direct ↓ '. Tab. 5 shows that the performance of multi-scale by standard down-sampling process is very poor. Clearly multi-scale representation by standard downsampling process is not as effective as resolution-free INR.

4.5 Comparison of computational effciency

The computational effciency of the propoposed method is compared to three most related selfsupervised BID methods. The results are reported in terms of running time, number of parameters, and memory usage, when processing a 256 × 256 image with a 31 × 31 blur kernel on an NVIDIA 3090 RTX GPU. See Table 6 for the details. It can be seen that the proposed method achieves a balance between computational cost and deblurring performance.

4.6 More details, studies, experiments and visual comparisons in the appendix

The appendix includes (1) ablation study on setting scale S 0 , (2) details on hyper-parameter settings, (3) details on NN architecture, (4) more comparison on running time; (5) visualization of some sample results, (6) visualization of intermediate results, and (7) experiments on Levin et al. 's dataset [28].

5 Discussion and Conclusion

This paper introduces a self-supervised method for BID, which does not require GT images. Leveraging on the resolution-free representation of INR for image/kernel, we propose a self-supervised cross-scale consistency loss function and a progressive coarse-to-fine training strategy. The proposed method significantly outperforms existing methods in extensive experiments.

There are two limitations of the proposed self-supervised method. The first is the computational cost for processing a large number of images, as the method requires training the model for each individual sample. A potential solution is to explore the usage of the proposed techniques in meta-learning or testing-time adaptation. This would allow the proposed technique to rapidly adapt a pre-trained model instead of to train the NN from scratch. The second limitation is that the proposed method is only applicable to handle uniform blurring, as it relies on the convolution model. Extending this approach to handle non-uniform blur will be another important direction for future research.

Acknowledgments

Yuhui Quan would like to acknowledge the support from National Natural Science Foundation of China under Grant 62372186, Natural Science Foundation of Guangdong Province under Grants 2022A1515011755 and 2023A1515012841, and Fundamental Research Funds for the Central Universities under Grant x2jsD2230220. Hui Ji would like to acknowledge the support from Singapore MOE Academic Research Fund (AcRF) Tier 1 Research Grant A-8000981-00-00.

References

Muhammad Asim, Fahad Shamshad, and Ali Ahmed. Blind image deconvolution using deep generative priors. IEEE Transactions on Computational Imaging , 6:1493-1506, 2020.

S Derin Babacan, Rafael Molina, Minh N Do, and Aggelos K Katsaggelos. Bayesian blind deconvolution with general sparse image priors. In Proceedings of the European Conference on Computer Vision , pages 341-355, 2012.

Sefi Bell-Kligler, Assaf Shocher, and Michal Irani. Blind super-resolution kernel estimation using an internal-gan. Advances in Neural Information Processing Systems , 32, 2019.

Jian-Feng Cai, Hui Ji, Chaoqiang Liu, and Zuowei Shen. Blind motion deblurring from a single image using sparse approximation. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 104-111, 2009.

Jian-Feng Cai, Hui Ji, Chaoqiang Liu, and Zuowei Shen. Framelet-based blind motion deblurring from a single image. IEEE Transactions on image Processing , 21(2):562-572, 2011.

Jianrui Cai, Wangmeng Zuo, and Lei Zhang. Dark and bright channel prior embedded network for dynamic scene deblurring. IEEE Transactions on Image Processing , 29:6885-6897, 2020.

Ayan Chakrabarti. A neural approach to blind motion deblurring. In Proceedings of the European Conference on Computer Vision , pages 221-235, 2016.

Tony F Chan and Chiu-Kwong Wong. Total variation blind deconvolution. IEEE Transactions on Image Processing , 7(3):370-375, 1998.

Mingqin Chen, Yuhui Quan, Yong Xu, and Hui Ji. Self-supervised blind image deconvolution via deep generative ensemble learning. IEEE Transactions on Circuits and Systems for Video Technology , 33:634647, 2023.

Zhiqin Chen and Hao Zhang. IM-Net: Learning implicit fields for generative shape modeling. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5939-5948, 2019.

Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. Rethinking coarse-to-fine approach in single image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4641-4650, 2021.

Sunghyun Cho and Seungyong Lee. Fast motion deblurring. ACM Transactions on Graphics , 28(5):145, 2009.

Hyungjin Chung, Jeongsol Kim, Sehui Kim, and Jong Chul Ye. Parallel diffusion models of operator and image for blind inverse problems. IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023.

Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T Roweis, and William T Freeman. Removing camera shake from a single photograph. ACM Transactions on Graphics , 25(3):787-794, 2006.

Dong Gong, Mingkui Tan, Yanning Zhang, Anton Van den Hengel, and Qinfeng Shi. Blind image deconvolution by automatic gradient activation. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1827-1836, 2016.

Michael Hirsch, Christian J Schuler, Stefan Harmeling, and Bernhard Schölkopf. Fast removal of nonuniform camera shake. In Proceedings of the European Conference on Computer Vision , pages 463-470, 2011.

Zhe Hu, Sunghyun Cho, Jue Wang, and Ming-Hsuan Yang. Deblurring low-light images with light streaks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3382-3389, 2014.

Dong Huo, Abbas Masoumzadeh, Rafsanjany Kushol, and Yee-Hong Yang. Blind image deconvolution using variational deep image prior. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(10):11472-11483, 2023.

Meiguang Jin, Stefan Roth, and Paolo Favaro. Normalized blind deconvolution. In Proceedings of the European Conference on Computer Vision , pages 668-684, 2018.

Adam Kaufman and Raanan Fattal. Deblurring using analysis-synthesis networks pair. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5811-5820, 2020.

Chaewon Kim, Jaeho Lee, and Jinwoo Shin. Zero-shot blind image denoising via implicit neural representations. arXiv preprint arXiv:2204.02405 , 2022.

Rolf Köhler, Michael Hirsch, Betty Mohler, Bernhard Schölkopf, and Stefan Harmeling. Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database. In Proceedings of the European Conference on Computer Vision , pages 27-40, 2012.

Dilip Krishnan and Rob Fergus. Fast image deconvolution using hyper-laplacian priors. In Advances in Neural Information Processing Systems , pages 1033-1041, 2009.

Dilip Krishnan, Terence Tay, and Rob Fergus. Blind deconvolution using a normalized sparsity measure. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 233-240, 2011.

Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. DeblurGAN-v2: Deblurring orders-ofmagnitude faster and better. In IEEE International Conference on Computer Vision , pages 8878-8887, 2019.

Wei-Sheng Lai, Jia-Bin Huang, Zhe Hu, Narendra Ahuja, and Ming-Hsuan Yang. A Comparative study for single image blind deblurring. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1701-1709, 2016.

Zoe Landgraf, Alexander Sorkine Hornung, and Ricardo Silveira Cabral. Pins: Progressive implicit networks for multi-scale neural representations. arXiv preprint arXiv:2202.04713 , 2022.

Anat Levin, Yair Weiss, Fredo Durand, and William T Freeman. Understanding and evaluating blind deconvolution algorithms. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1964-1971, 2009.

Anat Levin, Yair Weiss, Fredo Durand, and William T Freeman. Efficient marginal likelihood optimization in blind deconvolution. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2657-2664, 2011.

Richard A Levine and George Casella. Implementations of the monte carlo EM algorithm. Journal of Computational and Graphical Statistics , 10(3):422-439, 2001.

Dasong Li, Yi Zhang, Ka Chun Cheung, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. Learning degradation representations for image deblurring. In Proceedings of the European Conference on Computer Vision , pages 736-753, 2022.

Ji Li, Yuesong Nan, and Hui Ji. Un-supervised learning for blind image deconvolution via Monte-Carlo sampling. Inverse Problems , 38(3):035012, 2022.

Ji Li, Weixi Wang, Yuesong Nan, and Hui Ji. Self-supervised blind motion deblurring with deep expectation maximization. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13986-13996, 2023.

Lerenhan Li, Jinshan Pan, Wei-Sheng Lai, Changxin Gao, Nong Sang, and Ming-Hsuan Yang. Learning a discriminative prior for blind image deblurring. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6616-6625, 2018.

Aristidis C Likas and Nikolas P Galatsanos. A variational approach for bayesian blind image deconvolution. IEEE transactions on signal processing , 52(8):2222-2233, 2004.

Boyu Lu, Jun-Cheng Chen, and Rama Chellappa. Unsupervised domain-specific deblurring via disentangled representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10225-10234, 2019.

Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy Networks: Learning 3D reconstruction in function space. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4460-4470, 2019.

Tomer Michaeli and Michal Irani. Blind deblurring using internal patch recurrence. In Proceedings of the European Conference on Computer Vision , pages 783-798, 2014.

Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3883-3891, 2017.

Jinshan Pan, Jiangxin Dong, Yu-Wing Tai, Zhixun Su, and Ming-Hsuan Yang. Learning discriminative data fitting functions for blind image deblurring. In IEEE International Conference on Computer Vision , pages 1068-1076, 2017.

Jinshan Pan, Risheng Liu, Zhixun Su, and Xianfeng Gu. Kernel estimation from salient structure for robust motion deblurring. Signal Processing: Image Communication , 28(9):1156-1170, 2013.

Jinshan Pan, Deqing Sun, Hanspeter Pfister, and Ming-Hsuan Yang. Blind image deblurring using dark channel prior. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1628-1636, 2016.

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 165-174, 2019.

Daniele Perrone and Paolo Favaro. Total variation blind deconvolution: The devil is in the details. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2909-2916, 2014.

Valeriya Pronina, Filippos Kokkinos, Dmitry V Dylov, and Stamatios Lefkimmiatis. Microscopy image restoration with deep Wiener-Kolmogorov filters. In Proceedings of the European Conference on Computer Vision , pages 185-201, 2020.

Yuhui Quan, Xin Yao, and Hui Ji. Single image defocus deblurring via implicit neural inverse kernels. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 12600-12610, 2023.

Dongwei Ren, Kai Zhang, Qilong Wang, Qinghua Hu, and Wangmeng Zuo. Neural blind deconvolution using deep priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3341-3350, 2020.

Jaesung Rim, Haeyun Lee, Jucheol Won, and Sunghyun Cho. Real-World blur sataset for learning and benchmarking deblurring algorithms. In Proceedings of the European Conference on Computer Vision , pages 184-201, 2020.

Vishwanath Saragadam, Jasper Tan, Guha Balakrishnan, Richard G Baraniuk, and Ashok Veeraraghavan. Miner: Multiscale implicit neural representation. In European Conference on Computer Vision , pages 318-333, 2022.

Christian J Schuler, Michael Hirsch, Stefan Harmeling, and Bernhard Schölkopf. Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence , 38(7):1439-1451, 2015.

Nimrod Shabtay, Eli Schwartz, and Raja Giryes. PIP: Positional-encoding image prior. arXiv preprint arXiv:2211.14298 , 2022.

Qi Shan, Jiaya Jia, and Aseem Agarwala. High-quality motion deblurring from a single image. ACM Transactions on Graphics , 27(3):1-10, 2008.

Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems , pages 7462-7473, 2020.

Libin Sun, Sunghyun Cho, Jue Wang, and James Hays. Edge-based blur kernel estimation using patch priors. In IEEE International Conference on Computational Photography , pages 1-8, 2013.

Guangpin Tao, Xiaozhong Ji, Wenzhuo Wang, Shuo Chen, Chuming Lin, Yun Cao, Tong Lu, Donghao Luo, and Ying Tai. Spectrum-to-kernel translation for accurate blind image super-resolution. Advances in neural information processing systems , 34:22643-22654, 2021.

Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia. Scale-recurrent network for deep image deblurring. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8174-8182, 2018.

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9446-9454, 2018.

Subeesh Vasu and AN Rajagopalan. From local to global: Edge profiles to camera motion in blurred images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4447-4456, 2017.

Martin Vetterli and Jelena Kovacevic. Wavelets and Subband Coding . Prentice Hall PTR, 1995.

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing , 13(4):600-612, 2004.

Yang Wen, Jie Chen, Bin Sheng, Zhihua Chen, Ping Li, Ping Tan, and Tong-Yee Lee. Structure-aware motion deblurring using multi-adversarial optimized CycleGAN. IEEE Transactions on Image Processing , 30:6142-6155, 2021.

Oliver Whyte, Josef Sivic, and Andrew Zisserman. Deblurring shaken and partially saturated images. International Journal of Computer Vision , 110(2):185-201, 2014.

David Wipf and Haichao Zhang. Revisiting bayesian blind deconvolution. Journal of Machine Learning Research , 15(111):3775-3814, 2014.

Shaowen Xie, Hao Zhu, Zhen Liu, Qi Zhang, You Zhou, Xun Cao, and Zhan Ma. DINER: Disorderinvariant implicit neural representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6143-6152, 2023.

Li Xu and Jiaya Jia. Two-Phase kernel estimation for robust motion deblurring. In Proceedings of the European Conference on Computer Vision , pages 157-170, 2010.

Li Xu, Jimmy SJ Ren, Ce Liu, and Jiaya Jia. Deep convolutional neural network for image deconvolution. In Advances in Neural Information Processing Systems , pages 1790-1798, 2014.

Li Xu, Shicheng Zheng, and Jiaya Jia. Unnatural ℓ 0 sparse representation for natural image deblurring. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1107-1114, 2013.

Xiangyu Xu, Jinshan Pan, Yu-Jin Zhang, and Ming-Hsuan Yang. Motion blur kernel estimation via deep learning. IEEE Transactions on Image Processing , 27(1):194-205, 2017.

Yong Xu, Ye Zhu, Yuhui Quan, and Hui Ji. Attentive deep network for blind motion deblurring on dynamic scenes. Computer Vision and Image Understanding , 205:103169, 2021.

Yanyang Yan, Wenqi Ren, Yuanfang Guo, Rui Wang, and Xiaochun Cao. Image deblurring via extreme channels prior. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4003-4011, 2017.

Liuge Yang and Hui Ji. A variational EM framework with adaptive edge selection for blind motion deblurring. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10167-10176, 2019.

Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5728-5739, 2022.

Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Multi-Stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14821-14831, 2021.

Hongguang Zhang, Yuchao Dai, Hongdong Li, and Piotr Koniusz. Deep stacked hierarchical multi-patch network for image deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5978-5986, 2019.

Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Bjorn Stenger, Wei Liu, and Hongdong Li. Deblurring by realistic blurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2737-2746, 2020.

Lin Zhong, Sunghyun Cho, Dimitris Metaxas, Sylvain Paris, and Jue Wang. Handling noise in single image deblurring using directional filters. In Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 612-619, 2013.

Zhong Zhuang, Taihui Li, Hengkang Wang, and Ju Sun. Blind image deblurring with unknown kernel size and substantial noise. International Journal of Computer Vision , 132(2):319-348, 2024.

Wangmeng Zuo, Dongwei Ren, David Zhang, Shuhang Gu, and Lei Zhang. Learning iteration-wise generalized shrinkage-thresholding operators for blind deconvolution. IEEE Transactions on Image Processing , 25(4):1751-1764, 2016.

6 Appendix

Below we provide additional details and results which are not presented in the main manuscript.

A Proof of Proposition 1

Proof. The proof is based on the convolution theorem and the properties of the DFT. Without loss of generality, we assume that the image x and the kernel k are of size 2 M × 2 N . Let x ↓ 2 and k ↓ 2 denote the down-sampled images of x and k , respectively. Let F denote the discrete Fourier transform (DFT). Recall that for a signal z ∈ R 2 M × 2 N , its DFT is defined as:

Then, for a down-sampled version of the signal z ↓ 2 , whose DFT is

The DFTs of the down-sampled signals x ↓ 2 and k ↓ 2 can be expressed as follows: For each frequency [ m,n ] , we have

and

By convolution theorem, for y = x ⊗ k , we have that

Thus,

Applying convolution theorem on x ↓ 2 ⊗ k ↓ 2 again, we have

Plugging in (16) and (17) into the above equation, we have

Define 3 filters g 1 , g 2 , g 3 by

Then, theirs DFTs are given by

Then, the expression of the right-hand side of Equation (21) can be rewritten as:

By convolution theorem, we have then

The proof is complete.

B Ablation study on performance impact by different maximum scale S 0

In this ablation study, we investigate how different choices of scale S 0 impact the performance on motion deblurring tasks. The experiments are conducted using Lai et al. 's dataset. We compare results using the same framework with different S 0 values: single-scale, two-scale, three-scale, and four-scale (i.e., S 0 ∈ { 0 , 1 , 2 , 3 } ). Each setting undergoes a total of 5000 training epochs, with each stage except the last one consisting of 500 training epochs.

As shown in Tab. 7, multi-scale frameworks consistently outperform single-scale setups. Among the multi-scale configurations, the three-scale model demonstrates the most significant improvement. However, the four-scale model shows only a slight improvement over the single-scale setup. This minimal gain can be attributed to the initial stages of the four-scale framework, where the target image, resized to 1/8 of the original size, is far from the true image. This highly erroneous initialization causes more harm than benefit, hindering the NN's convergence to the true image.

C Implementation Details

C.1 Hyper-parameter settings for different Datasets

The following Tab. 8 summarizes the hyper-parameters, specifically the range of kernel sizes and the coarsest scale S 0 used in our model across various datasets. These parameters are selected based on the specific characteristics of each dataset, including image size.

C.2 Details of image generators

Please see Tab. 9 for the details of the architectures used for image generative network Φ x . In Tab. 9, the "CBR" denotes the subblock for three successive layers: Conv2d+BatchNorm2d+ReLU. The "CBR/Down" means modifying the Conv2d layer in "CBR" with stride (2 , 2) , which performs downsampling operations. The "Up" means the upsampling layer with bilinear interpolation. The "Cat( m , n )" denotes the concatenation of the outputs from layers No. m and No. n .

D Additional comparison of computational cost

Tab. 10 reports the running times for processing a 256 × 256 image with a 31 × 31 blur kernel, comparing severe existing BID methods, including those beyond self-supervised approaches. The inference time is measured using an NVIDIA 3090 RTX GPU. Unlike supervised methods, which require considerable time for model training but is very fast on processing images using pre-trained model, our approach involves training a model for each specific image that needs processing. This makes our method similar to traditional iterative methods, focusing on a case-by-case basis rather than relying on a pre-trained model. The running time for processing an image is comparable to many iterative optimization methods and existing self-supervised learning solutions. One of future works would be on how to reducing this per-image training time, possibly through the implementation of meta learning or testing-time model adaption.

E Broader impacts

The proposed self-supervised learning method for deblurring images has the potential to impact a wide range of applications, including surveillance security, scientific research, and digital media restoration. By improving the accuracy and clarity of images, our research can facilitate deeper insights and more effective interventions in these fields.

In surveillance, higher clarity images can enhance public safety by providing more detailed visual information. However, this also raises concerns about privacy and the potential for mass surveillance. In scientific research, improved image quality can lead to better data and more significant discoveries, though there is a risk that overly processed images could misrepresent the original data. In digital media restoration, while the technique helps preserve cultural heritage, it also poses the risk of altering historical records.

Despite these possible concerns, our goal is to contribute to enhancing image clarity in critical areas such as public safety, scientific research, and cultural preservation. We emphasize the responsible application and continuous improvement of this technology to mitigate potential risks and maximize its positive impact.

F Visual comparisons of the results in the experiments

F.1 Visual comparisons on Lai et al. 's dataset

In this section, we visualize the results of different methods on some examples from Lai et al. 's dataset [26], which is known for its severe blurring effects. Figs. 1 - 6 demonstrate that our method consistently produces results with sharper details and fewer artifacts compared to existing methods, showcasing its effectiveness in addressing significant blur challenges. In contrast, supervised learning methods trained on external datasets yield poor quality results, highlighting the limited generalization performance of supervised approaches when dealing with complex real-world blurring.

Cho

[12]

Pan-DCP [42]

MPRNet [73]

MIMO-UNet [11]

Restormer [72]

SelfDeblur [47]

MCEM[33]

VDIP [18]

Ours

GT

Cho

[12]

Xu & Jia [65]

[67]

Michaeli & Irani [38]

Perrone & Favaro [44]

Pan-DCP [42]

Kaufman & Fattal [20]

MPRNet [73]

MIMO-UNet [11]

Restormer [72]

SelfDeblur [47]

MCEM[33]

VDIP [18]

Ours

GT

Cho

[12]

Xu & Jia [65]

[67]

Michaeli & Irani [38]

Perrone & Favaro [44]

Pan-DCP [42]

Kaufman & Fattal [20]

MPRNet [73]

MIMO-UNet [11]

Restormer [72]

SelfDeblur [47]

MCEM[33]

VDIP [18]

Ours

GT

action

acbion

developing countrie

emiss

developing countrie

OTT

HiiParties

Hii Parties

nanues

Ditcateur

nent

enb]-

climate

developed

CO2

nent.

Renbl-

p

七

developed,

ddns

incl

ddns

incl

Heneol

rPanios

HuParbies

AritigationA

Llas

6666

action

erelognAcou

developingcountrie

IParG

CiesiParties

enbJ~

ddns

incl

action

action

action

developingcountrie

emissions:

e developingcountrie

commil

developingcountrie

commit

emissions

PanoiosuliParties?

Parties!

iParties

ent!

nent

mitigationdeve

climate

developed

eul-mitigationldeve

nent.

developed

enul-8mitigation deve

nent.

developed

oddns

inclu

incl

ddns

inclu

Pan-DCP [42]

MIMO-UNet [11]

SelfDeblur[47]

MCEM[33]

VDIP [18]

Ours

GT image

Cho

[12]

Xu & Jia [65]

[67]

Michaeli & Irani [38]

Perrone & Favaro [44]

Pan-DCP [42]

Kaufman & Fattal [20]

MPRNet [73]

MIMO-UNet [11]

Restormer [72]

SelfDeblur [47]

Maln Corference

Rebudtal Ferlod

Der hiorg ta Athon

lon Deadine

Frdnymb3.2

ry701

Mondej,ur 70.2014

Aell 3r

Main Conference

Paper Submi

Retbutal Period

AeaChal Noeting

ntary Material.Sube

ta-Auth

lon Deadline:

Em.

Frdny13

Mondey.January20.,2014

SaturdeyMarch1,2014

Main Conference

AeaChalk Meting

Rebuttat Period

Frdyb0133

Mondey, Jaruary 20. 2014

Febnuary723,2014

January 20:27, 2014

Saturdey, March 1, 2014

ber 8, 2013

2217.2339

Loyr a peieyg ay

Rebutal Ferliod

lre Oal Moeting

auppl

n Deafliner

ay.Nnb .201]

Modery. Jaary 20 2014

fetruary 7.1.204

Prday.e,2013,235

Jaary 20-17,2014

Main Conference

Paper Sobnt

Decsiens to Author:

nentary Material Sub

Friday. Nove

Seturdey, arth 1, 2014

Fridny, Nvember 1, 2013, 2359 GaIT

Mondey mary 20, 2014

nber 8, 2013

-4.24/

Agell,3054

Meln Conferense

Paper %sds

Ratbuttal Perliod

lev

Ion Deadlis

Prndny, er 12013,235GT

Fridey, Ioven

Jaruary7207.204

fetruary 22 23. 2014

Mondry. Jargary 20. 2014

nber I, 2013

Aprl 2514

Main Conference

PaperSsabeisr

AraChair Meting

SupplementaryMaerial

oysg

Rebuttal Period

Frda201

Frdyb3

Seturdey, March 1, 2014

Main Conference

AreaChair Meting

Paper Subn

sopyopsesmaa

Rebuttal Perod

eg p-ae

isions to Authors:

lon Deadline:

sson：Frdo2013

Frdy,Novembe1,20132359GMT

April2014

MainConference

Paper Subis

Rebutal Perod

Dectsionst Authors:

ReveRelsed to Authors

Aea Chalr Meting

Camera-ready Deadline:

lion Deadline:

Friday,Novmer20132359GMT

Mondy.January202014

Saturdey, March1,2014

Aprll2014

Main Conference

Caera-ready Dedline:

Paper Subission Deadline:

Rebuttal Period

Area Chair Meting

Revevs Relesed tAuthors

Decisions to Author:

Frday，Ne201335

Aprl 2014

MCEM[33]

VDIP [18]

Ours

GT

Pan-DCP [42]

MIMO-UNet [11]

SelfDeblur [47]

MCEM[33]

VDIP [18]

Ours

GT image

F.2 Visual comparisons on Köhler et al. 's dataset

In this section, we display the visual results on Köhler et al. 's dataset [22], which is characterized by its non-uniform blurring effects. From Fig. 7-Fig. 8, our method demonstrats satisfying performance on this challenging dataset, effectively handling the intricate and variable blurring effects. Furthermore, we compare our results with those obtained using the Diffusion-based method, BlindDPS [13], revealing its difficulty in adapting to blurring effects and image types that deviate from its pre-trained model's training data.

Cho &Lee [12]

Kaufman & Fattal [20]

BlindDPS [13]

SelfDeblur[47]

MCEM[33]

Ours

GT image

~

Cho &Lee [12]

Kaufman & Fattal [20]

BlindDPS [13]

SelfDeblur[47]

MCEM[33]

Ours

GT image

D

F.3 Visual comparisons on Lai et al. 's real dataset

Fig. 9 illustrates a comparison between our method and other competitive self-supervised methods on Lai et al. 's real-world data, where our method consistently generates higher quality images.

SelfDeblur [47]

MCEM[33]

Ours

国

®

Bean

Bean

Bean

IT Legend

IT Legend

IT Legend

成就工下傳奇

成就工T傳奇

成就工下傳奇

fobepotkk

obopot.hk

F.4 Visual comparisons on RealBlur dataset

In this section, we present a visual comparison of the results from state-of-the-art supervised learning methods [73, 11, 72] and our approach on the RealBlur dataset. Although our method achieves slightly lower average PSNR and SSIM values across the entire dataset compared to these representative supervised learning methods, the visual evidence presented in Fig. 10 demonstrates that our results are often sharper. In other words, the gap between our method and the supervised methods is smaller in terms of visual quality than indicated by the quantitative metrics.

MIMO-Unet [11]

MPRNet [73]

Restormer [72]

Ours

GT image

F.5 Visual comparisons on Microscopic dataset

This section provides a comparison between our method and other methods in the task of microscopic deconvolution. Our approach consistently restores images with more texture and details.

0

1

INIKNet

SelfDeblur

MCEM

VDIP

Ours

GT

G Visual comparison on the ablation study

This section provides a comprehensive visual comparison for the ablation study in Fig. 12, illustrating the impact of various configurations on our model's performance. These visuals serve as a supplement to the detailed quantitative results discussed in the main text, found in Sec. 4.4.

w/o

L

cross

Single-Scale

w/o Progressive

INR/CNN as

Φ

k

/

Φ

x

MLP/INR as

Φ

/

Φ

x

MLP/CNN as

Φ

/

Φ

x

Multi-scale by

↓

s

Ours

GT image

k

k

H Visualization of Intermediate Results

Iteration=500

Iteration=1000

Iteration=1500

Iteration=5000

x

(2)

[

I

(2)

x

]

x

(1)

[

I

(1)

x

]

x

(0)

[

I

(0)

x

]

x

(0)

[

I

(0)

x

]

GT image

Apart from validating the efficacy of our approach through the quality of the final results, it is also interestting to see the intermediate status of the proposed method. To this end, we use "manmade_01" from Lai et al. 's dataset [26] as a case study, visualizing intermediate results to better understand the progress through different stages to the final output. As shown in Figure 13, in the first stage, we obtain an initial representation at the smallest scale during initialization. In the second and third stages, we achieve results at larger scales with finer details. In the final stage, the neural network builds on the groundwork laid by the preceding stages to refine the output, ultimately generating the desired clear image.

I Experiments on Levin et al. 's dataset [28] with small kernel size.

Levin et al. 's dataset contains 32 images generated by convolving 4 clear images using 8 motion-blur kernels and adding Gaussian white noise with s.t.d. 1% . The size of these kernels is small, ranging from 11 × 11 to 27 × 27 . Following [47], besides PSNR and SSIM, Error Ratio [30] is also used as a quantitative metric. From Tab. 11, our approach performs well across all three metrics. For visual comparisons, please see Figs. 14- 15 where our results more closely resemble the GT image, with fewer artifacts and more details.

Krishnan

[24]

Cho

[12]

[29]

Xu & Jia [65]

Sun

[54]

[78]

Pan-DCP [42]

MPRNet [73]

Restormer [72]

BlindDPS [13]

SelfDeblur [47]

MCEM[33]

VDIP [18]

Ours

GT

Krishnan

[24]

Cho

[12]

[29]

Xu & Jia [65]

Sun

[54]

[78]

Pan-DCP [42]

MPRNet [73]

Restormer [72]

BlindDPS [13]

SelfDeblur [47]

MCEM[33]

VDIP [18]

Ours

GT

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: This paper introduces a novel self-supervised method for blind image deconvolution that does not require ground truth images. We provide a detailed introduction in the Methodology section, and the contributions proposed are proven through extensive experiments across various datasets, showing improvements over existing methods. These results directly support the claims made in our abstract and introduction, accurately reflecting our research contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of this work in the appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: For the proposition used in our paper, we provide detailed proof in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We present the implementation details in Sec. 4 and Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We utilize publicly available datasets, which allows researchers to easily access and use the same data for verification and comparison purposes. Additionally, we have detailed the implementation specifics of our method in the Sec. 4 and Appendix C.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes, the paper meticulously specifies all the necessary training and test details required to understand and replicate the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: The paper does not include error bars or other measures of statistical significance, such as confidence intervals or p-values. This omission is due to the nature of the proposed self-supervised method, which differs fundamentally from traditional supervised learning. In our approach, training is performed on a per-sample basis without randomness or variations introduced by different training/test splits. Consequently, each model is tailored specifically to the individual sample being processed, eliminating the need for multiple repetitions of the training process. As a result, the reporting of variability measures like error bars is not applicable. Additionally, the supervised methods referenced in the paper employ fixed training/test splits, which similarly do not require error bars for their results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We report our compute resources in Appendix D.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research presented in this paper adheres fully to the NeurIPS Code of Ethics. We ensured that all experiments were conducted with strict compliance to ethical standards, including transparency, fairness, and respect for all participants involved. Additionally, the methodologies used do not involve any harmful practices or potential for misuse.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the broader impacts of this work in Appendix E.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The paper utilizes several pre-existing datasets and codebases, which are properly cited within the manuscript.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a blind image blurring method which reparametrizes the latent images the blurring kernel by the implicit neural representations (INRs). In addition, the authors also propose a cross-scale consistency loss. The authors validate the effectiveness of their method on several datasets.\n\nSTRENGTHS:\n1. The cross-scale consistency loss sounds an interesting idea.\n2. The presentation of this paper is good.\n\nWEAKNESSES:\n1) The idea of using a deep neural network for latent images and blurring kernel reparametrizatrion is not new. The current proposed method is very similar to the one in ref1. But I do not see the authors have discussed the differences between the proposed method and the method in Ref1. Thus, I doubt the novelty of this proposed method.\n\n2) The current design, the authors use different network architectures to generate images and kernel. But how to control or balance the complexity between latent images generator and the kernel generator?  The authors need to make it more clear why use these specific network design for images and kernel.\n\n3) If the batch size is 1, will it be better to use other norms instead of batch norm?\n\n4) The authors should also compare their method with the one in Ref1 as the Ref1 seems to be the more recent one compared to other self-supervised learning method.\n\n5) The authors should also show how the generated kernel looks like. Is the generated kernel feasible? Is it close to the ground-truth kernel?\n\nRe1: Zhuang, Zhong, et al. \"Blind image deblurring with unknown kernel size and substantial noise.\" International Journal of Computer Vision 132.2 (2024): 319-348.\n\n\n=========\nscore has been revised after reading rebuttals.\n\nQUESTIONS:\nI only see very limited novelty of this paper. The proposed idea is similar to the one in Ref1. The only contribution I can see is the cross-scale consistency loss. I list the details in the [Weaknesses].\n\nRe1: Zhuang, Zhong, et al. \"Blind image deblurring with unknown kernel size and substantial noise.\" International Journal of Computer Vision 132.2 (2024): 319-348.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a self-supervised method for BID that does not require GT images. By leveraging an exact relationship among the blurred image, latent image, and blur kernel across consecutive scales, this paper propose an **effective cross-scale consistency loss** implemented by representing the image and kernel with **implicit neural representations** (INRs), whose resolution-free property **enables consistent yet efficient computation for network training at multiple scales**. The experimental part  verifies that proposed method outperforms some existing self supervised methods on several datasets.\n\nSTRENGTHS:\n- The paper proposes a new method on the existing self-supervised niform BID task, which is complete and sound. The main contribution is the cross-scale estimation consistency constraint, and provides a complete theoretical proof of the rationality of the regularization term. The cross-scale consistency is used to train progressive scale IRNs from convergencing to trivial solutions.  which has a certain originality and provides an inspiring new analytical way for BID.\n- The paper conducts in-depth theoretical and experimental analysis on uniform-based deconvolution, verifies the superiority of the proposed method through experimental comparison on two types of motion blurred image synthetic data and real data, and combines sufficient ablation experiments to verify the impact of each module of the framework on performance.\n- Aside from a few details, the submission is clearly written and generally well organized, and the supplementary materials are accompanied by a large amount of quantitative and visualization materials.\n- The authors are careful and honest about evaluating both the strengths and weaknesses。\n\nWEAKNESSES:\n- The INRs framework and progressive learning mechanism on which the paper is based are common in image restoration, the degree of innovation is average except for cross-scale estimation consistency constraint.\n- With the development of BID or the larger blind image restoration area, the degradation settings of different methods are somewhat different. When describing the paper’s setting (Eq.1), in addition to comparison methods, introducing the existing blurring degradation settings (uniform kernel/non-uniform kernel/implicit modeling) and their differences and application value is necessary. The discussion of this part is missing, which causes readers/researchers confused by different method settings and applicability.\n- The related work section of the paper does not fully cite and introduce recent learning-based self-supervised or frequency estimation degradation and some important works, such as KernelGAN(NeurIPS2019), FCA(AAAI2021), S2K(NeurIPS2021), etc. These solutions also involve deconvolution and propose estimation methods.\n- Some descriptions of the method principles or experimental settings are vague or problematic, see Questions.\n\nQUESTIONS:\n- There are some confusing points in Tab.1: (1) The results of the supervised method are of little reference value for they are trained on non-convolutional modeling datasets(GoPro). (2) According to the ablation experiment, even without the main contribution (1) mentioned in line 73, the PSNR(24.44) is almost the same as the second best method(MCEM) (24.55). The uniform blur setting is inconsistent with some existing methods (such as the non-uniform modeling used by MCEM). Are those self-supervised method retrained for comparison? \n- The proposed uniform blurring method is better than MCEM in the non-uniform deblurring experiment (Tab.2), which is a bit contradictory. Is this an advantage brought by individual sample training rather than the method itself?\n- In the multi-scale training process, what is the effect of x and k estimated in the previous scale on the next scale? From the image, it can be understood that coarse-to-fine is getting clearer. How does k change in this process? In addition, in Tab.5, the average PSNR of training only at the original scale is two points higher than that of training with at three scales, indicating that the optimization direction of BID at different scales is inconsistent. In this case, why does training with progressive scales bring better performance?\n- According to the existing conclusions in the BID field in lines 94-98 of the paper, the second contribution point of the paper summarized in lines 107-108 is not important. In addition, the use of multi-scale coarse-to-fine can alleviate the ill-posed problem of multiple solutions. There is no analysis or literature references provided though some work has been done before indeed.\n- In Lines 191-192, it claims the role of the cross-scale estimation consistency is to limit the ambiguous solution of BID. Though a rational analysis of the approach and ablation experiments are provided, readers may wonder what is the motivation for designing this regularization term and its actual impact in BID process. If there is some intuitive analysis or explanation, it will be easier for them to accept.\n- For the standard downsampling implemented by the coordinate index in line 165, is interpolation  need  when the image size is not even? Is the downsampling used in the ablation experiment to compare image-to-image NNs done in the same way or with more common interpolation methods such as bicubic or bilinear?\n\nLIMITATIONS:\nThe authors have  adequately addressed the limitations and potential negative societal impact of their work as follows:\n- Computational cost for processing a large number of images, as the method requires training the model for each individual sample.\n- The proposed method is only applicable to handle uniform blurring, as it relies on the convolution model. \n\nConsidering the practical application value and the single-image training and reasoning method proposed, the paper should compare the computational cost of the single-image reasoning results and the comparison with existing methods.\n\nAs mentioned above, it is necessary to add discussions about different setting and it is confusing that proposed method performs better than MCEM in non-uniform setting.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a self-supervised method for blind image deconvolution (BID). The main idea is to introduce the implicit neural representation (INR) technique for representing both the blur kernel and the image, such that they can be parameterized at different scales using a single model. With such a INR representation, supervision can be enforced in a cross-scale way to enhance the efficacy of the model. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS:\n1. Introducing INR to the studied BID problem is new and brings benefits in cross-scale representation.\n2. Various datasets have been considered in experiments, which makes the results convincing.\n\nWEAKNESSES:\n1. The presentation of this manuscript should be improved. For example, a large part of Section 1.2 can be put into Section 3.\n2. Experiments are insufficient in the following sense:\n- A recent self-supervised BID method [1] is not compared.\n- The cross-scale supervision training strategy can also be applied other self-supervised methods by with simple downscale operation, which is not tested.\n- The use of the loss functions is not well justified. For example, it is not clear what the performance could be if other more popular ones are used, such as L2 and L1 loss.\n- Model complexity comparison in addition to the running time in Appendix D should be provided.\n\nReferences:\n\n[1] D. Huo et al. Blind Image Deconvolution Using Variational Deep Image Prior. IEEE Transactions on Pattern Analysis and\nMachine Intelligence, 45(10): 11472-11483, 2023.\n\nQUESTIONS:\n1. To my knowledge, the performance of fully supervised methods highly depends on the training data. So I would like to know how the supervised methods were trained on the Lai, Kohler, and Levin datasets, since only test images are contained in them.\n2. The method is constructed based on the uniform blurring assumption, and thus it should be discussed why it can perform well on the Kohler dataset which focuses on non-uniform blurring.\n3. Typos: In Line 31, $\\delta,y=k\\otimes x$ should be $\\delta\\otimes y=k\\otimes x$.\n\nLIMITATIONS:\nThe authors discussed the limitations of the method, which are reasonable.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an approach to solve blind image deconvolution. The authors use multiscale Implicit Neural Representations (INRs) to depict both the latent image and the blur kernel. In addition, they propose a method that incorporates a cross-scale consistency loss and a progressive scale optimization process. Experimental results demonstrate superior performance on small-scale simulated datasets and competitive results compared to state-of-the-art methods on large-scale and real-world datasets.\n\nSTRENGTHS:\n1. The authors propose a cross-scale loss function that compensates for the inaccuracies introduced by simply downscaling the latent image and the blur kernel, and then calculating their convolution at the lower scale. This is achieved through the use of Quadrature Mirror Filters (QMFs).\n    \n2. The authors conduct ablation studies, providing evidence of the effectiveness of each component within the proposed method.\n    \n3. The proposed method outperforms other considered unsupervised methods in synthetic uniform blind deconvolution.\n\nWEAKNESSES:\n1. There exist prior works on Multiscale Implicit Neural Representations. The authors should provide a more detailed discussion on the similarities and differences between their proposed method and these previous works. For instance:\n    \n\n- “PINs: Progressive Implicit Networks for Multi-Scale Neural Representations” by Landgraf, Zoe, Alexander Sorkine Hornung, and Ricardo S. Cabral, presented at ICML 2022.\n    \n- “Miner: Multiscale Implicit Neural Representation” by Saragadam, Vishwanath, et al., presented at ECCV 2022.\n    \n\n2. The method appears to require at least one hyperparameter (the number of scales) to be set based on the target dataset. This suggests that the method is not entirely blind.\n\nQUESTIONS:\n\\- Minor points:\n\n    \\- In Algorithm 1, please indicate whether the loop end condition in step 4 includes ‘0’.\n\n    \\- On line 259, the sentence “the test dataset consists of 120 images“ appears to be repeated.\n\nLIMITATIONS:\nBlind deconvolution, when limited to the uniform case, may not be particularly useful for real-world problems. While the authors have acknowledged this, it would be beneficial to include experiments demonstrating the effectiveness of the proposed method’s components on widely-cited non-uniform datasets, such as GoPro [40]. This would provide valuable guidance for future research in the field.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}