Self-diffusion for Solving Inverse Problems
Authors: Guanxiong Luo
Shoujin Huang
Venue: NeurIPS 2025 poster ()
Keywords: inverse problems
computational imaging
image reconstruction
diffusion models
Primary area: applications
TL;DR: Self-diffusion solves inverse problems without the need of pretrained generative models via a self-contained iterative process that alternates between noising and denoising steps to progressively refine its estimate of the solution.

Abstract: We propose ***self-diffusion***, a novel framework for solving inverse problems without relying on pretrained generative models. Traditional diffusion-based approaches require training a model on a clean dataset to learn to reverse the forward noising process. This model is then used to sample clean solutions---corresponding to posterior sampling from a Bayesian perspective---that are consistent with the observed data under a specific task. In contrast, self-diffusion introduces a self-consistent iterative process that alternates between noising and denoising steps to progressively refine its estimate of the solution. At each step of self-diffusion, noise is added to the current estimate, and a self-denoiser, which is a single untrained convolutional network randomly initialized from scratch, is continuously trained for certain iterations via a data fidelity loss to predict the solution from the noisy estimate. Essentially, self-diffusion exploits the spectral bias of neural networks and modulates it through a scheduled noise process. Without relying on pretrained score functions or external denoisers, this approach still remains adaptive to arbitrary forward operators and noisy observations, making it highly flexible and broadly applicable. We demonstrate the effectiveness of our approach on a variety of linear inverse problems, showing that self-diffusion achieves competitive or superior performance compared to other methods.

Self-diffusion for Solving Inverse Problems
Guanxiong Luo 1uoguan50gmail.c com
Shoujin Huang solor. www.or.piiachhummaa.ccmmm com
Abstract
We propose self-diffusion, novel framework for solving inverse problems without relying pretrained generative models. Traditional diffusion-based approaches require training a model on a clean dataset to learn reverse the forward noising process. This model is then used to sample clean solutions-ccorrepponeee corresponding posterior sampling from Bayesian perspective that are consistent with the observed data under specific task. In contrast, self-diffusion introduces self- consistent iterative process that alternates between noising and denoising steps to progressively refine its estimate of the solution. At each step of self-diffusion, noise is added to the current estimate, and a self-denoiser, which a single untrained convolutional network randomly initialized from scratch, is continuously trained for certain iterations via a data fidelity loss to predict the solution from the noisy estimate. Essentially, self-diffusion exploits the spectral bias of neural networks and modulates it through a scheduled noise process, Without relying on pretrained score functions external denoisers, this approach still remains adaptive arbitrary forward operators and noisy observations, making it highly flexible and broadly applicable. We demonstrate the effectiveness of our approach on variety of linear inverse problems, showing that self dif--difusion achieves competitive or superior performance compared to other methods.
Introduction
Inverse problems involve reconstructing hidden cause mouse-often called the source e.g., image or signal-fron from observed data. The observed data often incomplete and noisy, which leads ill- ill-posed problems. These problems occur in wide range of domains and are common in fields like signal and image processing, including tasks like medical imaging reconstruction, image denoising, and low-level vision. Prior knowledge about the source is particularly valuable when addressing ill ill-posed problems, where observed data alone may not be sufficient for an accurate and faithful reconstruction. A Bayesian framework offers approach in such scenarios by combining observed data with prior information. In this view, the goal is to infer the underlying source given the observations by estimating its posterior distribution. This posterior reflects balance hetween the likelihood, which is determined by the forward model and noise, and the prior, which encodes assumptions or knowledge about the source. Because the likelihood is often fixed by the physical model, the choice of prior plays a critical role in improving the quality and stability of the reconstruction.
The conventional approach often relies handcrafted priors to regularize the solution space and steer optimization toward more plausible outcomes. Classic techniques such as Tikhonov regularization methods [3, 4] have been widely used. While these methods can be effective in certain settings, they often struggle to faithfully recover complex signals when the observed data is insufficient. Thereafter, supervised learning approaches, since deep neural networks have gained popularity, are used to learn mappings from large datasets of input-output pairs [5, 6]. These learned models have demonstrated strong performance in specific tasks. However, their effectiveness typically depends on the availability of large amounts of labeled training data, which may not always be accessible. Furthermore, these models often lack robustness distribution shifts,
[1], total variation [2], and
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

lipainting
(b) deblurning
(c) SR
(1 3D
3D MRI
Demonstration of self-diffusion applied to various inverse problems across natural and medical imaging domains. (a) Image inpainting (b) Motion deblurring (c) 4x image super-resolution (SR) (d) 2D MRI reconstruction from 6x undersampled data (e) 3D MRI reconstruction from 5.7 undersampled data In each set, the leftmost image shows the degraded input, the middle shows the self-diffusion result, and the rightmost shows the ground-truth or reference.
Figure 1:
such varying noise levels or different acquisition settings, limiting their practical utility in some situations. Recent advances in generative modeling have introduced powerful alternative: using generative models as learned priors. Models such as variational autoencoders (VAEs) [7], generative adversarial networks (GANs) [8], and conditional autoregressive models [9] have shown impressive capabilities in capturing complex data distributions. In particular, diffusion models [10, 11, 12] have emerged as leading class of generative models with abilities to produce high-quality samples. Leveraging the denoising step in diffusion models, recent works have adapted them for inverse problems [13, 14, 15], using a pretrained model to guide the reconstruction process. These generative priors effectively encode structural knowledge of the data and yield robust reconstructions. However, their applicability depends heavily on access to large-scale curated datasets for training. When the data for training priors is limited not available, these methods may not be suitable.
Interestingly, recent research has shown that deep neural networks can serve as strong implicit priors, even without pretraining. The Deep Image Prior (DIP) approach [16, 17, 18] exploits the tendency of overparameterized networks capture natural image statistics. DIP, a randomly initialized neural network is optimized directly on the observed measurements to minimize reconstruction error. Despite not being trained on external data, the network's inductive biases the solution toward natural-looking images. While promising, DIP's performance is sensitive to the optimization process and often suffers from premature convergence or limited expressiveness, which can lead suboptimal results.
In this work, we propose self-diffusion, novel approach for solving inverse problems without relying on pretrained generative models. Our method is motivated by the observation that diffusion models generate images progressively from low high frequencies through the denoising process. Building on this insight, we design self-diffusion process that iteratively reconstructs the source starting from noise and using spectrum- reguultede self-denoiser based on the DIP framework. During the process, single neural network (serving as the denoiser) is continuously optimized to minimize the discrepancy between estimated reconstruction and observed data across different noise levels. This allows the model to refine the reconstruction in progressive manner, guided entirely by the observed measurements, without any prior training or external datasets. Our contributions are as follows:

---

1. We introduce self-diffusion, novel method for solving inverse problems that leverages the diffusion process without requiring pretrained generative models. To the best of our knowledge. this is the first such approach.
2. We provide theoretical form of the self-diffusion process, offering solid foundation for understanding its behavior and convergence properties from the perspective of spectral bias. We evaluate self-diffusion on a range of inverse problems, including MRI reconstruction, low- level image restoration, and radar angle estimation. Experimental results demonstrate that our method achieves competitive performance compared to other methods.
optitelze: i- (MGc))) -11"
optimize: 4- [ADaOk--)) ->I
optimize: ((10p0r)) -yl
XL
X
Xp
XQ
weights inherited
weights
weights inherited
inheriled
Overview of the self-diffusion framework. At each diffusion step i, Gaussian noise is added to the current estimate xiree, producing a noisy input. single self-denoiser (initialized with an untrained network) is then trained to produce true by minimizing the task-specific data fidelity loss. This iterative alternation of noising and denoising from xy to x1 yields a coarse-to-fine reconstruction process without the need for pretrained models or external supervision.
Figure 2:
2 Self-diffusion for inverse problem
Problem Setup. We consider the inverse problem defined by the equation
Ax Inue
where A E m is a known forward operator, xtree E IR" is the unknown deterministic solution, and y R represents the observed data. In many practical scenarios, the problem is underdetermined, ie., m < nt, which results in more unknowns than equations and makes the it potentially non-unique and ill-posed. To resolve this, regularization techniques typically applied to introduce prior knowledge and ensure the existence and uniqueness of a stable solution.
Bayesian framework is adopted in many works, where treat xIree random variable with a prior distribution p(x), reflecting our prior knowledge about the solution. The observed data y modeled observation lne with likelihood function p(y)x x) that quantifies how probable the observed data is, given the solution x. Bayes' theorem allows us to compute the posterior distribution of tree given the observed data and the prior p(x)
p(xly) ply x)x)))
p(y)
where p(y) = (py[|x)p(x) dx is the marginal likelihood. The solution xime is then obtained by maximizing the posterior distribution. In this framework, the prior p(x) can be interpreted as form of regularization, as it enforces certain desirable properties on the solution (such as smoothness, sparsity, structure).
Self-diffusion.
To map the estimated noisy solution x, = x + (t)e to the true solution tie self-diffusion process trains a self-denoiser Dø., each noise step t T l,... 0 over k == O,..., K- - iterations to minimize the loss
Lr = AD.., (x"
(x! + +()))) - 2
where x!!" is the estimate of the within noise step t, initialized as xhr = cn, with co >N(0,) I). The noise e, ~ NO, /) is sampled once at the start noise timestep and fixed across all K iterations within that timestep. It is then resampled for the next noise step / 1. The noise schedule is o, VI- de, where ā; = II 3. and 3 Bend (B Gena). After K iterations at each noise step, the self-denoiser De... produce the estimated solution x! for the next noise step - 1. As the noise level (t) decreases to zero, the predicted solution converges to the true
3

---

solution x under Theorem in Heckel and Soltanolkotabi (2020). The detailed demonstration of this process appendix A. The pseudo-code for implementation shown in Algorithm
Since é, is fixed within each noise step but resampled at each subsequent noise step, the process X, is piecewise stochastic. In the continuous limit, this process be modelled with stochastic differential equation (SDE) over time. Considering the noise resampling across steps, the SDE is
dx. = (x, x x )tt + (t))
(2)
where (t) is Wiener process capturing the resampling of noise across time steps. The drift term (x, x reflects the self-denoiser's ' push toward x The derivation this SDE is provided in appendix B. This self-diffusion process is combination of the forward and reverse processes because: 1) the forward process self-referential (x, = x + (t), not starting from the curated noise-frce training dataset; 2) the proposed self-denoiser directly learns to predict x!rle at each noise step instead of relying on the pretrained score function or denoiser.
Algorithm Self-diffusion (SDI) for solving the inverse problem
1: Input: x noise steps Ti iterations-Ke K;; learning rate y, forward operator - A; initialize
0r.0: default noise schedule Axtart 0.0001, Bend 0.01
2: for T to 0 do
3: Sample E ~ N (0. 1).x; (xx!"e + (t)e:)
4: for = 0 to K
do
5: Compute loss LL, and Vo.. Le.k
6: Update the weights using optimizer and learning rate
7: end for
8: Set 0: = f2.K
9: Compute x Do. (x:)
10: end for
11:
return xfuue
Noise-Modulated Spectral Bias. Neural networks, especially overparameterized regimes, are known to exhibit spectral bias- tendency leam low-frequency components of a target function before high-frequency ones [19]. This behavior has been theoretically linked to the eigenspectrum of the Neural Tangent Kernel (NTK) [20], which governs the dynamics of gradient descent during training. In particular, the NTK often has larger eigenvalues associated with smooth, low-frequency modes and smaller ones for high-frequency oscillations. Consequently, gradient updates naturally biased toward reconstructing coarse, low-frequency structures earlier in training.
The training of self-denoiser in self-diffusion exploits this spectral bias and further regulates it through the structured noise schedule. Considering the self-denoiser Equation (1), we expand the expected loss function using first-order Taylor expansion into
E., [ILADo. (xx) - y| ||ADe,, (x."") y11 +o())))))A))()
Data Fidelity Term
Regularization Term
where the second term acts as a regularizer, discouraging the network from fitting sharp, high- frequency fluctuations. The derivation of this expansion is provided in appendix C.I.
This regularization effect is modulated by the noise level o,. In carly steps of diffusion, where noise is large, the regularization strong, enforcing smooth outputs and prioritizing low low-frequency components. As the process progresses and a, decreases, the regularization weakens, allowing the network to focus finer, gighffreeeency details. This design leads to implicit multi-scale learning regime, where the reconstruction transitions from global structure to local refinement. We reveal this process in Fourier space detailed in appendix C.2.
Simulation
We apply the self-diffusion inference to recover sparse 1D signal from compressed sensing mea- surements. The original signal is generated sum of sine wave with varying frequencies and

---

amplitudes and has sparse representation in the frequency domain. The signal x is constructed as
Σ sin 2 / (A,f)es
N
We have predefined set of iuplitude-frequenng pairs S as
$ = ((10,, 1), (0.5, 15), (0.3,20), (1.0, 6), (0.8,3), (0.6, 4), (0.7,5))
and is the total signal length, with t E {0, N This signal is deliberately designed to span both low and high frequencies with non-uniform amplitudes to challenge the network's ability to recover both coarse and fine details from compressed measurements. The signal is measured using a random Gaussian sampling matrix A Rm*x where m < The measurement matrix is normalized to have unit Frobenius norm. The measurements are then obtained as y Ax. In this experiment, the signal length is 128, and the number of measurements 35.
To recover the signal, we employ a ID U-Net architecture with bottleneck layers and normalization layers self-denoiser. All convolutional layers in the denoising network initialized using Gaussian distribution with zero mean and standard deviation of 0.02, while biases are initialized to zero. This initialization helps stabilize training and ensures the early network outputs maintain consistent scale and variance across layers. The parameters for Algorithm is (T =T= 40; K = 200; v le 3,mm 4e 3; Aed with the same learning rate and initialized network as SDI. The loss used both methods are enhanced with L1 penalty in frequency domain. We also ADMM-basis pursuit in [21], which classical compressed sensing method, as another baseline. Figure shows the original signal and the recoverd signal in time domain and frequency domain. The progression of signal recovery over denoising steps is illustrated in the supplementary video freq ltree/rrreggaeeeeeeeeeeeeereeeee prog.mp4. In the early iterations, the model primarily recovers the low-ffeqqe wwwfireuenccy components the signal, which dominate the coarse structure in the time domain. As the process continues, higher-frequeng details gradually emerge. This progressive reconstruction from low to high frequencies demonstrates the self-denoising capability of refining the initial estimate iteratively through successive denoising steps while benefitting from the spectral bias of the self-denoiser.
*) and Adam optimizer used. We use DIP to reconstruct the signal
Recovery Time Domain. Error: ADMM: 5.5741; DIP: 2.4329; SDI: 0.5326.
Ampittude
2
0
True Signal
-2
ADMM-BP
SDI
-4
DIP
-6
Mea sured Sam plmples(( (A"Ty)
Sample Index
Recovery Frequency Domain
60
Signal FFT
SDI FFT
ADMM FFT
Magnitude
40
FFT
x
10
20
30
40
50
60
Frequency (cycles per signal length)
Figure 3: Recovery of sparse signal composed of multiple sine waves from compressed measure- ments. The top plot illustrates time-domain reconstruction, comparing self-diffusion inference (SDI) ADMM-basis pursuit (ADMM-BP) and Deep Image Prior (DIP). The bottom plot displays the frequency spectra, where SDI more accurately captures both low- and high-frequency components compared ADMM-BI BP and DIP.
5

---

4 Application
To demonstrate the versatility and generality of self-diffusion across range of inverse problems spanning both natural and medical imaging domains, we evaluate our method on several representative tasks, including 2D and 3D MRI reconstruction, low-level vision restoration (inpainting, deblurring, denoising, super-resolution) and radar angle estimation recovery. The implementation details for each task are provided in the following sections. The code is available at github:ggluo/self. -ifffuusso.
4.1
MRI reconstruction
2D sampling. We evaluate the effectiveness of self-diffusion on 2D MRI reconstruction from undersampled k-space data. The raw k-space is subsampled using a equispaced Cartesian mask with an acceleration factor of 4x/6x along the phase-encoding direction. And 20 central auto-calibration signal (ACS) lines are obtained. From the fastMRI validation set [22], we randomly selected 20 samples for each contrast (TI, TI post contrast, T2, and FLAIR) to form a test set of 80 subjects. We estimated coil sensitivity maps estimated using BART's ecalib command. Measurements are formed with A, which consists of the undersampling mask, Fourier transform, and coil sensitivities.
For self dif-diffusion, we initialize the 2D U-Net with Gaussian initialization {n = 0, a = 0.02) for convolutional kernels and zero initialization for biases. To ensure stable optimization and consistent data fidelity weighting, we normalize the k-space measurements before input to the reconstruction process. Specifically, we compute scale factor to match the norm of the initialized network's 's output to the norm of A The k-space measurements are scaled accordingly, preventing the network from unstable gradients due to mismatched signal energy. We use Adam optimizer in Algorithm I and set parameters {T = 40; K = 50; rj = 0.001; S = Le "; Bend = -eel). We compare SDI to the following baseline methods: Aseq [23], IMJENSE [24], and Deep Image Prior (DIP). For these Aseq and IMJENSE, we use authors's 's official implementations. The loss for DIP and SDI is enhanced with the total variation term, weight set to 0.0001. As an ablation study, we also include SDI without resampling noise in Algorithm and refer it as "w/o e, SDI". The PSNR, SSIM, and NRMSE metrics are computed against the fully sampled ground truth. Table shows the reconstruction quality for all methods. SDI achieves the highest PSNR and SSIM, and the lowest NRMSE, outperforming Aseq and DIP that share the characteristic of using untrained networks but lack the iterative diffusion process. Figure 4 compares the reconstructed images for 2D MRI from 4x undersampled k-space data without ACS lines. Visually, SDI reconstructs sharper anatomical boundaries and finer textures with fewer aliasing artifacts compared to other baselines. In Appendix E, Figures 10 and 11 illustrate the visual results for 4x and 6x undersampled k-space data, respectively, and Figure 8 reveals the evolution of the estimate over the self- self-denoising process.
Table 1: Comparison for MRI reconstruction from 4x and 6x undersampled data.
4x
6x
4x w/ ACS PSNRT/SSIMT/RMMSLL
Method
(ANRT/SSIM//RMSSSS
PSNR//SSIM//NMMS)
A
26.78 0.7411 /0.472
25.84/ 25.84/0.6990 0.6990/ / 0.0526
15.01 / 0.44////5/6//////// / 0.1815 N/A
IMJENSE
38.80/ 0 0.9604 / 0.0125
35.30/0 0,939/////////// / 000188
Aseg
38.18 8661 /0.0272
29.74/ 0 82449 /0.0353
30.87/0.8460 / /0.0318
DIP
38.35 0.9574 / 0.00336
$4.92/0.9363 00.0192
36.34/ 0.9454 / 0.0178
wfo
39.09 0.96699/ 0.0120
36.21 /.94474 0.0016
37. 0.9497 0.0016
SDI
39.21 /.996009///000 00.01
36.84/0.9497 0..4//0.492/0.0000
38.20/ 0.9541 /0.0136 36
Comparison with methods using pretrained diffusion model. We compared SDI with CSGM [25] on reconstructing MR images of different contrast (T2 and FLAIR) as shown in Table 2. The diffusion model released in Ref. [25] was trained on T2 contrast images. CSGM performs better in the reconstruction of T2 contrast images except in the case without ACS lines. CSGM requires 1150 the network forward evaluations per image. When CSGM apply this pre-trained model to FLAIR images, its performance is not as good as T2 contrast images. However, this does not mean that SDI is superior to methods using pretrained diffusion model as Ref [13] showed better cross-contrast generalization performance. Further cammain-specific comparisons are omitted as they are beyond the scope of this work.
Table 2: Compare SDI to CSGM on reconstructing different contrasted images.
/SSNR
Ansellration
CSGM /T2
who SD1/12
SDE/TR T2
CSCM FI AIR
whor, FLARR
SDJ/ FLR
4x
78/ 19
38. 30 0.9633 F00035
38.08 0.988 00033
0.9435 (10153
57.95 37955/ / 0L0138
38.41 0.9564 /00011
37,55/ 0.9636/ / 0.0136
35.11 000185
/ 0.9555 00016
35.46/ 0.9286 0.0190
35.21 0.9367 / 00187
35.81 / /0.015
ACS
34,00/ / 000256
36.55 0.9543
37.66 0.962//
0..53/0.0015 /00222
36.39 00950/04/4/000
00.18 /0.05)

---

DIP
Aseq
w/o noise SDI
SDI
GT
A
Figure 4: Reconstructions and error maps GT from 4x undersampled k-space w/o ACS lines. A"y y is the degraded.
4.2 Radar angle estimation recovery
Radar sensors are widely used in various applications, including automotive, aerospace, and etc. However, the resolution of radar angle estimation is often limited by the number of antennas (spatial sampling rate). This is typical task that ground truth data hard to obtain. Ref. [26] formulates inverse problem, and apply self-diffusion recover unknown high-resolution angle estimations x from low-resolution radar range-azimuth heatmaps y which represent the spatial locations of objects through signal power. Following the setup in [26), we leverage large-scale autonomous driving perception RADIal dataset
[27]. The measurement of low-resolution radar aange-azimuth heatmap is synthesized by applying the radar angle measurement process to the objects that captured by LiDAR sensors under 86 antennas (spatial sampling rate). The same as previous settings, the denoising network randomly initialized. The parameters for Algorithm is (T 30; K 100; 1c Buan 2c 22; Bem 1e -= and Adam optimizer is used. One thing deserves be mentioned is that because the supposed output of our model is 2D binary mask which can be easily thresholded to obtain the final point cloud, we use sigmoid activation function as the last layer of the U-Net. Figure illustrates the radar recovery using self-diffusion on the RADIal dataset and four different scene from the dataset are shown. The results demonstrate significant resolution improvement despite hardware limitations making labeled data for radar enhancement challenging obtain.
Camera
Rade Mesurement
Recovery
Camara
Radar Measarencal
Reoner
The camera column shows the visual context captured by the camera, depicting the road, vehicles, and surroundings. The radar measurement column displays the low-resolution radar range- azimuth heatmaps y. The recovery column presents the recovered high-resolution angle estimations.
Figure 5:

---

4.3 Low-level vision tasks
Evaluation. We evaluate self-diflusion on three representative low-level vision tasks: image inpainting, motion deblurring, and single-image super-resolution Each task is formulated as y Ax. where x is the unknown image, y is the observation and A is a task-specific degradation operator. performed on 1000 images that are from the dataset ImageNet and resized
The experiments to 256x256. For image inpainting, random rectangular masks remove regions of the image and the goal is to successfully fills missing regions with coherent structure and texture. For motion deblurring, the blur is generated using code from BattLeewBoroddnko/oottbbbbbl with kernel size 61 x 61 and intensity value 0.3, and goal is to reconstruct sharp images from blurry inputs. In super-resolution tasks, low-resolution images are generated via average pooling and the goal is to recover high-resolution images from low-resolution inputs. We the Algorithm 1 for all the tasks above and set it with parameters {T' = 40; K = 150; 0.001, Suunt = le feee = =). We also compare SDI two baselines: DIP [16] and Aseq [23]. The loss for DIP is enhanced with the total variation term. Evaluation metrics like PSNR, SSIM, and LPIPS are computed against the ground truth and are shown in Table 3. The visual results are illustrated in Figures 12 to 15 listed in Appendix F.
Table 3: Reconstruction quality comparison for low-level vision tasks. (PSNRT / SSIM /LPIPSL)
Task
Aseq
DIP
wio SDI
SDI
2x SR
26.91 /0.9944/ 0.0502
27.26 0.8359 0.0531
29.06/ 0.8530/ / 0.0388
26.97 02.97/0.80 0.88852/////// 00.053
30.30/0.9186 0.0351
4x SR
22.45 /..33/0.////////////// 000833
24.14/ /.7535/
0.0719
24.55 0.66555/ 0.0650
24.09/0.755, 0.7553 0.0704
25.82 /.81100//////// 0.0585
Motion
18.84 08,56/ 0.1222
22.65 / 0.6686 0,0840
25.94/
0,7375 0.0589
26.92 0.8202 0.0513
27.76 0.8459 0.0488
Inpainting
18.06 /0,885 /0.134//0.1339
28.06 0.8589
0,0516
29.41 0.9310/ 0.0384
25.42/
0,8840 0.0804
30.78 0.929 0.0422
Denoising. We included denoising results on CBSD68 dataset [28] with noise level o = 25. SDI use the parameters {T 30; K = 100; 0.002; 8ann 1e 7; Bemd Se ")= DIP has 3000 iterations enhanced with the total variation term whose weight is set to 5e 44 The other baselines use default parameters. As shown in Table 4, SDI performances worse than FFDNET and IR-SDE but better than DIP. The supervised method FFDNET provides the best performance terms of PSNR and SSIM. IR-SDE is the best in terms of LPIPS. In addition to the numerical metrics, we provided visual comparisons in Figure 6.
Noisy
IR-SDE
FFDNET
SDI
Ground Truth
20.40/ / 0,5000 / 0.3357 25.11 / 0.3874 / 0.7500 30.95 / 0.1006 / 0.8708 32.11 / 0.1867 / C.997 27.42 / 0.3566 0.8257 PSNR/ / LPIPS / SSIM
Figure 6: Visual comparison of denoising results across different methods against the ground truth. IR-SDE by Luo et al., 2023; FFDNET by Zhang et al., 2018.
Large image We further evaluated SDI on a large image of size (5000x3000), which is from an inexperienced shooter and corrupted by dust spots and minor noise. We masked the dust
Table 4: Comparison for denoising
Method
PSNR1 / LPPPS SSIM
DIP
25.44 15.44/0.300 0.307 / 0.6600
spots using a binary mask then restored the image using SDI via parameters [T = 40; K = 500; 1 0.001; Sam = 12 Bemt le ". Results are shown in Figure 7. The noise on the cloudy sky and sea surface are removed, while the details on the architecture
FFDNET
31.22 0.121 0.8821
IR-SDE
28.09 0.100 0.7866
SDI
28.10/ 0.207 0,7860
are preserved. The restoration takes around hours on RTX A6000. As limited by the size of paper, we will post the boosted image at ggluooseef-difffsin for better display.
8

---

Full view
Box
Box 2
Box 3
Box 4
Original
Boosted
Masked
Figure 7: SDI boosts image quality. The original image corrupted by dust spots and minor noise.
4.4 Hyperparameters sensitivity
Table 5 presents PSNR (dB) values for various combinations of iterations (K) and noise steps (T) when reconstructing an MRI image from a 6x-undersampled k-space. The following trends are observed: 1) For each fixed T, PSNR generally increases with K up to certain point, then plateaus or slightly declines; 2) Higher T values tend to yield higher PSNR at lower K, but the advantage diminishes K increases; 3) After reaching peak, PSNR tends to stabilize or decrease slightly. Total iterations (T x K) range from 250 (T = 10, K 25) to 20000 (T = 40, K = 500). Generally, more iterations lead to higher PSNR before plateauing. Therefore, when choosing hyperparameters, first select to balance performance and computational cost, then consider K for optimal PSNR.
Table 5: PSNR for different iterations K and steps choices.
T
25
50
75
100
150
200
300
400
500
24.65
28.89
31.16
33.12
34.41
35.11
35.95
35.96
36.06
20
29.35
33.27
34.04
35.30
35.69
36.15
36.33
36.02
36.24
40
32.13
34.54
35.60
35.96
36.19
36.17
36.07
36.04
36.11
Table 6 summarizes runtime and iteration comparison for different reconstruction methods.
Table Runtime and iteration comparison for different reconstruction methods.
IMJENSE
CSGM
DUNM
SDI
Tack, Stzc:
MRL 320 120
MRL 320-330
MRI 320- 320
MRI 520x<100
Iow-lesel vicuu, 356x255
low-level visier, 256-226 inpointing 5000 3000
Iterhiec
11.92
10.94
13.39
16.94
8,51
19.64
1.52
Tonal iters
1500
2000
1156
3000
1000
100
20000
GPU info
RTX
4090, 240B
RTX 3000 TI, 24GB
RTX 4090, 24GB
RTX 3050 TI, 24GB
RTX 4090, 240B
RTX 4090, 24CB
RTX A6000, 45GB
5 Discussions
In this work, we proposed self-diffusion novel approach for solving inverse problems, which combines elements of denoising, diffusion processes, and the spectral bias of neural networks. Self- diffusion leverages the spectral bias of overparameterized neural networks, which naturally prioritize low-frequency components during optimization. This bias is enhanced by a decreasing noise schedule, creating a hierarchical, coarse-to-fine reconstruction process. Early denoising steps focus on smooth, low-frequency structures, while later steps refine high-frequency details. This behavior mimics multiscale optimization-wee withoot requiring explicit architectural scale separation- and is well suited to many inverse problems as demonstrated by our experiments.
Compared DIP, self-diffusion exhibits three key advantages: 1) Initialization robustness: DIP performance is sensitive to weight initialization, often requiring careful tuning. Self-diffusion, in contrast, is largely robust random initializations due to its iterative, noise-modulated training scheme; 2) Frequency generalization: DIP tends to overfit training when data is noisy. Self-diffusion, through its structured noise schedule, maintains better control over frequency content during optimization; 3) Better zero-shot understanding: The noise in self-diffusion promotes understanding of image, which leads to more coherent structure and texture shown in Figure 12. While this is the opposite case for the "woo noise SDI" and DIP.
components early in

---

The self- diffusifu framework introduces a form of implicit regularization driven by noise. The de- noiser's Jacobian is captured by the differential VD and the term the frequencies bias. This mechanismis naturally similar to the NTK perspective, where low-frequency modes have larger eigenvalues and thus converge faster. The regularization term amplifies this bias by suppressing first-order derivatives, effectively damping high-frequency components as shown in Appendix C.2.
)|? quantifies
While self-diffusion shows strong generalization and performance across different tasks, several promising directions remain for further exploration: 1) Adaptive noise scheduling: learning task- specific or image-adaptive noise schedules may yield better convergence and fidelity; 2) Neural architecture search: automatically discovering architectures better suited for self-diffusion could improve both efficiency and reconstruction quality; 3) Hybriding with pretrained models: combining self-diffusion with pretrained features or priors may boost performance on more complex or semantic tasks.
6 Conclusions
Self-diffusion introduces simple yet powerful framework for solving inverse problems without external data or pretrained models. By leveraging the intrinsic spectral bias of neural networks and regulating it through a structured noise schedule, it enables progressive, coarse-to-fine reconstruction. This approach achieves strong performance across diverse inverse problems like medical image reconstruction, signal recovery, and low-level vision tasks while remaining architecture- and data- agnostic. The results highlight the potential of self-supervised optimization guided by structured noise, opening new directions for learning-free and data-efficient image restoration.
References
[1] Andrey N. Tikhonov and Vasiliy Y. Arsenin. Solutions of ill-posed problems. H. Winston & Sons, Washington, D.C.: John Wiley & Sons, New York, 1977.
[2] Leonid I. Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1-4):259 268, 1992. ISSN 0167-2789. doi: 10.101660167-227899999002228
URL https://wwet www.scedireecteeeeeeceeeeeeeeeee wwt.ecccooccooeoeoooe scinne/ httpp://www.cctenniieec.ccm
arricleopp1//117777899999992
[3] David L. Donoho. Compressed sensing, IEEE Transactions Information Theory, 52(4): 1289-1306, 2006. doi: 10.1 100911..002000..811115
[4] Emmanuel J. Candès, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 52(2):489-500 2006. doi: 10. 10.1109/TTT.2005.66222 866.88888....
[5] Kyong Hoon Jin, Michael T. McCann, Emmanuel Froustey, and Michael Unser. Deep convolu- tional neural network for inverse problems in imaging, IEEE Transactions on Image Processing, 26(9):450944522, 2017. doi: 10. 1099T11..22112 3099.
[6] Jonas Adler and Ozan Öktem. Solving ill pooed inverse problems using iterative deep neural networks. Inverse Problems,
2017. doi: 10.1888/136 36188/131-20 -621/2/9916622291...
[7] Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
[8] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Varle-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[9] Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Con- ditional image generation with pixelenn decoders. Advances in neural information processing systems, 29, 2016.
[10] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on International Conference on Machine Learning Volume 37, ICML' 15, page 2256-2265. JMLR.org, 2015.
10

---

[11] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances neural information processing systems, 33:6840-6851, 2020.
[12] Yang Song, Jascha Sohl-Dickstein Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https::/ httbp:://opeerviee.. net/fforum//ddPXXIIG112R
[13] Guanxiong Luo, Moritz Blumenthal, Martin Heide, and Martin Uecker. Bayesian mri recon- struction with joint uncertainty estimation using diffusion models. Magnetic Resonance in Medicine, 90(1)0095-53) 2023.
[14] Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In International Con- ference Learning Representations, 2023. URL https://openrevieu OnD9ZGAGTOx.
[15] Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[16) Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9446-9454, 2018.
[17] Dave Van Veen, Ajil Jalal, Mahdi Soltanolkotabi, Eric Price, Sriram Vishwanath, and Alexan- dros G. Dimakis. Compressed sensing with deep image prior and learned regularization. arXiv preprint arXiv:
118866006438, 2018. URL https://arxiv, org/ has///bs//100 06438.
[18) Reinhard Heckel and Mahdi Soltanolkotabi. Compressive sensing with un-trained neural networks: Gradient descent finds a smooth approximation. In International conference on machine learning. pages 4149-4444 4158. PMLR, 2020.
[19] Reinhard Heckel and Mahdi Soltanolkotabi.
Denoising and regularization via exploiting the structural bias of convolutional generators. In International Conference Learning Representations, 2020. URL http::: net/forumm HJeeqhAYDDS
[20] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
[21] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, al. Distributed opti- mization and statistical learning via the alternating direction method of multipliers. Foundations and Trends@ in Machine learning, 3(1):1-122, 2011.
[22] Jure Zhontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, et al. fastmri: An open dataset and benchmarks for accelerated mri. arXiv preprint 1e1iv
2018.
[23] Ismail Alkhouri, Shijun Liang, Evan Bell, Qing Qu, Rongrong Wang, and Saiprasad Ravishankar. Image reconstruction via autoencoding sequential deep image prior. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httpss//openreviiee net/forum?icee 2itfformmm?ddkKKBBAABB
[24] Ruimin Feng, Qing Wu, Jie Feng, Huajun She, Chunlei Liu, Yuyao Zhang, and Hongjiang Wei. Imjense: Scan-specific implicit representation for joint coil sensitivity and image estimation in parallel mri. IEEE Transactions on Medical Imaging, 43(4):1539-1553, 2024. doi: 10.1 TMI.2023.3342156.
[25] Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G Dimakis, and Jon Tamir. Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems, 34:14938-14954 2021.

---

[26] Yanlong Yang, Jianan Liu, Guanxiong Luo, Hao Li, Euijoon Ahn, Mostafa Rahimi Azghadi, and Tao Huang. Unsupervised radar point cloud enhancement arbitrary lidar guided diffusion prior, 2025. Uttthtts::// //te:i////xxvvveee org/ abs /abb/b/0 2505. 098877.
[27] Julien Rebut, Arthur Ouaknine, Waqas Malik, and Patrick Pérez. Raw high-definition radar for multi-task learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition (CVPR), pages 17021-17030 2022.
(28) D. Martin, C. Fowlkes, D. Tal, and Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedingy Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume pages 416-423 vol.2, 2001. doi: 10.1 10.1109/CCC..200193339
[29] Ziwei Luo, Fredrik K Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B Schön. Image restoration with reverting stochastic differential equations. International Conference on Machine Learning, 2023.
[30] Kai Zhang, Wangmeng Zuo, and Lei Zhang. Fidnet: Toward fast and flexible solution for enn-based image denoising. IEEE Transactions on Image Processing, 27(9):4608-400 2018.
12

---

A Error Propagation under Ideal Denoiser Performance
We follow the setup for self-diffusion described in Section 2. All solutions to the ill-posed inverse problem can be parameterized as:
A + Zoull
where At is the Moore-Penrose pseudoinverse of A, and ?uoll E null((A) is an arbitrary vector in the null space of A. The challenge is to identify x from this affine solution space. The self-diffusion algorithm iteratively refines an estimate of x irue, The process iterates backward in "noise time" steps t = 1,T .,..... O. minimizing the loss:
At each step i, neural network denoiser Dø..: (e.g., CNN) is trained by
L = IIADo (xt tmue + (U)ee) - -yl?
where x R" is the image estimate step We initialize it with x cD, eo (O, Ln) is random noise vector. 4 ~N( In) is a sample of Gaussian noise, fixed during the K inner training iterations within given timestep i, and resampled for the next step - o(l) > 0 positive noise schedule, monotonically decreasing such that (() - 0 as ( - 0. Or,* are parameters of the denoiser at inner iteration i of step I. Then, the iterative update is given by:
1. Sample Et (10 In) set x, xtrue + o(l)ez
2. For k ... K 1, compute the gradient of Lep w.r.t. 02,2 and update parameters;
3. Set the denoiser for step as Dg. Do. k
4. Update the image estimate via x = Dø. (xx)
We define the relative change eтror as ||x - x 1/2//11//////////// ||xx the behavior the relative change error e, under idealized denoiser performance i.e., the following holds as 0:
[|2. In this section, we analyze
[e, + 0 and [x,1|xx!-u - | +
(3)
Heckel and Soltanolkotabi (2020) claimed that there exists unique true image x!rx that satisfies Ax = y and untrained network possesses a structure (c.g., piccewise smoothness, low-frequency dominance) learnable by Do.- Therefore, view the trained Do, act as denoiser, implicitly favoring the structure of xim For each step f, the inner training iterations are sufficient for the denoiser Do to approximately satisfy the measurement constraint ADo, (-) = y when applied to inputs of the form x!! + (t) The denoiser's 's error with respect to the true signal, o, (Xamixy) == De, (Xnaisy) has an expected squared norm Gi == E., [x! + (tt))e))l? We assume that Çe - as + when the measurements y Ax is enough to recover the true signal using the implict structure of denoiser Do,- In the following, we show that the convergence of e₁ in Equation (3) is determined by the denoiser's error with respect to the true signal, i.e., Gt.
tme
A.1 Update Rule and Error Dynamics
Let d, = x tru be the absolute error of the estimate at step t. The update rule for the image estimate is:
x Do. (x! mie + (())))
Using the definition of &, we have:
x Inue = x true + dx(x) Inue + (t))))
Subtracting tne from both sides gives the error recurrence:
d = (x true + (t)ee)
Taking the expected squared Euclidean norm:
E, [|| ||]] = E., [1 (x. tue + (t)) Se Shifting the index, we get the expected squared error for the iterate x!!ee:
E,, [|] de
G++1
13

---

Now, let's analyze the relative change enror ce
x
112
[x xx
x
||d,
d
C2
mie
2
|
tue
||
The norms of the iterates remain bounded below by positive constant, i.e., ||x relevant I, which is reasonable as x - xtrue and |xtue|| > Then, we get
2 6² > 0 for all
1
Re, [e.] S c²
Re, (I.d - d.-111].
Using the inequality ||a bil? - 2||a|1 2 + 2/|b// 2, we have
2
E,, [e:] s e²
(Lee
[||d ] + B. [1/d/11111
Substitute the expressions in terms of Ç:
2 (Se+1 + Sc)
E,, [e,
S ca
Since Se + 0 as - 0, then
Ex [eo] + 0 t +0
A.2 Denoiser as a Proximal Operator
With sufficient training iterations (i.e., large K), the denoiser learns to produce outputs that are consistent with the measurements. This consistency enforced through the untrained network leads implicitly favoring of smoothed images instead of the least-squares solution, which often suffers from artifacts and poor perceptual quality. The optimization dynamics of training the denoiser impart a spectral bias on the solution, favoring smoother, low-frequency components. This behavior can be interpreted through the lens of proximal optimization, whereby the denoiser acts approximately as proximal operator:
D9. (x:) ~ prox R (xx) arg min 1 xzl1² AeH(z),
1
where R(z) is an implicit regularization functional that discourages undesirable features such as high-free nigh-frequency noise or implausible structures. We will analyze in more detail, in Appendix C, how the spectral bias shaped by the noise schedule.
B Continuous Time Approximation
The discrete update rule in previous section is written as
x Do, (x + (t)c)) x
Since t decreases from to and let xxt) x!_e, we have
x(t + At) - xt)) = x(t - 1) xtt) = Do. (x(t) + (t)ec) I(t),
where the change in is negative (At = 1). In the continuous limit At - O, we have
dr(t) lim
xtt + At) x(t)
dt Ar-0
At
Since At -1, the right hand side becomes
z(t x(t) Do, (x(t+ =((t),
which leads to
dx(t)
R I(t)= x(t -1) (x()(xx(1) + ((())s)) x()))
di
1
With Do. (x(() + (f) = live + (x((() + +(()))), we have
dx((t)
((xtrue + 6, (x(()) + (t)e))) -x())
dt
x(t)) - true (,(x)tt+
14

---

A + 0, (t) + O, and 6g + 0, we have the ordinary differential equation
dx(() x(t) mue
dt
Since is fixed within each noise step but resampled at each new noise step, the process is piecewise stochastic. In the continuous limit, we model the resampling as stochastic process over time. The SDE form, considering the noise resampling across steps, is
dr(()) (a((t) xt" )dt + a (t)dW((t)
where W (t) is Wiener process capturing the resampling noise across noise steps. The drift term <(() - thw reflects the denoiser's push towards xtrec, and the diffusion term a (I)dW(() models the noise resampling.
C Derivation of Noise-Modulated Spectral Bias
The regularization term,as described in the previous section, arises from the noise added to the input of the self-denoiser and plays a critical role in enhancing the spectral bias. The term is derived from the expected loss over the noise distribution and is approximated on the regularization term, its mathematical form, its effect in the Fourier domain, and its role the self-diffusion process.
regularizer. Below, we elaborate
C.1 Regularization Perspective
To find second-order approximation for the expected loss, [. -x)1))1 where x, x o (t)e; and E ~ NN00, I). For clarity, let's simplify the notation for a single step t: D(-) De. (-), To = x σ = (t). 6 = ft. and let the full transformation be f(x) = AD()). The loss is [0 + or) v/12. We
compute EE [4].
We start by approximating the output of the function f(x) for small perturbation de around the point zg. Using first-order multivariate Taylor expansion, get:
fzo + σc) f(zo) (zo))((oc)
where (2o) is the Jacobian matrix of the function f evaluated at æg. The (i.j)-th element of J, is 0f: Now, we substitute this approximation back into the loss expression: Baz"
L ~ [1(f(z)) - y) - (z)
To simplify, let's define the residual vector = f(ro) The expression becomes:
L L oJ
We expand the squared Euclidean norm (dot product) (v)
L ~ (r + o (r (=
L 2J r r 20r J (o
L |||| + 20r J, (zo)c o² (
This gives us three terms to analyze. We now take the expectation of the loss with respect the noise distribution ~ (0 T).
1. The residual f y does not depend
|| = ||||| = [lf( - y/1²
2. This term is linear in Since the expectation of the noise is zero, [e] 0, this term vanishes.
E, 22r (x())) = 20r J, =
3. This term is a quadratic form in For random vector e N/O, I) and a constant matrix Q, we have the property Ele" Qc] = Tr(Q).
(z(o) E. 2. J((() o²Tr(J)(20)) (20))
15

---

The trace of J is equal to the squared Frobenius norm of the matrix J. denoted || by convention.
('T(()((z))) = =
Combining the expectations of the three terms, we arrive at the final approximation for the expected loss:
E. [4] = (z)) y/1 {z Substituting back our original notation (f(x) (z) x x
Ca and J,
JAD AJD):
E., AD(()))
yl |] ||A - 3/1 + (xx )1?
Data Fidelity Term
Regularization Term
This term penalizes the squared Frobenius norm Jacobian
entire transformation from denoiser's 's input to its final output. The Jacobian measures how much the output changes in response to small changes in the input. Penalizing its norm forces the learned function to be smoother and less sensitive to high-frequ -reuufreyenccy noise. This directly achieves the goal of modulating the spectral bias. By penalizing large derivatives, the optimization is biased toward learning low-frequency components first, especially when the noise level ((t) is high. This provides rigorous foundation for the coarse-to- to-fe- reconstruction behavior described in the previous section.
C.2 Fourier Domain Analysis
To understand the effect of the regularization term, we decompose the denoiser output in the orthonor- mal Fourier basis {ex )== where each eę corresponds to Fourier mode with frequency index The denoiser's output can then be expressed as
D(-)
de. ek:
k=1
where d are the Fourier coefficients at time t. The derived regularizer is
R(D) = at)) [x
where Jn denotes the Jacobian of the denoiser D, A is known linear operator, and -| p is the Frobenius norm. A first-order differential operator acts a Fourier mode e₁ e*k-a by scaling its amplitude by its frequency magnitude |+]. Hence, by Parseval's theorem, the squared I--norm the gradient of D can be expressed in terms of its Fourier coefficients as
|VD12 / [VV(())) de LKKPIdeRn
Since the Frobenius norm the Jacobian, || equals the sum of the squared L2-norms of the gradients of each output component, it follows that
|
Σ k
11,
Thus, the Jacobian term introduces rrequency-dependee quadratic penalty the Fourier coeffi- cients, scaling as k?2. Substituting this into the regularizer gives
R(D) (t)² ||JJ = R(D) (t) Σ Ac
showing that nigher-frequemn modes are penalized more strongly. Moreover, the penalty strength decreases over time through the factor o1) We can therefore write an effective loss in the Fourier domain as:
E,, [e,e] R Σ Ae (dr, - c²)² + ot))
A x|b|
1 de
1=1
Fidelity
Jacobian Regularizer
where Cs are the Fourier coefficients of the true signal and x is proportionality constant. This formu- lation confirms that the noise-modulated regularizer naturally suppresses high-frequency components, with time-dependent weight controlled by (
16

---

D Evolution of the reconstruction process
Figure 8: The evolution of reconstruction x! over noise timestep from 39 to 0.
E MRI Reconstruction
3D sampling. To further demonstrate the flexibility of self-diffusion, apply it the struction of volumetric 3D MRI data from undersampled k-space measurements. This task poses additional challenges due the increased dimensionality and the need for spatial coherence across slices. We consider 12-coil 3D TI TI-weighted brain scan and apply Cartesian undersampling mask on the k-space that has dimension of 160x 160x 128. The undersampling mask is generated using BART's command poisson with an acceleration factor of 2.5 along two chase-encooing directions. This leads total acceleration factor of 8. The coil sensitivities are estimated using BART's command ecalib. The measurements are formed with A, which consists of the undersampling accommodate
mask, 3D Fourier transform, and coil sensitivities. denoising network to 3D U-Net architecture with 3D convolutional layers and which is initialized the same way as the 2D U-Net. The parameters for Algorithm are (T = 40; K 50; 7 = 0.0013 and Adam optimizer is used. Figure 9 shows the reconstructed sagittal, coronal, and axial slices.
dimensionality, we adapt the
17

---

A H y
SDI
GT
Coronal
Sagittal
Axial
Figure 9: Reconstruction of a 3D TI l1-weighted brain MRI using self-diffusion from 8x undersam- pled measurements. Shown are sagittal, coronal, and axial slices, demonstrating sharp anatomical structures and minimal artifacts. The initial measurement (A y), self-diffusion inference (SDI), and ground truth (GT).
IMJENSE
DIP
Aseq
wlo noise SDI
SDI
GT
Figure 10: Reconstruction from 4x undersampled k-space with 20 ACS lines using different methods
(A J.,.ENSE,SE, DIP, Aseq, w/o noise SDI, SDI) and corresponding error maps are shown.
18

---

A A
IMJENSE
DIP
Aseq
w/o noise SDI
SDI
GT
Figure 11: Reconstruction from 6x undersampled k-space with 20 ACS lines using different methods (IMJENSE, DIP, Aseq, w/o noise SDI, SDI).
Low-level Vision Tasks
Degraded
DIP
Aseq
w/o noise SDI
SDI
GT
Figure 12: Comparison of inpainting results across different methods (DIP, Aseq, w/o noise SDI, SDI) against the ground truth (GT).
19

---

Degraded
DIP
Aseq
w/o noise SDI
SDI
GT
Figure 13: Comparison of motion deblurring results across different methods (DIP, Aseq, w/o noise SDI, SDI) against the ground truth (GT).
Degraded
DIP
Aseq
w/o noise SDI
SDI
GT
Figure 14: Comparison of 2X super-resolution results across different methods (DIP, Aseq, w/o noise SDI, SDI) against the ground truth (GT), illustrating the enhancement of image details in various scenes including bear, necklace, an aircraft carrier, and a person in bathtub.
20

---

Degraded
DIP
Aseq
w/o noise SDI
SDI
GT
Cama
cama
Cano
Figure 15: Comparison of 4x super-resolution results across different methods (DIP, Aseq, w/o noise SDI, SDI) against the ground truth (GT), illustrating the enhancement of fine details in images, including a zebra, lens lid, honeycomb, and snake.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See the discussion in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
22

---

Justification:
See Appendices A to C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Sections and 4
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
23

---

Answer: [Yes]
Justification:
We will make sure that the data and code are open access after review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
coyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips.co.. wwtppblic/gguiiieecccceoooocccce http//uide///eee CCodeSubmiieeeeece
hotoli/c/giddeesodeeettbiiiiiiii wetpomisssiiooooooooooooooooo (eionnllccy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See Sections and 4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification: This work is not computation intensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttpps:/nnoee http://eeurp.p.eeeee cc
Answer:
[Yes]
Justification:
We follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This work has neutral societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
25

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work does not pose high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We will license our code and data under CC-BY 4.0.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
26

---

Answer: [Yes]
Justification: We will prepare documentation for our code and data.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification:
We did not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
27

---

Justification:
We use chatgpt for writing, editing, and formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy httt::://neurip..coo / for what should or should not be described.
28


Reviews:
[{"review_id": "frop6j6qPs", "reviewer": "Reviewer_kzzf", "summary": "This paper studies proposes a method for using diffusion models to solve image reconstruction tasks by leveraging the implicit regularization of the denoising network. This paper does this in a sequential process by training the denoising network to iteratively to find images that reconstruct that measurement given the diffusion step $t$.  This procedure operates like a hierarchical inversion scheme, with more regularization at the beginning and increasing the data weight over the course of the inversion scheme.", "questions": "- How did you come up with the hyperparameters for both the proposed algorithm and the DIP baseline and did you do this on a held out test set with ground truth? I would consider increasing my score if this is clarified.\n\n- Why did you choose to solve it step-by-step vs. end-to-end?\n\n- All of the experimental results are limited to linear inverse problems. Is this because the method itself is limited to linear inverse problems?\n\n- How does your method perform for solving denoising problems?", "limitations": "The authors claim they include limitations in section 5, but they are not clear. Please list them more carefully. Is computational speed one of them?", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "(+) The paper is clear and reads well.\n\n(+) This paper aims to solve the important problem of image reconstruction without explicit image priors. \n\n(+) The proposed method has very strong quantitative and qualitative results.\n\n(+) This paper also shows theoretical analysis on why the proposed method works by showing how the annealing of the noise through the diffusion process reduces the regularization over the course of the iterative procedure.  \n\n(-) The chosen baselines are somewhat outdated and there are missing comparisons to unrolled networks which are quite similar to the construction of this method. There has been a lot of work since DIP (e.g. DeepDecoder, DIP with only optimizing the latent vector $z$) that help prevent overfitting to noise, which was a feature listed for the proposed method.  Additionally, there was minimal discussion on how the stopping criteria was chosen for these baselines, which can significantly affect there results of DIP.\n\n(-) In terms of the method itself, is not clear in why the authors chose to solve it step by step rather than end-to-end and what the main advantages of solving it step by step are. \n\n(-) There are many missing quantitative ablation studies, and the hyperparameters across different experiments are different. It is unclear how the authors arrived at the hyper parameter choices for both the proposed method as well as the baselines.\n\n(-) Limited discussion on computational costs. How does this scale to reconstructing large image sizes?", "was_revised": true, "final_justification": "After reading the author response and the other reviews, I believe many of my concerns have been address and this paper should be published.", "rebuttal": "## 1. Hyperparameter selection and DIP baselines\n\nThe table shows PSNR (dB) values for different $K$ (iterations) and $T$ (noise steps) combinations when reconstructing an MRI image from a 6x-undersampled kspace.\n\n|Step, T\\Iteration, K|25|50|75|100|150|200|300|400|500|\n|-|-|-|-|-|-|-|-|-|-|\n|T=10|24.65|28.89|31.16|33.12|34.41|35.11|**35.95**|**35.96**|**36.06**|\n|T=20|29.35|33.27|34.04|35.30|35.69|**36.15**|**36.33**|**36.02**|**36.24**|\n|T=40|32.13|34.54|35.60|**35.96**|**36.19**|**36.17**|**36.07**|**36.04**|**36.11**|\n\n### a. Observations\n1. **Trend with $K$**: For each fixed $T$, PSNR generally increases with $K$ up to a certain point, then plateaus or slightly declines. \n2. **Effect of $T$**: Higher $T$ values tend to yield higher PSNR at lower $K$, but the advantage diminishes as $K$ increases. \n3. **Plateau and Decline**: After reaching a peak (e.g., 36.33 at $K=300, T=20$), PSNR tends to stabilize or decrease slightl\n4. **Computational Cost**: Total iterations ($K \\times T$) range from 250 ($K=25, T=10$) to 20000 ($K=500, T=40$). Generally, the more iterations, the higher PSNR before plateauing.\n\nTherefore, when choosing hyperparameters, first select $K$ to balance performance and computational cost, then consider $T$ for optimal PSNR.\n\n### b. DIP Baselines\n\nThe DIP used in our submission is enhanced with TV regularization. The results in the table below show the PSNR (dB) and SSIM values for DIP with and without TV regularization, when performing the same MRI reconstruction task in the previous table. Without TV regularization, the network can overfit to the data very quickly, leading to low PSNR. When evaluating DIP on each dataset used in our submission, we first tune the hyperparameters for one sample and then use the same hyperparameters for all samples.\n\n|Method|Iter=250|Iter=500|Iter=750|Iter=1000|Iter=2000|Iter=3000|Iter=4000|\n|-|-|-|-|-|-|-|-|\n|DIP w/ TV|31.1610/0.8138|32.2094/0.8342|33.9467/0.8531|34.6047/0.8613|35.1507/0.8783|33.6402/0.7990|35.5463/0.8466|\n|DIP w/o TV|29.8827/0.8418|30.2350/0.8199|29.8380/0.7951|29.6209/0.7761|28.9668/0.6701|28.0941/0.5870|26.8794/0.5366|\n\n|Method|Iter=5000|Iter=6000|Iter=7000|Iter=8000|Iter=9000|Iter=10000|\n|-|-|-|-|-|-|-|\n|DIP w/ TV|35.6637/0.8548|35.6827/0.8644|35.6032/0.8728|35.3603/0.8779|35.4510/0.8861|35.3384/0.8923|\n|DIP w/o TV|26.0204/0.5049|25.3416/0.4552|24.3752/0.4192|23.8758/0.4003|23.4355/0.3849|22.8755/0.3619|\n\n## 2. Comparison with pre-trained model based methods\n\nSuggested by the Reviewer UXaL, we used DPS[1] and DDNM[2] to perform the low-level vision tasks and CSGM[3] to perform MRI reconstruction tasks on the datasets used in our submission. This serves as a comparison of the proposed algorithm with the pre-trained model based methods. The results are shown in the tables below.\n\n### a. low-level vision tasks\nDDNM has the best performance over all tasks using only 100 network forward evaluations (NFEs) except for motion deblurring. But DDNM does not provide motion deblurring in their official reposity and we did not tune our implementation for that task thoroughly. Surprisingly, DPS does not perform close to the performance of DDNM using thousands of NFEs. This is consistent with the results that published in Aseq paper[4].\n\n|Method|2× SR<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓|4× SR<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓|Motion<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓|Inpainting<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓|\n|-|-|-|-|-|\n|Aseq|27.26/0.8359/0.0531/0.2141|24.14/0.7535/0.0719/0.3238|22.65/0.6886/0.084/0.4007|28.06/0.8589/0.0516/0.1788|\n|DIP|29.06/0.853/0.0388/0.0994|24.55/0.6855/0.065/0.3046|25.94/0.7375/0.0589/0.2522|29.41/0.931/0.0384/0.0684|\n|DPS|22.68/0.6685/0.0895/0.3655|20.32/0.5516/0.1099/0.4599|21.84/0.5981/0.0943/0.3394|25.67/0.7634/0.0635/0.267|\n|DDNM|**31.44**/**0.9199**/**0.031**/**0.0346**|**26.44**/**0.8117**/**0.0541**/**0.203**|22.40/0.6838/0.0886/0.3998|**34.04**/**0.9718**/**0.023**/**0.0173**|\n|w/o noise SDI|26.97/0.8852/0.0543/0.0907|24.09/0.7583/0.0704/0.2992|26.92/0.8202/0.0513/0.2375|25.42/0.884/0.0804/0.1037|\n|SDI|30.3/0.9186/0.0351/0.0858|25.82/0.811/0.0585/0.2563|**27.76**/**0.8459**/**0.0481**/**0.2071**|30.78/0.9296/0.0422/0.0534|\n\n### b. MRI reconstruction\n\nFastMRI dataset is used in this experiment, which has T2 and FLAIR contrast images.\nThe diffusion model released by CSGM[3] was pretrained on T2 contrast images. CSGM performs the best in the reconstruction of T2 contrast images except in the case without ACS lines. CSGM requires 1150 NFEs, while the other pre-trained models based methods[5,6] could achieve better performance even without ACS lines using less NFEs.\nWhen CSGM apply this pre-trained model to FLAIR images, its performance is not as good as T2 contrast images. However, this does not mean that SDI is superior to pre-trained models. Further comparison would lead to a more domain-specific discussion. We may not provide further comparisons during this rebuttal session.\n\n***Reconstruction quality comparison for T2 contrast***\n|Method|4×<br>PSNR↑/SSIM↑/NRMSE↓|6×<br>PSNR↑/SSIM↑/NRMSE↓|4× w/o ACS<br>PSNR↑/SSIM↑/NRMSE↓|\n|-|-|-|-|\n|IMJENSE|37.89/0.9650/0.0141|34.04/0.9434/0.0211|N/A|\n|Aseq|31.71/0.8867/0.0295|28.93/0.8423/0.0390|30.40/0.8693/0.0349|\n|DIP|37.66/0.9644/0.0147|33.85/0.9438/0.0214|36.05/0.9523/0.0201|\n|CSGM|**38.78**/**0.9693**/**0.0119**|**37.55**/**0.9636**/**0.0136**|34.00/0.9376/0.0208|\n|wo_noise_SDI|38.30/0.9683/0.0135|35.11/0.9519/0.0186|36.55/0.9543/0.0191|\n|SDI|38.08/0.9681/0.0131|35.83/0.9555/0.0166|**37.66**/**0.9620**/**0.0152**|\n\n\n***Reconstruction quality comparison for FLAIR contrast***\n|Method|4×<br>PSNR↑/SSIM↑/NRMSE↓|6×<br>PSNR↑/SSIM↑/NRMSE↓|4× w/o ACS<br>PSNR↑/SSIM↑/NRMSE↓|\n|-|-|-|-|\n|IMJENSE|37.68/0.9493/0.0144|34.63/0.9268/0.0201|N/A|\n|Aseq|31.09/0.8735/0.0307|28.33/0.8263/0.0416|30.0/0.8579/0.0345|\n|DIP|37.28/0.9473/0.0154|34.18/0.9217/0.0213|35.67/0.9389/0.0184|\n|CSGM|37.41/0.9435/0.0153|35.46/0.9286/0.0190|32.53/0.9015/0.0262|\n|w/o noise SDI|37.95/**0.9572**/0.0138|35.21/0.9367/0.0187|36.39/**0.9454**/0.0164|\n|SDI|**38.41**/0.9564/**0.0131**|**35.81**/**0.9387**/**0.0175**|**37.18**/0.9408/**0.0151**|\n\n\n## 3. Denoising task\nWe did denoising tasks on the test sets of CBSD68[7] with noise level $\\sigma = 25$ and the results are shown in the table below. Note that the total steps of IR-SDE[8] is 100, while the Denoising SDE/ODE only require 22 steps to recover the clean image.\n\n|Method|PSNR↑|LPIPS↓|SSIM↑|\n|-|-|-|-|\n|DIP|25.44|0.307|0.6602|\n|FFDNET|**31.22**|0.121|**0.8821**|\n|IR-SDE|28.09|**0.101**|0.7866|\n|SDI|28.10|0.207|0.7860|\n\n## 4. Apply SDI to a large image\nWe performed inpainting task on a noisy image of 2500$\\times$1500 pixels on a NVIDIA GeForce RTX 3090 Ti with 24GB memory, setting T=40, K=500. The training speed is 1.2it/s and the total time is around 4 hrs and 40 mins. The network has 88M parameters. Due to rebuttal rules, we cannot show the impressive inpainting results here. But we will include them in our revised paper.\n\n## 5. Why step-to-step instead of end-to-end?\nThe step-by-step approach in self-diffusion is motivated by the diffusion process’s ability to progressively refine estimates from low to high frequencies. This avoids premature convergence issues seen in single-step DIP and we explained it from the perspective of regulated spectral bias. Other unrolling network based methods adopt end-to-end training for a specific task and may be not have adaptability to diverse forward operators in inverse problems with a single pre-trained model.\n\n## 6. Ablation study\nWe did the ablation study by stopping the injection of noise at each time step and dubbed the method as \"wo noise SDI\". But we did not do ablation study for the U-Net used in SDI.\n\n## 7. Computational cost \nWe included a comprehensive runtime comparison in the table below. Methods like Aseq, DIP and SDI use the same U-Net of 88M parameters for all the tasks. The newly developed pre-trained model based method, DDNM, is very efficient, which requires only 100 iterations, while most DIP-based methods require thousands of iterations.\n\n|Method|Task, Image size|Iter/sec|Toatal iterations|GPU info|\n|-|-|-|-|-|\n|IMJENSE|MRI recon, 320x320|11.92|1500|RTX 4090,  24GB|\n|Aseq|MRI recon, 320x320|16.94|2000|RTX 3090 Ti, 24GB|\n|DIP|MRI recon, 320x320|16.94|2000|RTX 3090 Ti, 24GB|\n|CSGM|MRI recon, 320x320|11.39|1156|RTX 4090, 24GB|\n|wo noise SDI|MRI recon, 320x320|16.94|2000|RTX 3090 Ti, 24GB|\n|SDI|MRI recon, 320x320|16.94|2000|RTX 3090 Ti, 24GB|\n|DPS|low-level vision, 256x256|8.31|1000|RTX 4090, 24GB|\n|DDNM|low-level vision, 256x256|19.64|100|RTX 4090, 24GB|\n|SDI|low-level vision, 256x256|38.73|6000|RTX 3090 Ti, 24GB|\n|SDI|large image inpainting, 1500x2500|1.25|20000|RTX 3090 Ti, 24GB|\n\n## 8. Non-linear inverse problems\nWe are working on the non-linear inversion for MRI reconstruction using SDI. It is feasible, but we won't focus on it in this rebuttal. \n\n[1] H. Chung et al., \"Diffusion Posterior Sampling for General Noisy Inverse Problems,\" in ICLR, 2023.  \n[2] Wang, Yinhuai, Jiwen Yu, and Jian Zhang. \"Zero-shot image restoration using denoising diffusion null-space model.\" in ICLR, 2023.  \n[3] A. Jalal, et al., \"Robust Compressed Sensing MRI with Deep Generative Priors,\" in NIPS, 2021.  \n[4] Alkhouri, Ismail, et al. \"Image reconstruction via autoencoding sequential deep image prior.\" in NIPS, 2024.  \n[5] Luo, Guanxiong, et al. \"Bayesian MRI reconstruction with joint uncertainty estimation using diffusion models.\" MRM 90.1 (2023): 295-311.  \n[6] Huang, Shoujin, et al. \"Noise level adaptive diffusion model for robust reconstruction of accelerated mri.\" in MICCAI, 2024.  \n[7] Martin, David, et al. \"A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics.\" in ICCV 2001.  \n[8] Luo, Ziwei, et al. \"Image Restoration with Mean-Reverting Stochastic Differential Equations.\" in ICML, 2023."}, {"review_id": "zFI6LgIFOy", "reviewer": "Reviewer_UXaL", "summary": "The authors propose self-diffusion, a novel approach to image reconstruction which leverages a spectrum-regulated self-denoiser based on the Deep Image Prior framework. The authors establish a theoretical foundation for self-diffusion and empirically demonstrate its performance in MRI reconstruction and in other traditional inverse problems on ImageNet data.", "questions": "In addition to what I added to \"Strengths and Weaknesses,\" I wonder how sensitive SDI is to hyperparameter selection and how challenging it is to tune. Some plots showing the performance for different hyperparameter choices would be beneficial in understanding how sensitive SDI is.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper is very well-written and easy to follow.\n- To the best of my knowledge, this approach is novel, and builds on the very interesting work done in Deep Image Prior [1].\n- The authors evaluate SDI on a broad set of experiments where it performs quite well relative to baselines.\n\n**Weaknesses**\n- Qualitatively, I am not impressed by the visual quality of the reconstructions. I feel they looks too smooth. This observation is supported by quite high PSNR values in Tables 1 and 2. I suspect that, in spite of the spectral bias regularization, the network is not really able to produce high frequency detail well. This is in stark contrast to diffusion models which produce incredible high frequency details. There are two pieces needed to address this concern:\n    1. While I think it is unlikely that SDI would outperform a diffusion model, it is still important to understand the performance difference between the two approaches. I think that each experiment would be stronger with a diffusion baseline. For the ImageNet problems, I would recommend [2] and for the MRI problems I would recommend [3]. Neither are state-of-the-art, but both are trusted diffusion baselines. The inclusion of this comparison would allow readers to better understand how SDI performs more broadly.\n    2. Include an additional perceptual quality metric, such as LPIPS [4] or DISTS [5]. This is straightforward to do for the ImageNet problems, and I would suggest using VGG features for MRI since they align with radiologists' perceptions [6].\n- There is no runtime study. I think it is critical to understand how fast/slow SDI is. Please add runtime results to your tables for all methods.\n\n**Minor Issues**\n- The same sentence is repeated twice on Lines 142 and 143.\n- Best metrics should be bolded in the tables for better readability.\n\n[1] D. Ulyanov, A. Vedaldi, V. Lempitsky, \"Deep Image Prior,\" in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.\n\n[2] H. Chung et al., \"Diffusion Posterior Sampling for General Noisy Inverse Problems,\" in The Eleventh International Conference on Learning Representations , 2023.\n\n[3] A. Jalal, et al., \"Robust Compressed Sensing MRI with Deep Generative Priors,\" in Advances in Neural Information Processing Systems, 2021.\n\n[4] R. Zhang et al., \"The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,\" in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.\n\n[5] K. Ding et al., \"Image Quality Assessment: Unifying Structure and Texture Similarity,\" arXiv:2004.07728, 2020.\n\n[6] S. Kastryulin, J. Zakirov, N. Pezzotti, and D. V. Dylov, “Image quality assessment for magnetic resonance\nimaging,” arXiv:2203.07809, 2022.", "was_revised": true, "final_justification": "Thank you for addressing my concerns. After reading the other reviews and authors' responses, I find myself in agreement with reviewer rgwW, particularly on the point that the limited performance gap with great computational cost limits the interest in this method. I think that this work needs more development in order to show a clear benefit. For these reasons, I will keep my current score.", "rebuttal": "## 1. Comparison with pre-trained model based methods\nWe used DPS[1] and DDNM[2] to perform the low-level vision tasks and CSGM[3] to perform MRI reconstruction tasks on the datasets used in our submission. This serves as a comparison of the proposed algorithm with the pre-trained model based methods. The results are shown in the tables below. Generally, methods could recover high frequency details better with pre-trained prior models. Due to the rules of rebuttal, we cannot show the images for that, but we will include the results in the revised paper.\n\n### a. low-level vision tasks\nDDNM has the best performance over all tasks using only 100 network forward evaluations (NFEs) except for motion deblurring. But DDNM does not provide motion deblurring in their official reposity and we did not tune our implementation for that task thoroughly. Surprisingly, DPS does not perform close to the performance of DDNM using thousands of NFEs. This is consistent with the results that published in Aseq paper[4].\n\n| Method | 2× SR<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓ | 4× SR<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓ | Motion<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓ | Inpainting<br>PSNR↑/SSIM↑/NRMSE↓/LPIPS↓ |\n|--------|-------|-------|--------|------------|\n| Aseq | 27.26/0.8359/0.0531/0.2141 | 24.14/0.7535/0.0719/0.3238 | 22.65/0.6886/0.084/0.4007 | 28.06/0.8589/0.0516/0.1788 |\n| DIP | 29.06/0.853/0.0388/0.0994 | 24.55/0.6855/0.065/0.3046 | 25.94/0.7375/0.0589/0.2522 | 29.41/0.931/0.0384/0.0684 |\n| DPS | 22.68/0.6685/0.0895/0.3655 | 20.32/0.5516/0.1099/0.4599 | 21.84/0.5981/0.0943/0.3394 | 25.67/0.7634/0.0635/0.267 |\n| DDNM | **31.44**/**0.9199**/**0.031**/**0.0346** | **26.44**/**0.8117**/**0.0541**/**0.203** | 22.40/0.6838/0.0886/0.3998 | **34.04**/**0.9718**/**0.023**/**0.0173** |\n| w/o noise SDI | 26.97/0.8852/0.0543/0.0907 | 24.09/0.7583/0.0704/0.2992 | 26.92/0.8202/0.0513/0.2375 | 25.42/0.884/0.0804/0.1037 |\n| SDI | 30.3/0.9186/0.0351/0.0858 | 25.82/0.811/0.0585/0.2563 | **27.76**/**0.8459**/**0.0481**/**0.2071** | 30.78/0.9296/0.0422/0.0534 |\n\n### b. MRI reconstruction\n\nFastMRI dataset is used in this experiment, which has T2 and FLAIR contrast images.\nThe diffusion model released by CSGM[3] was pretrained on T2 contrast images. CSGM performs the best in the reconstruction of T2 contrast images except in the case without ACS lines. CSGM requires 1150 NFEs, while the other pre-trained models based methods[5,6] could achieve better performance even without ACS lines using less NFEs.\nWhen CSGM apply this pre-trained model to FLAIR images, its performance is not as good as T2 contrast images. However, this does not mean that SDI is superior to pre-trained models. Further comparison would lead to a more domain-specific discussion. We may not provide further comparisons during this rebuttal session.\n\n***Reconstruction quality comparison for T2 contrast***\n| Method | 4×<br>PSNR↑/SSIM↑/NRMSE↓ | 6×<br>PSNR↑/SSIM↑/NRMSE↓ | 4× w/o ACS<br>PSNR↑/SSIM↑/NRMSE↓ |\n|--------|--------|--------|--------|\n| IMJENSE | 37.89/0.9650/0.0141 | 34.04/0.9434/0.0211 | N/A |\n| Aseq | 31.71/0.8867/0.0295 | 28.93/0.8423/0.0390 | 30.40/0.8693/0.0349 |\n| DIP | 37.66/0.9644/0.0147 | 33.85/0.9438/0.0214 | 36.05/0.9523/0.0201 |\n| CSGM | **38.78**/**0.9693**/**0.0119** | **37.55**/**0.9636**/**0.0136** | 34.00/0.9376/0.0208 |\n| wo_noise_SDI | 38.30/0.9683/0.0135 | 35.11/0.9519/0.0186 | 36.55/0.9543/0.0191 |\n| SDI | 38.08/0.9681/0.0131 | 35.83/0.9555/0.0166 | **37.66**/**0.9620**/**0.0152** |\n\n\n***Reconstruction quality comparison for FLAIR contrast***\n| Method | 4×<br>PSNR↑/SSIM↑/NRMSE↓ | 6×<br>PSNR↑/SSIM↑/NRMSE↓ | 4× w/o ACS<br>PSNR↑/SSIM↑/NRMSE↓ |\n|--------|--------|--------|--------|\n| IMJENSE | 37.68/0.9493/0.0144 | 34.63/0.9268/0.0201 | N/A |\n| Aseq | 31.09/0.8735/0.0307 | 28.33/0.8263/0.0416 | 30.0/0.8579/0.0345 |\n| DIP | 37.28/0.9473/0.0154 | 34.18/0.9217/0.0213 | 35.67/0.9389/0.0184 |\n| CSGM | 37.41/0.9435/0.0153 | 35.46/0.9286/0.0190| 32.53/0.9015/0.0262 |\n| w/o noise SDI | 37.95/**0.9572**/0.0138 | 35.21/0.9367/0.0187 | 36.39/**0.9454**/0.0164 |\n| SDI | **38.41**/0.9564/**0.0131** | **35.81**/**0.9387**/**0.0175** | **37.18**/0.9408/**0.0151** |\n\n\n## 2. Running Time Comparison for different methods\nThank you for raising the concern about computation cost. We included a comprehensive runtime comparison in the table below. Methods like Aseq, DIP and SDI use the same U-Net of 88M parameters for all the tasks. The newly developed pre-trained model based method, DDNM, is very efficient, which requires only 100 iterations, while most DIP-based methods require thousands.\n\n\n| Method       | Task, Image size                  | Iter/sec | Toatal iterations  | GPU info         |\n|--------------|-------------------------    |----------|-------------------|-----------------------|\n| IMJENSE      | MRI reconstruction, 320x320 | 11.92    | 1500            | NVIDIA RTX 4090,  24GB  |\n| Aseq         | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| DIP          | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| CSGM         | MRI reconstruction, 320x320 | 11.39    | 1156            | NVIDIA RTX 4090, 24GB |\n| wo noise SDI | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| SDI          | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| DPS          | low-level vision, 256x256   | 8.31     | 1000            | NVIDIA RTX 4090, 24GB |\n| DDNM         | low-level vision, 256x256   | 19.64    | 100             | NVIDIA RTX 4090, 24GB |\n| SDI          | low-level vision, 256x256   | 38.73    | 6000            | NVIDIA RTX 3090 Ti, 24GB |\n| SDI          | large image inpainting, 1500x2500 | 1.25 | 20000         | NVIDIA RTX 3090 Ti, 24GB |\n\n## 3. Perception metrics\nWe add LPIPS metric for low-level vision tasks. But the source code of LPIPS for MRI images in the given reference is not available. We will find a suitable for the revised paper.\n\n## 4. Hyperparameter sensitivity\n\nThe table shows PSNR (dB) values for different $K$ (iterations) and $T$ (noise steps) combinations when reconstructing an MRI image from a 6x-undersampled kspace.\n\n| Step, T\\Iteration, K | 25    | 50    | 75    | 100       | 150       | 200       | 300       | 400       | 500       |\n|----------------------|-------|-------|-------|-----------|-----------|-----------|-----------|-----------|-----------|\n| T=10                 | 24.65 | 28.89 | 31.16 | 33.12     | 34.41     | 35.11     | **35.95** | **35.96** | **36.06** |\n| T=20                 | 29.35 | 33.27 | 34.04 | 35.30     | 35.69     | **36.15** | **36.33** | **36.02** | **36.24** |\n| T=40                 | 32.13 | 34.54 | 35.60 | **35.96** | **36.19** | **36.17** | **36.07** | **36.04** | **36.11** |\n\n### a. Observations\n1. **Trend with $K$**: For each fixed $T$, PSNR generally increases with $K$ up to a certain point, then plateaus or slightly declines. \n2. **Effect of $T$**: Higher $T$ values tend to yield higher PSNR at lower $K$, but the advantage diminishes as $K$ increases. \n3. **Plateau and Decline**: After reaching a peak (e.g., 36.33 at $K=300, T=20$), PSNR tends to stabilize or decrease slightl\n4. **Computational Cost**: Total iterations ($K \\times T$) range from 250 ($K=25, T=10$) to 20000 ($K=500, T=40$). Generally, the more iterations, the higher PSNR before plateauing.\n\nTherefore, when choosing hyperparameters, first select $K$ to balance performance and computational cost, then consider $T$ for optimal PSNR.\n\n### b. DIP Baselines\n\nThe DIP used in our submission is enhanced with TV regularization. The results in the table below show the PSNR (dB) and SSIM values for DIP with and without TV regularization, when performing the same MRI reconstruction task in the previous table. Without TV regularization, the network can overfit to the data very quickly, leading to low PSNR. When evaluating DIP on each dataset used in our submission, we first tune the hyperparameters for one sample and then use the same hyperparameters for all samples.\n\n| Method    | Iter=250 | Iter=500 | Iter=750 | Iter=1000 | Iter=2000 | Iter=3000 | Iter=4000 | \n|-----------|-------|-------|-------|-------|-------|-------|-------|\n| DIP w/ TV | 31.1610/0.8138 | 32.2094/0.8342 | 33.9467/0.8531 | 34.6047/0.8613 | 35.1507/0.8783 | 33.6402/0.7990 | 35.5463/0.8466 | \n| DIP w/o TV | 29.8827/0.8418 | 30.2350/0.8199 | 29.8380/0.7951 | 29.6209/0.7761 | 28.9668/0.6701 | 28.0941/0.5870 | 26.8794/0.5366 |\n\n|Method    | Iter=5000 | Iter=6000 | Iter=7000 | Iter=8000 | Iter=9000 | Iter=10000 |\n|-------|-------|-------|-------|-------|-------|-------|\n| DIP w/ TV  | 35.6637/0.8548 | 35.6827/0.8644 | 35.6032/0.8728 | 35.3603/0.8779 | 35.4510/0.8861 | 35.3384/0.8923 |\n| DIP w/o TV | 26.0204/0.5049 | 25.3416/0.4552 | 24.3752/0.4192 | 23.8758/0.4003 | 23.4355/0.3849 | 22.8755/0.3619 |\n\n[1] H. Chung et al., \"Diffusion Posterior Sampling for General Noisy Inverse Problems,\" in ICLR, 2023.  \n[2] Wang, Yinhuai, Jiwen Yu, and Jian Zhang. \"Zero-shot image restoration using denoising diffusion null-space model.\" in ICLR, 2023.  \n[3] A. Jalal, et al., \"Robust Compressed Sensing MRI with Deep Generative Priors,\" in NIPS, 2021.  \n[4] Alkhouri, Ismail, et al. \"Image reconstruction via autoencoding sequential deep image prior.\" in NIPS, 2024.  \n[5] Luo, Guanxiong, et al. \"Bayesian MRI reconstruction with joint uncertainty estimation using diffusion models.\" MRM 90.1 (2023): 295-311.  \n[6] Huang, Shoujin, et al. \"Noise level adaptive diffusion model for robust reconstruction of accelerated mri.\" in MICCAI, 2024."}, {"review_id": "DHx4stBj4r", "reviewer": "Reviewer_rgwW", "summary": "The authors propose a non-blind image restoration method based on a prior encoded in a diffusion-based network.\nMore precisely, the restoration algorithm minimizes a standard squared $L^2$ loss $\\|Ax-y\\|^2$ with $x$ being progressive reconstructions obtained by using a denoiser at several time steps of a diffusion model.\nThe minimization is run with respect to the parameters of the diffusion denoiser.\nThis restoration method can be seen as a generalization of the celebrated \"Deep Image Prior\" method, with an iterative alternative that is adapted to diffusion-based denoising architectures.\nThe proposed algorithm seems to give decent results on several inverse problems related to various imaging modalities.", "questions": "- Can the authors demonstrate whether the algorithm is robust to noise or not?\n\n- Can the authors give more details on the U-net architecture used for each considered inverse problem?\n\n- Why did the authors choose Aseq and IMJENSE for the comparisons? Can the authors say a few words on these two methods? Are they based on a neural network with a similar architecture? \n\n- How does this method compare to other restoration algorithms that are based on a pre-learned diffusion model (like DPS for example)? Also, for image deblurring or image super-resolution, one may also compare to some plug-and-play algorithm that also adopts a denoiser-based regularization.\n\n- Can the authors better explain why the \"curvature regularizer\" would \"discoura[ge] the network from fitting sharp, high-frequency fluctuations\"? \n\n- What do the authors mean by \"SDI without resampling noise\"? Is it that the same $\\mathcal{N}(0,I)$ is used for all $\\epsilon_t$'s?", "limitations": "- For each instance of an inverse problem, the proposed algorithm requires to optimize the parameters of the diffusion-based neural network from scratch. Thus it may involve much more computational resources than competing approaches (which may exploit a pre-learned denoiser or generative model).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# STRENGTHS\n\n- The algorithm does not rely on a pre-trained generative model. The only used prior is encoded in the architecture of the denoisers used.\n\n- The methodology is applied to several inverse problems in various applications (inpainting, super-resolution, motion deblurring, magnetic resonance imaging).\n\n\n# WEAKNESSES\n\n- The considered observation model contains no noise component. Thus the restoration method may be very unstable. For example, for image deblurring or image super-resolution, it is very unusual to showcase numerical results in the noiseless case. In my opinion, this is a major experimental flaw.\n\n- In order to reach good performance, the used denoising networks should have a large number of parameters, and thus the optimization algorithm is costly to run (and may require many iterations to converge).\nConsidering that several image foundational models are available nowadays, one can question why we should turn to such a methodology.\n\n- It seems that the paper does not contain comparison with respect to other restoration algorithms that are based on a pre-learned diffusion model (like DPS [14] for example).\n  In particular, the comparison in terms of computational time would be crucial in order to understand why one should favor the proposed methodology.\n\n- Convergence issues with respect to the denoiser parameters are not discussed, nor evaluated numerically.\n\n- I was not able to understand the relation with the SDE formulation, because I do not understand what the authors mean by \"piecewise stochastic process\".\n\n- The convergence statement on the reconstruction and its proof (given in Appendix A) seem shaky, since there are some hypothese that are somehow given along the proof.\n  See for example the sentence \"we analyze the behavior of the relative change error $e_t$ under idealized denoiser performance\" and following equation (3).\n  Besides, the proof at the end of page 11 uses that $x_t^{true} \\to x^{true}$ which is quite close to the desired conclusion.\n (This is actually difficult to discuss because the authors' claim is not stated as a theorem.)\n\n- There is almost no comment on the results obtained in Sections 4.1 and 4.3. For example, for the inpainting results, outside the mask, the image seems more regular than the input (see e.g. Fig 10 row 4).\n\n- The paper contains several sentences that are unclear or not well written. Examples:\n\n> Thereafter, supervised learning approaches based on deep neural networks have gained popularity that is used to learn mappings from large datasets of input-output pairs [5, 6].\n\n> the predicted solution converges to the true solution $x^{true}$ under the assumed circumstances of the problem\n\n> The measurements are then obtained as y = Ax. The measurements are then obtained as y = Ax.\n\n> The raw k-space is subsampled using a equispaced Cartesian mask with an acceleration factor of 4 or 6 along the phase-encoding direction, and 20 central auto-calibration signal (ACS) lines.\n\n> The undersampling mask is generated using BART’s command poisson with an acceleration factor of 2.5 along two phase-encoding directions.\n\n> This is a typical task that ground truth data is not available, and it can be formulated as an inverse problem y = Ax as well.\n\n> One thing deserves to be mentioned is that because the supposed output of our model is a 2D binary mask which can be easily thresholded to obtain the final point cloud, we modified the last layer of the U-Net to a sigmoid activation function.\n\n> Heckel and Soltanolkotabi (2020) claimed that there exists a unique true image $x$ that satisfies $Ax^{true} = y$ and untrained network possesses a structure (e.g., piecewise smoothness, low-frequency dominance) learnable by $D_{\\theta_t}$ .\n\n\n## Minor Remarks and Typos\n\n- There seems to be a typo in equation (1) in the index of the denoiser.\n\n- The notation $\\theta_{t,k}$ is not introduced in the text.\n\n- The initialization $x_T^{true} \\sim \\mathcal{N}(0,I)$ could be written again in Algorithm 1.", "was_revised": true, "final_justification": "I increased my score from 2 Reject to 4 Borderline Accept, because I find the proposed algorithm to be an interesting extension of DIP, and because the authors provided additional experiments that better highlight the behavior of the algorithm on a variety of inverse problems, and its robustness to noise.\nHowever, the performance gain seems moderate compared to previously existing methods, and I am not convinced by the SDE connection that is announced in Section 2.", "rebuttal": "## 1. Robustness to noise\nIn some well-regarded publications[1,2], the authors did tasks in the \"noiseless\" setting. In many cases, we obtain a degraded image $y$ that inherently includes noise and treated it as so-called \"noiseless\". Regarding the concerns of noise robustness, we did MRI reconstruction under different noise level. The results are shown in the table below.\n\n### MRI reconstruction under different noise level\nThe kspace data has a value range from 0 to 0.0055493987. We added Gaussian noise to the kspace data from 0.00001 to 0.00005. Due to change of rebuttal rules, we cannot further show the images for that, but we will include the results in our revised paper. \n|Noise Level|PSNR|SSIM|\n|-|-|-|\n|0.00001|32.71|0.6741|\n|0.00002|29.01|0.5531|\n|0.00003|26.69|0.4781|\n|0.00004|24.92|0.4071|\n|0.00005|23.85|0.3819|\n\n### Denoising task\nFurthermore, we included denoising results on the test sets of CBSD68[7] with noise level $\\sigma = 25$ here. \n\n|Method|PSNR↑|LPIPS↓|SSIM↑|\n|-|-|-|-|\n|DIP|25.44|0.307|0.6602|\n|FFDNET|**31.22**|0.121|**0.8821**|\n|IR-SDE[8]|28.09|**0.101**|0.7866|\n|SDI|28.10|0.207|0.7860|\n\nIn addition to the numerical metrics presented in the main paper, we provided extensive visual comparisons in the appendix across tasks and settings—including denoising, super-resolution, and inpainting—to offer a more complete picture of SDI’s performance.\n\n## 2. Comparison with pre-trained model based methods\nWe used DPS[1] and DDNM[2] to perform the low-level vision tasks and CSGM[3] to perform MRI reconstruction tasks on the datasets used in our submission. This serves as a comparison of the proposed algorithm with the pre-trained model based methods. Due to the limited space, please refer to results table under Reviewer UXaL.\n\n## 3. Why not use foundational models\nThe strength of SDI lies in its data-agnostic and pretrained-model-free nature, though it is computationally expensive. While pretrained models can offer powerful priors, they are often trained on specific datasets and may not generalize well across different modalities or data distributions. As demonstrated in the MRI reconstruction experiments, priors trained on T2-weighted images performed poorly when applied to FLAIR images, highlighting the risk of domain-specific bias.\n\nMoreover, foundational models are not always available or practical to use—particularly in domains like radar imaging or for very high-resolution images (e.g., 3000×1500 pixels), where pretrained solutions may be lacking or computationally infeasible. In contrast, SDI does not rely on external training data and can adapt directly to the observed image, making it especially useful for tasks in underrepresented or out-of-distribution scenarios.\n\nTherefore, SDI offers a valuable complement to pretrained models by providing a flexible, unbiased alternative when domain-specific priors are unavailable, ineffective, or inappropriate.\n\n## 4. Computational cost\nSDI achieves strong performance but at a relatively high computational cost. As shown in the table below, we provide a detailed runtime comparison across various methods and tasks. Methods like Aseq, DIP, and SDI all employ the same U-Net architecture with 88 million parameters.\n|Method|Task, Size|Iter/sec|Toatal iterations|GPU info|\n|-|-|-|-|-|\n|IMJENSE|MRI reconstruction, 320x320|11.92|1500|NVIDIA RTX 4090,  24GB|\n|Aseq|MRI reconstruction, 320x320|16.94|2000|NVIDIA RTX 3090 Ti, 24GB|\n|DIP|MRI reconstruction, 320x320|16.94|2000|NVIDIA RTX 3090 Ti, 24GB|\n|CSGM|MRI reconstruction, 320x320|11.39|1156|NVIDIA RTX 4090, 24GB|\n|wo noise SDI|MRI reconstruction, 320x320|16.94|2000|NVIDIA RTX 3090 Ti, 24GB|\n|SDI|MRI reconstruction, 320x320|16.94|2000|NVIDIA RTX 3090 Ti, 24GB|\n|DPS|low-level vision, 256x256|8.31|1000|NVIDIA RTX 4090, 24GB|\n|DDNM|low-level vision, 256x256|19.64|100|NVIDIA RTX 4090, 24GB|\n|SDI|low-level vision, 256x256|38.73|6000|NVIDIA RTX 3090 Ti, 24GB|\n|SDI|large image inpainting, 1500x2500|1.25|20000|NVIDIA RTX 3090 Ti, 24GB|\n\n## 5. Hyperparameters selection (convergence issues)\nThe table shows PSNR (dB) values for different $K$ (iterations) and $T$ (noise steps) combinations when reconstructing a MRI image (396x396) from a 6x-undersampled kspace.\n\n|Step, T\\Iteration, K|25|50|75|100|150|200|300|400|500|\n|-|-|-|-|-|-|-|-|-|-|\n|T=10|24.65|28.89|31.16|33.12|34.41|35.11|**35.95**|**35.96**|**36.06**|\n|T=20|29.35|33.27|34.04|35.30|35.69|**36.15**|**36.33**|**36.02**|**36.24**|\n|T=40|32.13|34.54|35.60|**35.96**|**36.19**|**36.17**|**36.07**|**36.04**|**36.11**|\n\n### Observations\n1. **Trend with $K$**: For each fixed $T$, PSNR generally increases with $K$ up to a certain point, then plateaus or slightly declines. \n2. **Effect of $T$**: Higher $T$ values tend to yield higher PSNR at lower $K$, but the advantage diminishes as $K$ increases. \n3. **Plateau and Decline**: After reaching a peak (e.g., 36.33 at $K=300, T=20$), PSNR tends to stabilize or decrease slightl\n4. **Computational Cost**: Total iterations ($K \\times T$) range from 250 ($K=25, T=10$) to 20000 ($K=500, T=40$). Generally, the more iterations, the higher PSNR before plateauing.\n\nTherefore, when choosing hyperparameters, first select $K$ to balance performance and computational cost, then consider $T$ for optimal PSNR.\n\n## 6. Structure of U-Net\nThe implementation of U-Net is from Aseq[4]. We will mention it in our reviesed paper.\n\n## 7. Piecewise stochastic process\nThe term *“piecewise stochastic process”* refers to how our self-diffusion procedure alternates between fixed-noise optimization steps and noise resampling across those steps. Specifically, within each noise step $t$, the Gaussian noise $\\epsilon_t \\sim \\mathcal{N}(0, I)$ is sampled once and held fixed for all $K$ gradient descent iterations of the denoiser. At the next step $t - 1$, the noise is resampled, which introduces stochasticity at the inter-step level. This leads to a process that is deterministic within each timestep but stochastic across time—a *piecewise stochastic* trajectory.\n\nIn the continuous-time limit, this behavior is modeled as a **stochastic differential equation (SDE)**:\n\n$$\nd\\mathbf{x}_t = (\\mathbf{x}_t - \\mathbf{x}^\\text{true}) dt + \\sigma(t) d\\mathbf{W}(t),\n$$\n\nwhere the Wiener process $\\mathbf{W}(t)$ captures the noise resampling. The drift term encourages convergence toward $\\mathbf{x}^\\text{true}$, while the diffusion term captures the structured noise injection.\n\n## 8. Convergence statement\nWe agree that the assumptions (e.g., idealized denoiser performance, bounded iterate norms, small denoiser error) were introduced informally. In the revised version, we will state these assumptions explicitly before the derivation and clearly separate them from the proof steps to clarify the argument’s scope and structure. We also agree that the convergence result should be formally stated as a theorem or proposition. Our intent was to provide intuition under a simplified setting rather than a fully general proof. We will revise the text to reflect this and clarify that the analysis illustrates plausible convergence behavior under idealized, interpretable conditions.\n\n## 9. Discussion on results\nWe agree that our discussion of the results, particularly in Sections 4.1 and 4.3, was limited. We appreciate the reviewer’s observation about the inpainting results—indeed, in some cases (e.g., Fig. 10, row 4), we could obeserve that SDI understand the image better.  \nIn the revised version, we will also expand the discussion to cover computational cost, hyperparameter selection, and the comparison with pretrained-model-based approaches—topics already addressed in this rebuttal.\n## 10. Unclear sentences\n\n1. Thereafter, supervised learning approaches, since deep neural networks have gained popularity, are used to learn mappings from large datasets of input-output pairs [5, 6].\n\n2. the predicted solution converges to the true solution under the Theorem 1 in Heckel and Soltanolkotabi (2020)\n\n3. This sentence includes many domain-specific terms. We are sorry we could not explain one by one due to the limited space. For more details, please refer to www.mriquestions.com .\n    \n    `The ... 4x/6x along the phase-encoding.... ACS lines are set to 20. The... directions.`\n   \n4. Radar detects barrier and non-barrier in its coverage area.\n   \n   `This is a typical... U-Net to a sigmoid activation function.`\n\n5. Heckel and Soltanolkotabi (2020) showed that if there exists a unique true image consistent with the measurements, then an untrained network can recover it by leveraging structural properties such as piecewise smoothness and low-frequency dominance, which are implicitly captured by the network during optimization.\n## 12. Answer to remaining questions\n1. Aseq[4] and IMJENSE[11] are DIP-based methods, published in NIPS and IEEE TMI respectively. They does not rely on pretrained priors.\n2. The curvature regularizer arises from the second-order term in the Taylor expansion of the expected loss under input noise (see Appendix C.1). This term takes the form: $$\\sigma(t)^2 \\cdot \\mathrm{Tr}(A \\nabla^2 D_{\\theta}(x_t) A^\\top),$$  \nwhere $\\nabla^2 D_{\\theta}(x_t)$ is the Hessian of the denoiser with respect to its input. Intuitively, this trace penalizes the curvature of the network’s output along directions relevant to the forward operator $A$. In the Fourier domain (Appendix C.2), high-frequency components correspond to larger second derivatives, i.e., higher curvature. Therefore, the regularizer imposes a stronger penalty on high-frequency content.\n3. We stopped the injection of noise at each time step and dubbed the method as \"wo noise SDI\".\n## 12. Minors\nWe corrected the typos in the text, introduced notation $\\theta_{t,k}$, and also included $x_T^{true} \\sim \\mathcal{N}(0,I)$ in Algorithm 1. \n## 13. Some Refs listed under Reviewer kzzf.   \n[9] R. Feng et al., \"IMJENSE: Scan-Specific Implicit Representation for Joint Coil Sensitivity and Image Estimation in Parallel MRI,\" in IEEE TMI, 2023."}, {"review_id": "i0RlwkB1Cs", "reviewer": "Reviewer_HCfK", "summary": "This paper presents an unsupervised method for solving inverse problems using a diffusion-based approach that does not rely on pre-trained generative models. Instead, the solution is estimated directly from the observed data through an iterative framework that integrates denoising steps with updates to the denoiser’s parameters.", "questions": "## Suggestions\n\n- Consider including an ablation or sensitivity study on the effect of the number of iterations ($K$) and noise steps ($T$) to provide insight into how these parameters influence reconstruction quality.\n\n- To support the claim regarding multiscale behavior, a visual or quantitative analysis showing how image structures evolve across diffusion steps would be valuable.", "limitations": "The authors highlight potential improvements to their method at the end of the paper, while not explicitly discussing limitations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strengths\n\n- The proposed method is novel in that it uses diffusion models without requiring ground truth data and operates in a self-contained manner, akin to Deep Image Prior. The experimental results are promising and demonstrate the potential of the approach.\n\n- The paper is clearly written, with thoughtful explanations of both the methodology and the experimental setup. It is accessible and well-organized, making it easy to follow.\n\n\n### Weaknesses\n\n- The impact of the number of iterations ($K$) and diffusion noise steps ($T$) on performance is not fully explored. While Figure 9 refers to this, a more systematic empirical study would strengthen the work.\n\n- The paper claims that early denoising steps capture smooth, low-frequency structures, while later steps refine high-frequency details—mirroring multiscale optimization. However, this interesting observation is not supported by direct empirical evidence or visualizations.", "was_revised": true, "final_justification": "After reading the authors rebuttal and other reviews, i think the paper has value in the field of unsupervised inverse problems resolution, and deserves to be published. I agree with the concern of one other reviewer that the paper should include a study of their method behavior in a noisy setting, but I'm fine with the paper not comparing to methods which use ground truth data for training, like PnP or unrolled algorithms. Thus, I keep my rating as is and recommend acceptance.", "rebuttal": "## Hyperparameter selection\n\nThanks for raising the importance of hyperparameter selection. The table shows PSNR (dB) values for different $K$ (iterations) and $T$ (noise steps) combinations when reconstructing a MRI image from a six-times undersampled kspace (396x396x16).\n\n| Step, T\\Iteration, K | 25    | 50    | 75    | 100       | 150       | 200       | 300       | 400       | 500       |\n|----------------------|-------|-------|-------|-----------|-----------|-----------|-----------|-----------|-----------|\n| T=10                 | 24.65 | 28.89 | 31.16 | 33.12     | 34.41     | 35.11     | **35.95** | **35.96** | **36.06** |\n| T=20                 | 29.35 | 33.27 | 34.04 | 35.30     | 35.69     | **36.15** | **36.33** | **36.02** | **36.24** |\n| T=40                 | 32.13 | 34.54 | 35.60 | **35.96** | **36.19** | **36.17** | **36.07** | **36.04** | **36.11** |\n\n### Observations\n1. **Trend with $K$**: For each fixed $T$, PSNR generally increases with $K$ up to a certain point, then plateaus or slightly declines. \n2. **Effect of $T$**: Higher $T$ values tend to yield higher PSNR at lower $K$, but the advantage diminishes as $K$ increases. \n3. **Plateau and Decline**: After reaching a peak (e.g., 36.33 at $K=300, T=20$), PSNR tends to stabilize or decrease slightl\n4. **Computational Cost**: Total iterations ($K \\times T$) range from 250 ($K=25, T=10$) to 20000 ($K=500, T=40$). Generally, the more iterations, the higher PSNR before plateauing.\n\nTherefore, when choosing hyperparameters, first select $K$ to balance performance and computational cost, then consider $T$ for optimal PSNR.\n\n## Multiscale Behavior\nThe multiscale behavior is supported by the supplementary materials for 1D simulation experiment in Section 3, which includes animation illustrating the spectra evolution. We cannot further show Fourier spectra evolution of 2D image to quantify low-to-high frequency reconstruction due to the new rules of rebuttal, but we will include it in our revised paper.\n\n## Limitation: computational cost\n\nOne limitation we will mentioned is that the performance of the proposed algorithm comes with a high computational cost. We included a comprehensive runtime comparison in the table below. Methods like Aseq, DIP and SDI use the same U-Net of 88M parameters for all the tasks. The newly developed pre-trained model based method, DDNM, is very efficient, which requires only 100 evaluations, while most DIP-based methods require thousands of NFEs.\n\n| Method       | Task, Image size                  | Iter/sec | Toatal iterations  | GPU info         |\n|--------------|-------------------------    |----------|-------------------|-----------------------|\n| IMJENSE      | MRI reconstruction, 320x320 | 11.92    | 1500            | NVIDIA RTX 4090,  24GB  |\n| Aseq         | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| DIP          | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| CSGM         | MRI reconstruction, 320x320 | 11.39    | 1156            | NVIDIA RTX 4090, 24GB |\n| wo noise SDI | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| SDI          | MRI reconstruction, 320x320 | 16.94    | 2000            | NVIDIA RTX 3090 Ti, 24GB |\n| DPS          | low-level vision, 256x256   | 8.31     | 1000            | NVIDIA RTX 4090, 24GB |\n| DDNM         | low-level vision, 256x256   | 19.64    | 100             | NVIDIA RTX 4090, 24GB |\n| SDI          | low-level vision, 256x256   | 38.73    | 6000            | NVIDIA RTX 3090 Ti, 24GB |\n| SDI          | large image inpainting, 1500x2500 | 1.25 | 20000         | NVIDIA RTX 3090 Ti, 24GB |"}]

Author rebuttal:

Meta-review: The paper introduces self-diffusion for solving inverse problems. Here, the goal is to recover a target signal $x^{true}$ from observations $y = A x^{true}$. The paper proposes a diffusion based approach, where to reconstruct a given sample one trains a sequence of denoisers $D_{\theta_t}$ with a cost that measures the mean squared measurement error between the observations y and the projected denoiused signal. This setup is conceptually similar to deep image prior, albeit with the object of training as an iterative denoiser rather than a general neural network. Similar to deep image prior, regularization is induced by the implicit prior encoded in the network architecture. The paper argues that the combination of neural network training and this noise schedule induce a spectral bias towards low frequencies, regularizing fitting and reconstruction. The paper illustrates the approach through synthetic frequency-sparse signal reconstruction, 2D and 3D MR reconstruction, radar estimation angle recovery and image inpainting, reporting favorable results vis deep image prior based reconstruction methods such as DIP, Aseq and IMJENSE. Comparisons to additional pre-trained model baselines were added during the rebuttal and discussion phase, robustifying the comparison to the literature. 

The reviewers generally acknowledge the novelty of the approach, namely, solving inverse problems via a self-diffusion process trained on the given instance, regularized by the architectural bias of the denoiser. Reviewers raised a number of questions regarding the performance and computational cost of the proposed method, as well as its comparison to pretrained baselines. The authors response to review comments robustifies the experiments, showing quantitative performance that is generally comparable with the best of the considered models and in some cases the best (e.g., in MR reconstruction with FLAIR contrast). The response also addresses the issue of computational complexity, showing comparable performance to other DIP-based methods.
Decision: Accept (poster)