Stochastic Deep Restoration Priors for Imaging Inverse Problems
Authors: Yuyang Hu
Albert Peng
Weijie Gan
Peyman Milanfar
Mauricio Delbracio
Ulugbek S. Kamilov
Venue: ICML 2025 poster (2025)
Keywords: Inverse Problems
Image Reconstruction
Restoration Priors
Primary area: optimization
TL;DR: We introduce ShaRP, a novel framework that stochastically leverages an ensemble of deep restoration models beyond denoisers to regularize inverse problems.

Abstract: Deep neural networks trained as image denoisers are widely used as priors for solving imaging inverse problems. We introduce Stochastic deep Restoration Priors (ShaRP), a novel framework that stochastically leverages an ensemble of deep restoration models beyond denoisers to regularize inverse problems. By using generalized restoration models trained on a broad range of degradations beyond simple Gaussian noise, ShaRP effectively addresses structured artifacts and enables self-supervised training without fully sampled data. We prove that ShaRP minimizes an objective function involving a regularizer derived from the score functions of minimum mean square error (MMSE) restoration operators. We also provide theoretical guarantees for learning restoration operators from incomplete measurements. ShaRP achieves state-of-the-art performance on tasks such as magnetic resonance imaging reconstruction and single-image super-resolution,  surpassing both denoiser- and diffusion-model-based methods without requiring retraining.

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Yuyang Hu?
Albert Peng'
Weijie Gan
Peyman Milanfar?
Mauricio Delbracio* Ulugbek S. Kamilov
Abstract
problems (Romano et al., 2017; Kadkhodaie & Simoncelli, 2021; Zhang al., 2022). The combination of pre-trained Gaussian denoisers with measurement models has been shown to be effective in many inverse problems, including image super-resolution, deblurring, and medical imaging (see the recent reviews (Ahmad et al., 2020; Kamilov et al., 2023; Milanfar & Delbracio, 2024; Daras al., 2024)). This success has led to active research on novel methods based denoiser priors, their theoretical analyses, statistical inter- pretations, as well connections to related approaches such as score matching and diffusion models (Venkatakrishnan et al., 2013; Chan et al., 2017; Romano et al., 2017; Buzzard et al., 2018; Reehorst & Schniter, 2019; Sun et al., 2019; Sun et al., 2019; Ryu et al., 2019; Cohen et al., 2021; Hu- rault et al., 2022b; Laumont et al., 2022: Gan et al., 2023a; Renaud et al., 2024; Xiao et al., 2024; Nathan et al., 2024).
Deep neural networks trained as image denois- ers are widely used as priors for solving imag- ing inverse problems. We introduce Stochastic deep Restoration Priors (ShaRP), a novel frame- work that stochastically leverages ensemble of deep restoration models beyond denoisers to regularize inverse problems. By using general- ized restoration models trained on a broad range of degradations beyond simple Gaussian noise, ShaRP effectively addresses structured artifacts and enables self-supervised training without fully sampled data. We prove that ShaRP minimizes an objective function involving regularizer derived from the score functions of minimum mean square error (MMSE) restoration operators. We also pro- vide theoretical guarantees for learning restora- tion operators from incomplete measurements. ShaRP achieves state-of-the-ar performance on tasks such as magnetic resonance imaging recon- struction and single-image super-resolution, sur- passing both denoiser- and seffusioon-mdee-baee based methods without requiring retraining.
Although priors based on Gaussian denoising models have been extensively studied, there is little research on leverag- ing priors from pre- pre-trained restoration models that extend beyond Gaussian denoisers. In this paper, we present evi- dence that priors derived from deep models pre-trained as general restoration operators can surpass those trained exclu- sively for Gaussian denoising. We introduce novel frame- work called Stochastic deep Restoration Priors (ShaRP), which provides principled approach to integrate an ensem- ble of general restoration models as priors to regularize in- verse problems. By using more versatile restoration models, ShaRP improves upon traditional methods using Gaussian denoiser priors in two key ways: (a) ShaRP improved per- formance by using restoration models that are better suited for mitigating non-Gaussian structured artifacts arising dur- ing inference. (b) The restoration models in ShaRP can sometimes be directly trained in self-supervised manner without fully-sampled measurement data.
1. Introduction
Many problems in computational imaging, biomedical imag- ing, and computer vision can be viewed as inverse problems, where the goal is to recover an unknown image from its noisy and incomplete measurements. Inverse problems are typically ill-posed, thus requiring additional prior in- formation for accurate image reconstruction. While many approaches have been proposed for implementing image priors, the current research focuses on methods based on deep learning (DL) (McCann et al., 2017; Ongie et al., 2020; Kamilov et al., 2023; Wen et al., 2023).
We present novel theoretical and numerical results highlight- ing the potential of using an ensemble of restoration models as image priors. Our theoretical result introduces novel notion of regularization for inverse problems corresponding to the average of likelihoods associated with the degraded observations of an image. The proposed regularizer has an intuitive interpretation as promoting solutions whose multiple degraded observations resemble realistic degraded images. We show that ShaRP seeks to minimize an objective function containing this regularizer. Our second theoreti-
Deep neural networks trained image denoisers are widely- used for specifying image priors for solving general inverse
"aashU Gooole. Correspondence to: Ulugbek S. Kamilov < edu'..
Proceedings of the 4rnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
cal result analyzes the possibility of learning the minimum mean squared error (MMSE) restoration operators directly from noisy and undersampled measurements, We numeri- cally show the practical relevance of ShaRP by applying it to MRI reconstruction with varying undersampling patterns and rates, using a fixed-rate pre-trained MRI reconstruction network as a prior. We also show that ShaRP can use a pre-trained image deblurring model to perform single image super-resolution (SISR). Our numerical experiments show that ShaRP adapts the pre-trained restoration model as prior, outperforming existing methods based on image de- noisers and diffusion models, and achieving state-of-the-art results. Our experiments additionally highlight the benefit of using restoration models as priors by considering a setting where only undersampled and noisy MRI data is available for pre-training the prior. In such cases, self-supervised training of restoration model is feasible, whereas training a Gaussian denoiser requires fully sampled data.
(CSGM), and deep equilibrium models (DEQ) (Bora et al., 2017; Romano al., 2017; Zhang & Ghanem, 2018; Haupt- mann et al., 2018; Gilton et al., 2021a; Liu et al., 2022; Hu et al., 2024d). These approaches come with different trade-offs in terms of imaging performance, computational and memory complexity, flexibility, need for supervision, and theoretical understanding.
Denoisers as Priors. Score-based models (SBMs) are a powerful subset of DL. methods for solving inverse problems that use deep Gaussian denoisers imaging priors. Plug- and-Play (PnP) methods can be viewed as SBMs that in- corporate denoisers within iterative optimization algorithms (see recent reviews (Ahmad et al., 2020; Kamilov et al., 2023)). These approaches construct a cost function by com- bining an explicit likelihood with score function implicitly defined by the denoiser prior. Over the past few years, numerous variants of PnP have been developed (Venkatakr- ishnan et al., 2013; Romano et al., 2017; Metzler et al., 2018; Dong et al., 2019; Zhang et al., 2019; Wei et al., 2020; Hurault et al., 2022a), which has motivated an extensive research into their theoretical properties and empirical ef- fectiveness (Chan al., 2017; Buzzard al., 2018; Ryu et al., 2019; Sun al., 2019; Tirer & Giryes, 2019; Teodoro et al., 2019; Sun al., 2021: Cohen et al., 2021; Fang et al.,
2. Background
Inverse Problems. Many computational imaging tasks can be formulated as inverse problems, where the goal is to reconstruct unknown image IR. from its corrupted measurement
2024; Renaud et al., 2024; Hu et al., 2024a; Terris et al.,
y Ax e,
(1)
2024). Diffusion Models (DMs) represent another category of SBMs; they are trained to learn the score function of the underlying probability distribution governed by stochastic differential equations (SDEs) (Ho et al., 2020; Song et al., 2021). Once trained, these models can be used as powerful priors for inverse problems by leveraging their learned score functions. Specifically, pre-trained DMs facilitate poste- rior sampling by guiding the denoising process to generate data consistent with observed measurements. This approach enables DMs to address inverse problems, often achieving impressive perceptual performance even for highly ill pooep inverse problems (Chung et al., 2023; Zhu al., 2023; Wang al., 2023; Feng et al., 2023; Sun et al., 2024; Wu et al., 2024; Song et al., 2024; Hu et al., 2024b; Alçalar & Akçakaya, 2024; Zhao et al., 2024; Rout et al., 2024; Bai et al., 2024; Mardani et al., 2024).
where A E RR is measurement operator and I is the noise. A common approach to addressing inverse problems is to formulate them as an optimization problem
E arg min f(x) with f(x) = g(x)) + M(x)
(2)
=ER"
where g is the data-fidelity term that quantifies the fit to the measurement y and h is a regularizer that incorporates prior information on x. For instance, typical functions used in imaging inverse problems are the reastesquares term g(x) = 1 Ax- - 2 and the total variation (TV) regular- izer h(x) = T Dæ || i. where D is the image gradient and 0 is a regularization parameter.
powerful tool for addressing inverse problems (McCann et al., 2017; Ongie et al., 2020; Wen et al., 2023). Instead of explicitly defining a regularizer, DL methods use deep neural networks (DNNs) to map the measurements to the desired images (Wang et al., 2016; Jin et al., 2017; Kang et al., 2017: Chen et al., 2017: Delbracio et al., 2021; Delbracio & Milanfar, 2023). Model- based DL (MBDL) is widely-used sub-family of DL al- gorithms that integrate physical measurement models with priors specified using CNNs (see reviews by (Ongie et al., 2020; Monga et al., 2021)). The literature of MBDL is vast, but some well-known examples include huu-and--lay priors (PnP), regularization by denoising (RED), deep un- folding (DU), compressed sensing using generative models
Deep Learning. DL. has emerged
Restoration Networks as Priors. addition to denoiser- based methods, recent work has also considered using restoration models as implicit priors for solving inverse problems (Zhang et al., 2019; Liu et al., 2020; Gilton et al., 2021b; Hu et al., 2024c). It has been observed that pre- trained restoration models can be effective priors for ad- dressing unseen inverse problems, sometimes surpassing traditional denoiser- approaches (Hu et al., 2024c). However, existing methods present two main limitations. First, existing restoration priors have relied on fixed prior tailored to specific degradation. Although deep restoration models can be trained in various settings-suce as different
2

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
(a) Prior trained for degradation set
(b) ShaRP for target Problem A
H R-1
restoration
R
R
Gradient Step
H.z +
Iteration h
Figure 1: A restoration network trained on set of tasks (H;) can be used prior within ShaRP address different larget inverse problems without the need for retraining.
blur kernels for image deblurring or diverse undersampling masks for MRI reconstruction-cuume current approaches do not leverage this capability, limiting their robustness to diverse artifacts. Second, prior work has not explored the potential of learming restoration priors directly from undersampled measurements, without access fully sampled data. Unlike Gaussian denoisers, training without fully sampled data is natural for restoration models (Ulyanov et al., 2018; Krull et al., 2019; Quan et al., 2020; Yaman et al., 2020; Liu et al., 2020; Tachella et al., 2022; Chen et al., 2022; Millard & Chiew, 2023; Gan et al., 2023b; Terris Moreau, 2023; Gossard & Weiss, 2024). It also worth highlighting the re- lated work that has explored using corrupt measurements for training Ambient DMs (Daras et al., 2023; Aali et al., 2024). These Ambient DMs, trained directly on such undersampled measurements, address the reconstruction problem by sam- pling from their approximated posterior distribution using a standard diffusion sampler inference. ShaRP, conversely, adopts an optimization approach, defining novel objec- tive function from an ensemble of likelihoods of multiple degraded observations.
Algorithm Stochastic deep Restoration Priors (ShaRP)
1: input: Initial value x. E F. y 0, >>0 0
0, and
2: for k = 1,23 do
3: Sample a degradation operator: H ~ p(H)
4:
8 + Ha* + with n ~ -V(00
5:
x² + x== += - 10-11~0f(aa -(
with ff(x) = VgA(x) Vh(x)
where Vh(x)= Js H R(s,
6: end for
multiple restoration priors instead of relying on a single one. (2) We introduce novel regularization concept for inverse problems that encourages solutions that produce de- graded versions closely resembling real degraded images. For example, our regularizer favors an MR image solution only if its various degraded versions are consistent with the characteristics of actual degraded MR images. (3) We show that ShaRP be interpreted stochastic gradient method that seeks to minimize composite objective that incorporates our proposed regularizer. We discuss its con- vergence for both exact and approximate MMSE restoration operators. We also provide theoretical guarantees on learn- ing restoration operators from incomplete measurements. (4) We implement ShaRP with both supervised and self- supervised restoration models as priors and test on two inverse problems: compressed sensing MRI (CS-MRI) and single-image super-resolution (SISR). Our results highlight the capability of restoration models to achieve state-of-the- art performance. Notably, in the MRI context, we show that restoration networks trained directly on subsampled and noisy MRI data can serve as effective priors, scenario where training traditional Gaussian denoisers is infeasible.
Method
Prior Type
Prior Configuration
PnP
Only Gaussian Denoiser
Single Prior
SNORE
Only Gaussian Denoiser
Ensemble of Priors
DRP
General Restoration
Single Prior
ShaRP
General Restoration
Ensemble of Priors
Can include denoisers and extends other restoration tasks.
Table 1: ShaRP differs from existing methods by offering greater flexibility through the use of an ensemble of pri- ors trained on general restoration tasks, extending beyond Gaussian denoising priors.
Our contribution. (1) We propose ShaRP, new frame- work for solving inverse problems leveraging set of priors implicit in pre-trained deep restoration network. As sum- marized in Table 1, ShaRP generalizes Regularization by Denoising (RED) (Romano et al., 2017) and Stochastic De- noising Regularization (SNORE) (Renaud et al., 2024) by using more flexible restoration operators and generalizes Deep Restoration Priors (DRP) (Hu al., 2024c) by using
3. Stochastie Deep Restoration Priors
ShaRP is presented in Algorithm 1. It considers a prior based on a deep restoration model R(s, H) pre-trained using the family of degradation operators, such as blur kernels or MRI masks. More specifically, the deep restoration model
3

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
is trained to solve the following set of restoration problems
The function Go in (5) denotes the Gaussian density func- tion with the standard deviation
0.
Har + with
Pas H (HH;
(3)
We propose the ShaRP regularizer
where is the AWGN vector with variance o² and Pa denotes the probability distribution of the target images, and P(H) is the probability density of considered degradation operators. Importantly, the restoration problems (3) are used exclusively for training R and do not need to match the target inverse problem (I), which involves the measurement operator A.
h(x) TB..-., (a)) (-logi/-oooaaoo
(6)
where > is the regularization parameter and p(H) is the distribution of considered degradations. The regularizer h is minimized when degraded versions of æ are highly probable under the distribution p(s[H), with H sampled from p(H). In other words, solution é is considered effective if its degraded versions H& align with the degraded versions Hr of clean images x ~ Pw: for all H ~ p(H). This can be understood as searching for a fixed point that exhibits equivariance across multiple degradations. The key benefit of the proposed regularizer in (6) lies in its versatility, allowing wide range of degradation operators to be used in unified framework. In particular, this formulation remains compatible with Gaussian denoisers, as H = I can always be incorporated into p(H).
We prove below that ff corresponds stochastic approx- imation of an objective function of the form ʃ g + h. Here, (xx = A vll2 and the term h(xx) is defined as h(x) = TEg-G. (a- Har HH) (-log p(s H)].
. We will
further introduce the derivation and interpretation of h(z) in Section 4.1. Similar to traditional stochastic gradient meth- ods. ShaRP can be implemented using various selection strategies for the degradation operators.
Each iteration of ShaRP has an intuitive interpretation, where the next solution is obtained by combining the gra- dient of the data-fidelity term Vg and the residual of re- stored image corresponding to the selected degradation op- erator. worth highlighting that ShaRP compatible with multiple priors, each restoring an image from Gaussian or non- Gaussian degradations. Also, the computational cost of running ShaRP is comparable to those of single-moder approaches. This is due to the stochastic nature of our algo- rithm that uses a restoration operator in each iteration.
We are now ready to state our first theoretical result.
Theorem 1. Assume that the prior density Pa is non- degenerate over IR" and let R be the MMSE restoration operator (4) corresponding to the restoration problems (3). Then, we have that
= T H (E H))))))) 02
Vh(x)=
(7)
4. Theoretical analysis of ShaRP
where is the ShaRP regularizer in (6), the expectation with respect
Go(s -Hx) and H PH-
We now present several theoretical results on ShaRP. Our main contribution is the expression for the closed-form regu- larizer minimized by ShaRP. We also show how to interpret ShaRP as a stochastic gradient method, enabling its con- vergence analysis when using inexact MMSE operators. We finally prove theoretical guarantees on learning MMSE restoration operators from unsersampled measurements.
The proof is in the appendix. Note that the expression within the square parenthesis in (7) matches the ShaRP update in Line 4 of Algorithm which directly implies that ShakP using the exact MMSE restoration operator R* is stochastic gradient method for minimizing f = g + h, where g is the data-fidelity term and h is the ShaRP regularizer in (6).
4.1. Explicit formula for regularizer
4.2. Theoretical convergence of ShaRP
Consider a restoration model that perform MMSE estima- tion of æ E R" for problems (3)
Given the explicit regularized in (6), the iterations of ShaRP in Algorithm I can be seen as stochastic gradient method for minimizing f = g + h. In practical scenarios, the learned restoration model may be imperfect, meaning it cannot be considered perfect MMSE estimator. We now present the convergence analysis of ShaRP under a restoration operator R that approximates the true MMSE restoration operator R*. We adopt the analysis of biased stochastic gradient descent (SGD) from the optimization literature (Bertsekas, 2011; Ghadimi & Lan, 2016; Demidovich et al., 2023), by interpreting iterations of ShaRP as a variant of biased
/ -f /xp(( p(x
=
(" H) = E x
8. H.
s, H) dæ
(4)
1
x G (x()) dæ.
p(s H)
where we used the probability density of the observation s conditioned on the operator H
p(s(H)= / Go H )a (x) dx.
(5)
SGD. For a given degraded observation Hæ + n with

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
H Ph and (N II),, we define the stochastic gradient used by ShaRP
The general analysis of biased stochastic SGD has been ex- tensively discussed in the optimization literature (Bertsekas, 2011; Ghadimi & Lan, 2016; Demidovich et al., 2023). Our contribution here is the relationship between the iterates of ShaRP and those of biased SGD, leading to theoretical guarantees on the stability of ShaRP using restoration net- works that do not correspond to ideal MMSE estimators. Proposition states that in expectation. ShaRP minimizes the norm of the gradient V up an error term that has two components, 7Lv² and e². Since the first component de- pends y, it can be made as small as desired by controlling the step-size 7. The second component only depends on the magnitude of the bias E, which, in turn, directly depends on the accuracy of the restoration operator relative to the true MMSE restoration operator R
f(x) Vg(x) + Vh(x),
(8)
where
Vh(z) = H'H(< R(s, H))
Since R is an inexact MMSE restoration operator, we also define the bias vector
b(x) - M H HR" R(s, H))
(9)
which quantifies the average difference between the exact and inexact MMSE restoration operators with respect to H PH and (s Ha:). The analysis requires three assumptions that jointly serve sufficient conditions for our theorem.
Assumption 1. The function f has finite minimum f* >> -oo and the gradient f is Lipschitz continuous with con-
4.3. Learning restoration priors without groundtruth
In this subsection, we present a theoretical result that estab- lishes the feasibility of learning an MMSE estimator from undersampled measurements. Let the undersampled mea- surements be defined as Hæ + n, where H = PM. Here, P represents binary subsampling matrix, M de- notes the square transfer operator, and n corresponds to the noise vector.
stant L > O.
This is a standard assumption used in the analysis of gradient-based algorithms (see (Nesterov, 2004), for exam- ple). is satisfied by large number of functions, including the traditional leaat-squares data-fidelit, function.
Assumption 2. The stochastic gradient has bounded vari- ance for all x E R", which means that there exists a con- stant > 0 such that
To show that an MMSE estimator can be learned from under- sampled measurement, we need the following assumption.
Assumption 4. Ep PP has full rank where the expec- tation is taken over pp. The measurement operator M is orthogonal matrix.
(((a) -E [ [V) S
E
where expectations are with respect to H ~ PH and Go(s- - Hz).
This assumption implics that the union of all sampling ma- trices P spans the complete measurement domain, even though each individual P may remain undersampled.
This is another standard assumption extensively used in the analysis of online or stochastic optimization algo- rithms (Bertsekas, 2011; Ghadimi Lan, 2016; Demidovich al., 2023).
Theorem 2.
Under Assumption 4, the MMSE estimator R learned using the weighted sele-supervised loss (Psett) is equivalent to its supervised counterpart (Esup). Specifically, we have:
Assumption 3. which means that there exists > 0 such that for all x E R
The bias b(x), as defined in (9), is bounded,
Re (0) = RR (8)
(10)
[b(x)) 2|2 S
where
The only assumption on the bias is that it is bounded, which is relatively mild assumption, as image pixel values are typically constrained (e.g., to [0, 2551).
Csup =
2 1 x11
(11)
Note that Assumptions 1-2 are needed only for Proposition 1. They are not needed for our main
and
1-2.
Laef =
E P Mr - 8' 5/1
(12)
Proposition 1. Run ShaRP for t 2 1 iterations using the
2
step-size < S L under Assumptions 1-3, Then, the sequence h generated by ShaRP satisfies
where the vector = R(s) is MMSE estimation of R for s. s' = M is another independently subsampled measure- ment and W is a weighting matrix to compensate sampling imbalance measurement domain.
E
V fxx A- 21)1
<
(f(x))-f)+++ +e?. 7
2
Rm
5

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
5. Numerical Results
and P is the k-space subsampling mask. The term y de- notes the acquired complex k-space data, and e is the ad- ditive complex noise vector. We utilized the open-access fastMRI dataset; further experimental details can be found in Section B. of the supplementary material.
We numerically validate ShaRP on two inverse problems of the form y = Ax te: (Compressive Sensing MRI (CS-MRI) and (b) Single Image Super Resolution (SISR). In both cases, represents additive white Gaussian noise (AWGN). For the data-fidelity term in eq. (2), we use the f2-norm loss for both problems. Quantitative performance is evaluated by Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity In- dex (SSIM). Additionally, for the SISR task, we include the Learned Perceptual Image Patch Similarity (LPIIS) metric to evaluate perceptual quality. Additional numerical results are provided in the supplementary material.
Ensemble of restoration priors for CS-MRI. Recent methods like InDI (Delbracio & Milanfar, 2023) and IB (Liu et al., 2023) use controllable processes to train ensembles of restoration priors, each MMSE operator for specific setting. We build on this by training an 8x uniform subsampling CS-MRI model with 8 masks as our prior. Following InDI, we decompose the MRI degradation operator M = PFS into convex combinations of M and the identity I: H. = (1 - a + aM, with a controlling degradation. Varying a creates an ensemble of tasks. Train- ing R on all these tasks allows it to act as an ensemble of
Noise level
σ = 0.005
0 == 0.010 0= 0.015
Metries
PSNR SSIM PSNR SSIM PSNR SSIM
Zero-filled
26.93 0.848 26.92 0.847 26.90 0,848
TV
31.17 0.923 31.08 0.921 30.91 0.915
MMSE operators: R(s, H E [a (8, Ha). We used the MSE loss for training.
PnP-FISTA
35.88
0.938 31.14 0.894 30.32 0.846
PmP-ADMM
35.76
0.941 32.36 0.878 30.66 0.838
DRP
35.52
0.936 32.32 0.914 30.57 0,901
DPS
DDS
ShaRP
Groundtruth
DPS
32.62
0.888 31.39 0.870 30.29 0,856
29.66 0.09
39.46 0.977
45.78 0.90
PSNR SSIM
DDS
35.21
0.937 35.03 0.935 34.51 0.925
ShaRP
37.59 0.963 35.81 0.951 34.92 0.942
E2E- VarNet
38.10 0.971 36.80 0.967 35.79 0.954
VarNeVarNet needs to he retrained for each task.
0.250
Table 2: Quantitative comparison of ShaRP with several baselines for CS-MRI using 4x uniform masks. The best and second best results across general image restoration methods are highlighted. Notably, ShaRP outperforms SOTA baseline methods.
0.912
PSNR
Noise level
0.005
0.010
d 0.015
Metrics
PSNR SSIM PSNR SSIM PSNR SSIM
Figure 2: Visual comparison of ShaRP with baseline meth- ods on CR-MRI. The top row shows results for a 4x random mask with noise a = 0.005, and the bottom row for a 6x random mask with noise o = 0.015. PSNR and SSIM val- ues are in the top-left corner of each image. Error maps and zoomed-in in areas highlight differences. Notably, ShaRP with stochastic priors outperforms state-of-the-arr methods using denoiser and diffusion model priors.
ADMM-TV
28.14
0.866 28.06 0.863 27.96 0.859
GRAPPA
28.09
0.792 25.39 0.699 23.94
0.649
SPICER
31.87
0.901 31.67 0.889 31,50
0.887
ShaRpser 33.87 0.909 33.64 0.900 33.21 0.892
Table 3: Quantitative comparison of ShaRP with a self- supervised pre-trained restoration operator, compared to several baselines for CS-MRI using 4x uniform masks. The best and second best results are highlighted. Note the excellent performance of ShaRP even using priors trained without fully-sampled ground-truth data.
Training restoration priors without groundtruth. When fully-sampled images are unavailable for training, MRI restoration priors can be trained supf-super (Yaman et al., 2020: Millard & Chiew, 2023: Gan et al., 2023b; Hu et al., 2024d), as demonstrated in Theorem 2. Notably, the MRI imaging system aligns well with the assumptions required by Theorem 2, as FS represents an orthogonal matrix. This approach uses a separate subsampled mea- surement as the training label, instead of the ground-truth image. Importantly, and consistent with Theorem 2. our self- supervised training theoretically achieves MMSE restora-
5.1. CS-MRI setting
The complex-valued measurement model for CS-PMRI is expressed as: y = PFSx + e. Here, x represents the underlying complex-valued image, S are the complex multi- coil sensitivity maps, F is the Fourier transform operator,
6

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Bicubic
DPIR
DDNM 25.34/ 0.793
DifIPIR
ShaRP
Groundtruth PSNR / SSIM
22.68/ 0.509
24.64 0.827
27.95 / 0.857
28.30 / 0.883
24.04 0.733
27.75 0.901
26.32. 0.909
27.66 0.909
28.11 0.925
PSNR SSIM
Figure 3: Visual comparison of ShaRP with several well-known methods on 2x SISR. The top row shows results for SISR with gaussian blur kernel with a = 1.25, while the bottom row shows results for SISR with gaussian blur kernel with 1.5. The quantities in the top-left corner of each image provide PSNR and SSIM values for each method. The squares the bottom of each image visualize the zoomed area in the image.
tion priors equivalent to those trained fully-superviseelly For validation, we train an 8x uniform subsampling CS-MRI model to handle cight distinct restoration operators corre- sponding to different sampling masks. Additional training details are in Section B.1 of the supplement.
ShaRP's performance, highlighting the effectiveness of em- ploying multiple operators to maximize the regularization information provided by the restoration model. Figure 2 presents visual reconstructions for two test scenarios, where ShaRP accurately recovers fine brain details, particularly in the zoomed-in regions, while baseline methods tend to oversmooth introduce artifacts. These results highlight ShaRP's superior ability to manage structured artifacts and preserve fine details, outperforming both denoiser- based and diffusion model-based methods.
With the pre-trained 8x models as ensembles of restoration priors, we evaluate ShaRP's performance across variety of configurations, including two sub-sampling rates (4x and 6x), ), two mask types (uniform and random), and three noise levels (a 0.005, 0.01, and 0.015). Due to space constraints, only a subset of these settings is included in the main paper. Additional results can be found in Section B.I of the supplementary material.
Results with MMSE restoration operator learned from incomplete measurements. We further evaluate ShaRP's performance using an restoration model, learned in self- supervised manner, introduced in (Gan al., 2023b). In this setting, compare ShaRP against two classical methods for CS-MRI reconstruction without groundtruth: TV (Block et al., 2007) and GRAPPA (Griswold et al., 2002) and a recent state-of-the-art self-supervised deep unrolling method: SPICER (Hu et al., 2024d). As shown in Table 3, ShaRP demonstrates its effectiveness in leveraging self- supervised restoration prior for various reconstruction tasks, even when only incomplete measurements (8x subsam- pled) are available. Note that, ShaRP using self-supervised restoration prior even outperforms DPIR and DPS that use Gaussian denoisers trained using fully-sampled ground truth images (see Table 6 in the appendix). is important to high- light that training Gaussian denoisers infeasible when only undersampled measurements are available.
was compared against several base- line methods, including denoiser-based approaches (PnP- FISTA (Kamilov et al., 2017), PnP-ADMM (Chan et al., 2017)) and diffusion model-based methods (DPS (Chung et al., 2023), DDS (Chung et al., 2024)). To highlight the advantages using a stochastic set restoration operators, we also compared ShaRP with the DRP method (Hu al., 2024c), which applies only a single restoration operator. Additional details related to the baseline methods can be found in Section B.1 of the appendix.
Baselines. ShaRP
Results with supervised MMSE restoration operator. Table 2 provides quantitative comparison of reconstruction performance across different acceleration factors and noise levels using a uniform sub-sampling mask. In all configura- tions, ShaRP consistently outperforms the baseline methods. The use of set of restoration operators clearly enhances
We further evaluate the convergence performance of ShaRP using both supervised and self-supervised restoration priors.

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Due to space limitations, the detailed results and discussion provided in Section C.3.
which improves the perceptual quality. Additional results can be found in Section D) of supplementary material.
5.2. Single Image Super Resolution (SISR)
6. Discussion
The measurement operator in SISR can be written as A SK, where K represents convolution with the blur kernel, and S performs standard d-fold down-sampling. In our experiments, we use two Gaussian blur kernels k. each with distinct standard deviations (1.25 and 1.5), and with down-sampling factor of 2. Both noisy and noise-free cases are considered to evaluate the noisy robustness of ShaRP. We randomly selected 100 images from the ImageNet test set, as provided in DiftPIR!. Due space constraints, only a subset of comparisons is included in the main paper. Additional results can be found in Section D of the supplement.
ShaRP as a Unifed Framework. Unlike methods that rely on a fixed prior for all tasks, ShaRP operates without being constrained to any specific prior. Instead, it adapts flexibly to the unique characteristics of diverse inverse problems, en- abling the integration of a broad range of restoration priors, including- but not limited to do-deniisers. This adaptability not only enhances the versatility and applicability of the framework but also has the potential to achieve superior performance, as demonstrated Section 5. By building on a unified theoretical foundation, ShaRP represents a signif- icant advancement in addressing a wide variety of inverse problems effectively.
Ensemble of Restoration Priors for Image Deblurring. Similar to our CS-MRI prior training, we decompose the deblurring task (Gaussian blur operator K) into a convex combination of K and the identity mapping I: H. (1 [1) + aK with controlling degradation. Vary- ing yields multiple degradation operators, enabling our restoration network R to handle each. This allows R to function as an ensemble of MMSE restoration operators: R(s, H.) = E[x s, Ha), where s and æ are the degraded and original images, respectively. The original K uses a 31 x 31 Gaussian kernel with a standard deviation of 3. Further training details are in Section B.2 of the supplement.
Performance improved with Ensemble Priors. Conven- tional ensemble priors for inverse problems typically focus on Gaussian denoisers across varying noise levels (Zhang et al., 2019; Hurault et al., 2022a; Renaud et al., 2024). ShaRP advances this concept by integrating structured noise from diverse degradation operators, forming richer and more robust ensemble. Empowered by its flexible and novel regularizer, ShaRP seamlessly incorporates this diverse en- semble into a unified, closed-form prior that accommodates both Gaussian and more complex denoisers. This innova- tion highlights ShaRP's ability to leverage a broader range of priors, enabling adaptable and robust performance across wide variety of noise environments. Further evidence of the benefits of such expansive ensemble is provided by an ablation study in Section E.2.
Baselines. We compared ShaRP against baselines including DPIR (Zhang et al., 2022), a -tate-of-the-art PnP method using pre-trained denoisers, and diffusion-based methods DPS (Chung et al.. 2023). DDNM (Wang et al., 2023), DDRM(Kawar et al., 2022), and DiffPIR(Zhu et al., 2023), which use different sampling strategies for SISR.
ShaRP's ability to integrate diverse restoration priors within a unified framework enables task-specific adaptability, driv- ing significant performance improvements and advancing inverse problem-solving
Results on SISR with deblurring prior. Figure 3 shows the visual reconstruction results for two settings with dif- ferent blur kernels. As demonstrated, ShaRP successfully recovers most features and maintains high data consistency with the available measurements. Table 4 provides quan- titative comparisons against baselines across blur kernels and noise levels. ShaRP achieves the highest PSNR and SSIM, but ranks second in perceptual performance (LPIPS), consistent with SOTA Diffusion Model-based methods on SISR. However, the deblurring prior in ShaRP recovers fine details, ensuring competitive perceptual quality. One inter- pretation for the improved performance is that when con- sidering an ensemble of blur-then-deblur operations, ShaRP can be seen as masking and subsequently restoring portions of null-space- aaiseeaaaaaeeeeeeaaeeeeaa to the mechanism of masked (denoising) autoencoders. In this interpretation, MMSE restoration net- works effectively reconstruct the missing null-space signals,
Sampling from Restoration Priors. Our current work in- troduces novel objective function balancing data fidelity with implicit regularization from an ensemble of priors. Although this optimization-based approach achieves high reconstruction accuracy, its perceptual performance is in- herently limited by the well-known perceptual-distortee trade-off (Blau & Michaeli, 2018). To enhance the percep- tual performance, sampling-based extension is a promis- ing future direction. Specifically, we aim to adapt existing diffusion- based image restoration methods (Daras et al., 2023; Bansal et al., 2023; Delbracio & Milanfar, 2023; Liu et al., 2023), initially developed for specific tasks, to serve as priors for sampling in general image restoration.
8

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Noise level
Noiseless
= 0.01
Noiseless
= 0.01
Metries
PSNR
SSIM LPIPS
PSNR SSIM LPIPS PSNR SSIM LPIPS
PSNR
SSIM LPIPS
DPIR
28.10 0,809 0.305
28.05 0.807 0.308 27.90 0.803 0.314
27.87
0.800 0.314
DDNM
27.53 0.786 0.240
27.49 0.784 0.246
27.02 0.764 0.264
27.01
0.763 0.267
DPS
24.68
0,661 0.395
24.60 0.657 0.399
24.50 0.657
0.403
24.44
0.655 0.406
DifIIIR
28.92
0.852 0.152
28.63
0.839 0.169
28.59
0.834
0.172
28.02
0.819 0.185
DDRM
28.20
0.845
0.161
28.11
0.832 0.188
27.93 0.826
0.188
27.67
0.817 0.193
DRP
29.28
0.868
0.207
28.87
0.848 0.248
28.24
0.836
0.235
28.01
0.820 0.278
ShaRP
30.09
0.891 0.179
29.03
0.852 0.223
29.28 0.872 0.209
28.06
0.821 0.268
Table 4: Quantitative comparison of ShaRP with several baselines for SISR based on two different blur kernels on ImageNet dataset. The best and second best results are highlighted. Notably, ShakP outperforms SOTA methods based on denoisers and diffusion models.
7. Conclusion
References
This work presents ShaRP, a novel framework for imag- ing inverse problems that leverages pre-trained restoration networks as priors. Unlike methods limited Gaussian denoisers single prior, ShaRP integrates multiple priors tailored to diverse degradations within unified theoretical framework. The key findings are: (1) using priors beyond traditional Gaussian denoisers broadens the framework's s applicability and performance, and (2) stochastically inte- grating multiple degradation-specife priors achieves better performance than relying on single prior. Numerical ex- periments confirm ShaRP's superiority over conventional methods. By unifying diverse priors into flexible frame- work, ShaRP encourages exploring more complex restora- tion priors for inverse problem-solving.
Aali, A., Daras, G., Levac, B., Kumar, S., Dimakis, A. G., and Tamir, J. I. Ambient diffusion posterior sampling: Solving inverse problems with diffusion models trained on corrupted data. arXiv preprint arXiv:2403.08728, 2024.
Ahmad, R., Bouman, C. Buzzard, G. T., Chan, S., Liu, S., Reehorst, E. T., and Schniter, P. Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery. IEEE Sig. Process. Mag., 37(1):105- 116, 2020.
Alçalar, Y. U. and Akçakaya, M. Zero-shot adaptation for approximate posterior sampling of diffusion models in inverse problems. arXiv preprint arXiv:2407.11288 2024.
Ethics Statement
Bai, W., Chen, S., Chen, W., and Sun, H. Blind in- version using latent diffusion priors. arXiv preprint arXiv:2407.01027 2024.
To the best of our knowledge this work does not give rise to any significant ethical concerns.
Bansal, A., Borgnia, E., Chu, H., Li. J., Kazemi, H. Huang, F., Goldblum, M., Geiping, J., and Goldstein, T. Cold diffusion: Inverting arbitrary image transforms without noise. Proc. Adv. Neural Inf. Process. Syst., 36:41259- 41282, 2023.
Impact Statement
ShaRP makes advanced computational imaging more ac- cessible by reducing reliance on task-specific models and extensive supervision. Its versatility enables broad soci- etal impact across fields that depend high-quality image data, such as healthcare, scientific research, and consumer technology.
Bertsekas, D. P. Incremental proximal methods for large scale convex optimization. Math. Program. Ser. B, 129: 163-195, 2011.
Acknowledgement
Blau, Y. and Michaeli, T. The perception-distorte tradeoff. Proc. IEEE Conf. Comput. Vis. and Puttern Recognit. (CVPR), pp. 6228-6237, 2018.
Research presented in this article was supported in part by the NSF CAREER awards under grant CCF-2043134.
Block, K. T., Uecker, M., and Frahm, J. Undersampled radial MRI with multiple coils. iterative image recon- struction using a total variation constraint. Magn. Reson. Med., 2007.

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Bora, A., Jalal, A., Price, E., and Dimakis, A. G. Com- pressed sensing using generative priors. In Int. Conf. Mach. Learn.. pp. 537-546, August 2017.
Dong, W., Wang, P., Yin, W., Shi, G., Wu, F., and Lu, X. Denoising prior driven deep neural network for image restoration. IEEE Trans. Pattern Anal. Mach. Intell., 41 (10):3305-2315 Oct 2019.
Buzzard, G. T., Chan, S. H., Sreehari, S., and Bouman, C. A. Slug-and-play unplugged: Optimization free reconstruc- tion using consensus equilibrium. SIAM J. Imaging Sci., 11(3):001122200 Sep. 2018.
Fang. Z., Buchanan, S., and Sulam, J. What's 's in prior? Learned proximal networks for inverse problems. Proc. Int. Conf. on Learn. Represent. (ICLR), 2024.
Chan, S. H., Wang, X., and Elgendy, O. A. Plug-and-play ADMM for image restoration: Fixed-point convergence and applications. IEEE Trans. Comp, Imag. 3(1):84-98, Mar. 2017.
Feng, B. T., Smith, J., Rubinstein, M., Chang, H., Bouman, K. L., and Freeman, W. Score-based diffusion models as principled priors for inverse imaging. In Proc. IEEE Int. Conf. Comp. Vis. (ICCV), pp. 10520-10531, 2023.
Chen, D., Tachella, J., and Davies, M. Robust equivariant imaging: a fully unsupervised framework for learning to image from noisy and partial measurements. Proc. IEEE Conf. Comput. Vis. and Pattern Recognit. (CVPR), 2022.
Gan, W., Shoushtari, S., Hu, Y., Liu, J., An, H.,, and Kamilov, U. S. Block coordinate plug-and-play methods for blind inverse problems. Proc. Adv. Neural Inf. Process. Syst., 2023a.
Chen, H., Zhang, Y., Kalra, M. K., Lin, F., Chen, Y., Liao, P., Zhou, J., and Wang, Low-dose CT with residual encoder-decoder convolutional neural network. IEEE Trans. Med. Imag., 36(12):2524 2535, Dec. 2017.
Gan, W., Ying, C., Boroojeni, E., Wang, T., Eldeniz, C., Hu, Y., Liu, J., Chen, Y., An, H., and Kamilov, U. sulf-supe deep equilibrium models with theoretical guarantees and applications to mri reconstruction. IEEE Trans. Comp. Imag., 2023b.
Chung, H., Kim, J., Mccann, M. T., K., M. I.,, and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. In Int. Conf. Learn. Represent., 2023.
Ghadimi, S. and Lan, G. Accelerated gradient methods for nonconvex nonlinear and stochastic programming. Math. Program. Ser. A, 156(1):59 -9, Mar. 2016.
Chung. H., Lee, S., and Ye, C. Decomposed diffusion sampler for accelerating large-scale inverse problems. In Proc. Int. Conf. on Learn. Represent. (ICLR), 2024.
Gilton, D., Ongie, G., and Willett, R. Deep equilibrium architectures for inverse problems imaging. IEEE Trans. Comput. Imag., 7:1123-1133, 2021a.
Cohen, R., Blau, Y., Freedman, D., and Rivlin, It has potential: Gradient-driven denoisers for convergent lutions to inverse problems. In Proc. Adv. Neural Inf. Process. Syst. 34, 2021.
Gilton, D., Ongie, G., and Willett, R. Model adaptation for inverse problems in imaging. IEEE Trans. Comput. Imag., 7:661-674 July 2021b.
Daras, G., Shah, K., Dagan, Y., Gollakota, A., Dimakis, A., and Klivans, A. Ambient diffusion: Learning clean distributions from corrupted data. In Proc. Adv. Neural Inf. Process. Syst. 2023.
Gossard, A. and Weiss, Training adaptive reconstruction networks for blind inverse problems. SIAM Imaging Sci., 17()):314-1346, 2024.
Daras, G., Chung, H., Lai, C., Mitsufuji, Y., Ye, J. C., Mi- lanfar, P., Dimakis, A. G., and Delbracio, M. A survey on diffusion models for inverse problems. arXiv preprint arXiv:2410.00083. 2024.
Griswold, M. A., Jakob, P. M., Heidemann, R. M., Nit- tka, M., Jellus, V., Wang, Kiefer, B., and Haase, A. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn. Reson. in Med., 47(6):120--1210, 2002.
Delbracio, M. and Milanfar, P. Inversion by direct iteration: An alternative to denoising diffusion for image restoration. Trans. on Mach. Learn. Research, 2023. ISSN 2835-8856.
Hauptmann, A., Lucka, F., Betcke, M., Huynh, N., Adler, J., Cox, B., Beard, P., Ourselin, S., and Arridge, S. Model- based learning for accelerated, limited-view 3-D pho- toacoustic tomography. IEEE Trans. Med. Imag., 37(6): 1382-1393, 2018.
Delbracio, M., Talebei, H., and Milanfar, P. Projected distri- bution loss for image enhancement, In 2021 Int. Conf. on Comput. Photography (ICCP), pp. 1-12, 2021.
Demidovich, Y., Malinovsky, G., Sokolov, I., and Richtarik, P. A guide through the zoo biased SGD. In Proc. Adv. Neural Inf. Process. Syst., 2023.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Proc. Adv. Neural Inf. Process. Syst., 33: 6840-6851, 2020.
10

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Hu, J., Gan, W., Sun, Z., An, H., and Kamilov, U. A plug- and-play image registration network. In Proc. Int. Conf. on Learn Represent. (ICLR), 2024a.
Krull, A., Buchholz, T., and Jug, F. Noise2void-learning denoising from single noisy images. In Proc. IEEE Conf. Comput. Vis. and Pattern Recognit, (CVPR), pp. 2129- 2137,
Hu, J., Song, B., Xu, X., Shen, L., and Fessler, J. A. Learning image priors through patch-based diffusion models for solving inverse problems. arXiv preprint arXiv:2406.02262. 2024b.
Laumont, R., De Bortoli, V., Almansa, A., Delon, J., Dur- mus, A., and Pereyra, M. Bayesian imaging using plug & play priors: When Langevin meets Tweedie. SIAM J. Imaging Sci., 15(2)7001-37, 2022.
Hu, Y., Delbracio, M., Milanfar, P., and Kamilov, U. A Restoration Network as an Implicit prior. In Proc. Int. Conf. on Learn. Represent. (ICLR), 2024c.
Liang, J., Cao, J., Sun, G., Zhang, K., Van G., L., and Timofte, R. Swinir: Image restoration using swin trans- former. In Proc. IEEE Int. Conf. Comp. Vis. (ICCV), pp. 1833-1844, 2021.
Hu, Y., Gan, W. Ying, C., Wang, T., Eldeniz, C., Liu, J., Chen, Y., An, H., and Kamilov, U. S. Spicer: Self- supervised learning for mri with automatic coil sensitiv- ity estimation and reconstruction. Magn. Reson. Med., 2024d.
Liu, G., Vahdat, A., Huang, D., Theodorou, E. A., Nic, W., and Anandkumar, 12sb: image-to-image schrödinger bridge. In Proc. Int. Conf. Machine Learning (ICML), pp. 22042-22062, 2023.
Hurault, S., Leclaire, A., and Papadakis, N. Gradient step denoiser for convergent plug-and-play. In Proc. Int. Conf. on Learn. Represent. (ICLR), Kigali, Rwanda, May 1-5, 2022a.
Liu, J., Sun, Y., Eldeniz, C., Gan, W., An, H., and Kamilov, U. S. RARE: Image reconstruction using deep priors learned without ground truth. IEEE J. Sel. Topics Signal Process., 14(6))1088-1000 Oct. 2020.
Hurault, S., Leclaire, A., and Papadakis, N. Proximal de- noiser for convergent plug-and-play optimization with nonconvex regularization. In Iut. Conf. Mach. Learn. pp. 9503,9505, Kigali, Rwanda, 2022b. PMLR.
Liu, J., Xu, X., Gan, W., Shoushtari, S., and Kamilov, U. S. Online deep equilibrium learning for regularization by denoising. In Proc. Adv. Neural Inf. Process. Syst., 2022.
Jin,& K. H., McCann, M. T., Froustey, E., and Unser, M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process., 26(9):4509- 4522, Sep. 2017.
Mardani, M., Song, J., K., and Vahdat., A. A variational perspective on solving inverse problems with diffusion models. In Proc. Int. Conf. on Learn. Represent. (ICLR), 2024.
Kadkhodaie, Z. and Simoncelli, E. P. Stochastic solutions for linear inverse problems using the prior implicit a denoiser. In Proc. Adv.Neural Inf. Process. Syst. 34, pp. 13242-13254, December 6-14, 2021.
McCann, M. T., Jin, K. H., and Unser, M. Convolutional neural networks for inverse problems in imaging: re- view. IEEE Signal Process. Mag., 34(6):85 -5, 2017.
Metzler, C., Schniter, P., Veeraraghavan, A., and Baraniuk, prDeep: Robust phase retrieval with a flexible deep network. In Proc. 36th Int. Conf. Mach. Learn., pp. 3501- 3510, Jul. 10-15 15 2018.
Kamilov, U. S., Mansour, H., and Wohlberg, B. plug- and-play priors approach for solving nonlinear imaging inverse problems. IEEE Signal. Proc. Let., 24(12):1872 1876, Dec. 2017.
Milanfar, P. and Delbracio, M. Denoising: A powerful building-block for imaging, inverse problems, and ma- chine learning. arXiv preprini 2rXiv:2409.06219, 2024.
Kamilov, U. S., Bouman, C. A., Buzzard, G. T., and Wohlberg, B. Plug-and-play methods for integrating phys- ical and learned models in computational imaging. IEEE Signal Process. Mag., 40(1):86 (0(1):85-97 January 2023.
Millard, C. and Chiew, M. A theoretical framework for self- supervised mr image reconstruction using sub-sampling via variable density noisier2noise. IEEE Trans. Comp. Imag., 2023.
Kang. E., Min, J., and Ye, J. C. A deep convolutional neural network using directional wavelets for low-dose x-ray CT reconstruction. Med. Phys., 44(10):e360-cs) -375, 2017.
Monga, V., Li, Y., and Eldar, Y. C. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Process. Mag., 38(2):18 18 44, March 2021.
Kawar, B., Elad, M., Ermon, S., and Song. J. Denoising dif- fusion restoration models. Proc. Adv. Neural Inf. Process. Syst, 35:23593-23606, 2022.

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Nathan, O., Levy, D., Treibitz, T., and Rosenbaum, D. Os- mosis: Rgbd diffusion prior for underwater image restora- tion. In Euro. Conf. Comp. Vis. (ICCV), pp. 302-319. Springer, 2024.
IEEE Trans. Comput. Imag. 5(3):395 September 2019.
Sun, Y., Xu, S., Li, Y., Tian, L.,, Wohlberg, B., and Kamilov, U. Regularized Fourier ptychography using an online plug-and and- algorithm. In Proc. IEEE Int. Conf. Acous- ties, Speech and Signal Process. (ICASSP), pp. 7665- 7669, Brighton, UK, May 12-17, 2019.
Nesterov, Y. Introductory Lectures on Comvex Optimization: Basic Course. Kluwer Academic Publishers, 2004.
Ongie. G., Jalal, A., Metzler, C. A.. Baraniuk, R. G., Di- makis, A. G., and Willett, R. Deep learning techniques for inverse problems imaging. IEEE J. Sel. Areas Inf. Theory, 1(1):39-56, May 2020.
Sun, Y., Wu, Z.,, Wohlberg, B., and Kamilov, U. S. Scal- able plug-and-play ADMM with convergence guarantees. IEEE Trans. Comput. Imag. 7:849-86 863, July 2021.
Quan, Y., Chen, M., Pang, T., and Ji, H. Self2self with dropout: Learning self-supervise denoising from single image. In Proc. IEEE Conf. Comput. Vis. and Pattern Recognit. (CVPR), pp. 1890-1898, 2020.
Sun, Y., Wu, Z., Chen, Y., Feng, B. T., and Bouman, K. L. Provable probabilistic imaging using score- base- genera- tive priors. IEEE Trans. Comp. Imag. 2024.
Tachella, J., Dongdong, C., and Davies, M. Unsupervised learning from incomplete measurements for inverse prob- lems. In Pruc. Adv. Neural Inf. Process. Syst., pp. 4983- 4995, 2022.
Rechorst, E. T. and Schniter, P. Regularization by denois- ing: Clarifications and new interpretations. IEEE Trans. Comput. Imag. 5(1):22-77, March 2019.
Renaud, M., Prost, J., Leclaire, A., and Papadakis, N. Plug- and-play image restoration with stochastic denoising regu- larization. In Int. Conf. Machine Learning (ICML), 2024.
Teodoro, M., Bioucas-Dias, M., and Figueiredo, M. convergent image fusion algorithm using scene-adapted mwitmmii-teieee mixsed denoising. IEEE Trans. Image Process. 28(1):451 453-4433,, January 2019.
Romano, Y., Elad, M., and Milanfar, P. The little engine that could: Regularization by denoising (RED). SIAM Imaging Sei., (M(4):804440 1844, 2017.
Terris, M. and Moreau, T. Meta-prior: Meta learning for adaptive inverse problem solvers. arXiv preprint arXiv:2311.18711 2023.
Rout, I.., Chen, Y., Kumar, A., Caramanis, C., Shakkottai, S., and Chu, W. Beyond first-order tweedie: Solving inverse problems using latent diffusion. In Proc. IEEE Conf. Comput. Vis. and Pattern Recognit. (CVPR), pp. 9472 9481, 2024.
Terris, M., Moreau, T., Pustelnik, N., and Tachella, J. Equiv- ariant pldg-and-play image reconstruction. In Proc. IEEE Conf. Comput. Vis. and Pattern Recognit. (CVPR), pp. 25255-25264 2024.
Ryu, E. K., Liu, J.. Wang, S., Chen, X., Wang, Z., and Yin, W. Plug-and and-play methods provably converge with properly trained denoisers. In Proc. Inf. Conf. Machine Learning (ICML), volume 97, pp. 5546-5557, 2019.
Tirer, T. and Giryes, R. Image restoration by iterative de- noising and backward projections. IEEE Trans. Image Process, 28(3):1220-1233 12344 Mar. 2019.
Uecker, M., Lai, P., Murphy, M. J., Virtue, P., Elad, M., Pauly, J. M., Vasanawala, S. S., and Lustig, M. ESPIRiT- eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn. Reson. Med., 71 (3):990-1001, 2014.
Song, B., Kwon, S. M., Zhang, Z., Hu, X., Qu, Q., and Shen, L. Solving inverse problems with latent diffusion models via hard data consistency. In Proc. Int. Conf. on Learn. Represent. (ICLR), 2024.
Song. Y. J. K., P., K., E., and P., B. Score- based generative modeling through stochastic differen- tial equations. In Proc. Int. Conf. Learn. Represent. (ICLR), 2021.
Ulyanov, D., Vedaldi, A., and Lempitsky, V. Deep image prior. In Proc. IEEE Conf. Comput. Vis. and Pattern Recognit. (CVPR), pp. 9446-9454, 2018.
Venkatakrishnan, S. V., Bouman, C. A., and Wohlberg, B. Plug-and-play priors for model based reconstruction. In Proc. IEEE Global Conf. Signal Process. and Inf. Pro- cess., pp. 145-948, Dec. 3-5, 2013.
Sriram, A., Zbontar, J., Murrell, T., Defazio, A., Zitnick, C. L., Yakubova, N., Knoll, F,, and Johnson, M. End- to-end variational networks for accelerated MRI recon- struction. arXiv, 2004.06688, 2020.
Wang, S., Su, Z., Ying, L.,, Peng, X., Zhu, S., Liang, F,, Feng, D., and Liang, D. Accelerating magnetic resonance imaging via deep learning. Proc. Int. Symp. Biomedical
Sun, Y., Wohlberg, B., and Kamilov, U. S. An online plug- and-play algorithm for regularized image reconstruction.
12

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Imaging (ISBI), pp. 514-517, April 2016. doi: 10.1109/ ISBI.2016.7499 7.793300.
Wang, Y., Yu, J., and Zhang, J. Zero-shot image restoration using denoising diffusion null-space model. In Proc. Int. Conf. on Learn. Represent. (ICLR), 2023.
Wei, K., Aviles-Rivero, A., Liang, J., Fu, Y., Schönlieb, C.- B., and Huang, H. Tuning-free free plug-and-play proximal algorithm for inverse imaging problems. In Int. Conf. Machine Learning (ICML), 2020.
Wen, B., Ravishankar, S., Zhao, Z.,, Giryes, R., and Ye, J. C. Physics-driven machine learning for computational imaging. IEEE Signal Process. Mag., 40(1):28-30, 2023.
Wu, Z., Sun, Y., Chen, Y., Zhang, B., Yue, Y., and Bouman, K. L. Principled probabilistic imaging using diffu- sion models as plug-and-play priors. arXiv preprint arXiv:2405. 18782, 2024.
Xia, B., Zhang, Y., Wang. S., Wang. Y., Wu, X., Tian, Y., Yang, W., and Van Gool, L. Diffir: Efficient diffusion model for image restoration. In Proc. IEEE Int. Conf. Comp. Vis. (ICCV), pp. 13095-13105, 2023.
Xiao, J., Feng, R., Zhang, H., Liu, Z., Yang, Z., Zhu, Y., Fu, X., Zhu, K., Liu, Y., and Zha, Z. Dreamclean: Restoring clean image using deep diffusion prior. In Proc. Int. Conf. on Learn. Represent. (ICLR), 2024.
Yaman, B., Hosseini, A. H., Moeller, Ellermann, J., Ugurbil, K., and Akçakaya, M. Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data. Magn. Reson. Med., 84(6);
3172-3191, Jul. 2020.
Zhang. and Ghanem, B. ISTA-Net: Interpretable optimization-imspiee. deep network for image compres- sive sensing. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 1828-1837, 2018.
Zhang, K., Zuo, W., and Zhang, L.. Deep plug-and-play super-resolution for arbitrary blur kernels. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 1671- 1681, Long Beach, CA, USA, June 16-20, 2019.
Zhang, K., Li, Y., Zuo, W., Zhang, L.,, Van Gool, L., and Timofte, R. Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Patt. Anal. and Machine Intell., 44(10))6360-6377 October 2022.
Zhao, J., Song, B., and Shen, L. Cosign: Few-step guidance of consistency model solve general inverse problems. arXiv preprint 2xXiv:2407.12670 2024.
Zhu, Y., Zhang, K., Liang, J., Cao, J.. Wen, B., Timofte, R., and Van G., L. Denoising diffusion models for plug-and- play image restoration. In Proc. IEEE Conf. Comput. Vis. and Pattern Recognit. (CVPR), pp. 1219-1229, 2023.
13

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Here's what cover in the appendix:
Theoretical Analysis of ShaRP
Proof of Theorem I (detailed in Subsection A.I); Derivation of the gradient of the ShaRP regularizer.
Proof of Proposition (detailed in Subsection A.2): Convergence guarantees for ShaRP with inexact Minimum
Mean Square Error (MMSE) restoration operators using biased Stochastic Gradient Descent (SGD) analysis.
- Proof of Theorem 2 (detailed Subsection A.3): Equivalence of the MMSE estimator learned via weighted self-supervised loss to its supervised counterpart.
Experiment Details (Section B)
Implementation details for Compressed Sensing MRI (CS-MRI tasks (Subsection B.1):
Implementation details for Single Image Super- Super-Resolution (SISR) tasks (Subsection B.2):
Additional results for CS-MRI (Section C)
Quantitative performance of ShaRP for uniform (Table 5) and random (Table 6) subsampling settings.
Visual comparisons for CS-MRI tasks (Figure Figure 6).
Performance of ShaRP with self-supervised restoration priors (Subsection C.2, Table 7).
Convergence performance analysis ShaRP with both supervised and sulf-super priors (Subsection C.3, Figure 7), aligning with the convergence analysis for inexact MMSE estimators.
Performance comparison with additional baseline methods on matched and mismatched settings (Table 8, Figure 8).
Additional visual results for SISR (Section D)
Further visual comparisons against various baseline methods for SISR (Figure 9, Figure 10).
Additional visual comparisons against the DRP method (Figure 11).
Quantitative (Table 10) and visual (Figure 12) comparisons against additional diffussio--ased baselines.
Additional Experiments
Ablation study on using a pre-trained super-resolution prior for the CS-MRI task (Subsection E.1, Algorithm
Table 12).
- Ablation study on the impact of the number of restoration priors, b, in the ensemble on CS-MRI performance (Subsection E.2, Figure 13, Figure 14).
Ablation study the influence of the hyperparameter (controlling prior selection, as introduced in Subsec- tion B.1) on CS-MRI performance (Subsection E.3, Figure 15).
A. Theoretical Analysis of ShaRP
A.1. Proof of Theorem
Theorem. Assume that the prior density Ps non-degenerate over IR" and let R* be the MMSE restoration operator (4) corresponding to the restoration problems (3). Then, we have that
Vh(
-2 (Ea-Go Happ H(x R" H))),
where h is the ShaRP regularizer in (6).
Proof. The ShaRP regularizer h(x) is defined as
h(x)) 7 TB (e- (slogp(s|HH) [f
= -- pD(H)) p(H)
Go(s Hæ) H) ds dH,
(13)
14

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
where Go is the Gaussian probability density with variance o² and p(s(H) is the likelihood function for the degraded observation given the operator H. The expectation over (HH) accounts for the randomness of the restoration operator H.
We start by relating the MMSE restoration operator to the score of the degraded observation
H -
/
Vp(s
Ha 8) (s(s Hx))p(()) dæ,
where Pae is the prior. By using the definition of the MMSE estimator, we obtain the relationship
Vlogp(s H o² (HR H) - -)
(14)
Consider the function inside the parenthesis in the expression for the ShaRP regularizer (13)
x(z) (G, logP.)H)(2)= fGo(z
Go(2 8) logp(s (H) ds,
where z has the same dimensions as s and denotes convolution. The gradient of p is given by
vp(z) (VG, * logPs/H)(z) (Ga Vlogr' ())
d2 / Golz- s) [HR* (s, H) s] ds
H Rd H)o - 8) ds - z
o²
where we used (14). By using Hæ, write the gradient with respect
V V (H)( = - - H (fRi Hæ) ds x
By using this expression in (13), we obtain the desired result
Vh(x) = - _ o2 S p(H)) Go(s-Ho) - Ha) (H''(R" H) - x)) ds
HmpH σ E. H
[H'(2
(s.
A.2. Proof of Proposition! 1
We adopt the analysis of biased stochastic gradient descent (SGD) from the optimization literature (Bertsekas, 2011; Ghadimi & Lan, 2016; Demidovich et al., 2023) to provide theoretical convergence guarantees to ShaRP under inexact MMSE restoration operators. Our contribution here is the interpretation of the iterations of ShaRP as a variant of biased SGD, rather than general analysis of SGD, which is well-known in the optimization literature.
Proposition. Run ShaRP for t 2 iterations using the step-size < 7 S 1/T. under Assumptions 1-3, Then, the sequence k generated by ShaRP satisfies
E
1 < (f(x") -f) 7 +
Proof. First note that from the definition of the bias in eq. (9), we have that
f( k = Vf(a x b(xx
(15)
where the expectation is with respect to s Go Ha *-1) and ~ PH. In order to simplify the notation, we will drop these subscripts from the expectations the analysis below.
15

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Consider the iteration A: 2 1 of ShaRP with inexact MMSE operator
f(x") _ f(x Vf(x ( x x²11)+
1| 11
2
In²
|Vf
(11?
2
where in the first line used the Lipschitz continuity of f. By taking the expectation with respect to Go -H and H ~ PH on both sides of this expression, we get
2
Bif(x*) x k- 1 f(x - VV (V + b(z b(x
1V5 -
2
- 5(x
7
Vf ( )12 + |16((=
2
172
E
2
{((
? [V 2
)11
7E²
L2²v²
2
2
In the second row, we completed the square, applied eq. (15), and used the assumption that y S I/L. In the third row, we used the variance and bias bounds in Assumptions 2 and 3. By rearranging the expression, we get the following bound
2 (f(x E[f((=))<= + Lyv² + * f < y V
By taking the total expectation, averaging over t iterations, and using the lower bound f*, get the desired result
V a >> $ 2 (f(x") f³) + Iy2² + 6².
E
A.3. Proof of Theorem 2:
Let the undersampled measurements defined as s Hæ nz, where H PM. Here, P represents binary subsampling matrix, M denotes the square transfer operator, and corresponds to the noise vector.
To show that an MMSE estimator can be learned from undersampled measurement, need the following assumption. Assumption 5. Ep[PT P] has a full rank and M is an orthogonal matrix, where the expectation is taken over pp.
This assumption implies that the union of all sampling matrices P spans the complete measurement domain, even though each individual P may remain undersampled. This property can be achieved by incorporating an additional weight W into the loss function, where: W P'W (P'W)' RRmxm denotes subsampled variant of W given p'.
Proposition When Assumption 2 satisfited,
P II,
where the expectation with respect to PM- This proof is the same as provided previous work (Gan et al., 2023b).
Theorem 3. Under Assumption 5, the MMSE estimator R learned using the weighted self- supervieee loss (Esett) is equivalent to its supervised counterpart (Ezup) Specifically, we have:
Re 0
Ri (0)
(16)
where
Suup
E
x
(17)
2
16

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
and
[ 2
Self
P' P'M - s'llw
(18)
The vector œ- R(s) is MMSE estimation R for 8.
Proof. Note that the self-supervised loss involves the term a s', where s' = P P'Mx' + n'
P'ME - s' ( x) n'.
(19)
Thus, the sepe-supe loss becomes:
Sell = P'M((a x) -n'llw
(20)
2
Expanding the squared term:
IP'M - x) n'lliv PI x)ll 2(P'M( -x)) Wn' + +lln'lww
= - W'P'M'(' - Wn' + |n'
So that
E P'M - x)) - n']lw]
E (0 T pd WP'M(a a)] - -E -B(2(''M(a-=)) + [|]n [| x [
constant,
where the first term equals to E || equals to zero because n' is zero- mean and independent of P' and æ; The third term, || lli does not depend on æ and contributes a constant that does not affect the optimization for training the MMSE estimator R.
=| due the Proposition 2 that E[M PPPM I the second term
17

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
B. Experiment Details
B.1. Implementation details CS-MRI tasks
Dataset. We simulated multi-coil subsampled measurements using T2-weighted human brain MRI data from the open-access fastMRI dataset, which comprises 4,912 fully sampled multi-coil slices for training and 470 slices for testing. Each slice has been cropped into complex-valued image with dimensions 320 320. The coil sensitivity maps for each slice precomputed using the ESPIRIT algorithm (Uecker et al., 2014). We simulated Cartesian sampling pattern that subsamples along the ky dimension while fully sampling along the ka dimension.
Subsampling pattern for CS-MRL. In this paper, we explored two types of subsampling patterns for MRI reconstruction tasks. All undersampling masks were generated by first including a set number of auto-calibration signal (ACS) lines, ensuring a fully-sampled central k-space region.
Figure illustrates the k-space trajectories for both random and uniform (equidistant) subsampling at acceleration factors of 4, 6, and 8. Notably, different patterns were used for training and testing. During training, our restoration prior was only trained on uniform mask with subsampling rate of 6. However, for inference, we employed both uniform and random masks at subsampling rates of 4 and 6, creating mismatch between the pre-trained restoration prior and the test configurations.
(a) Restoration prior training setting (H)
Target problem Inference setting
sampled piad non sumpled pinel
8 Uniform masks SCt
4 x Uniform mask
x Random mask
6 Uniform mask
6x Random mask
Figure 4: Illustration of the undersampling masks used for CS-MRI in this work. (a) The eight different 8x uniform masks used for training the restoration prior. (b) The inference setting for ShaRP, demonstrating how the prior trained on the masks in (a) can be applicd to solve other problems without retraining.
Algorithm 2 Supervised Training of CS-MRI Restoration Network
Require: dataset: p(z), sampling operator set: [M1, M M,). Restoration model: Ra(:,a)
repeat:
x ~ p(x), M ~ {M1, M2, ...
12 a u((0.1)
y = Mæ te
ming || || ((1 a) + aM y; x112
until converge
B. I. IMPLEMENTATION OF SUPERVISED PRIOR FOR CS-MRI
Models training for supervised case. We use the same U-Net architecture as employed in the official implementation of 2DS for R(: a). For the supervised learning case, we select 1,000 different a values to train the model, following the schedule outlined by I2SB (Liu et al., 2023). The model is trained with Adam optimizer with learning rate of 5 10-5 As shown in Algorithm 2, train our supervised learning model using eight different masks for 8x uniform sampling CS-MRI reconstruction. In the pseudocode, {M, M2, M3} represent the eight different MRI degradation operators, each defined by a unique sampling pattern, as shown in Figure 4 (a). This results in a total of 8,000 possible combinations of a values and sampling masks, effectively creating an ensemble of restoration priors during training.
Inference with a Subset of the Ensemble (Supervised Case), During inference, to simplify computation and focus on the most effective priors, we use only subset of the supervised trained ensemble. Specifically, we fix the value a particular choice (e.g., a 0.5) and use the different sampling masks MM, M2,..... ..., ,M&), resulting in 8 restoration priors.
Pttpss: //i thub. com wtt..tcoo/..-aaaeeeeeaeee HJ-harrryDDDs
18

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Step size and regularization parameter. To ensure faimess, for each problem setting, each method- proposed and baseline-is is fine-tuned for optimal PSNR using 10 slices from validation set separate from the test set. The same step size Y and regularization parameter are then applied consistently across the entire test set.
Baseline details. We compare ShaRP with several variants of denoiser- and diffusion model-based methods. For denoiser- based approaches, we include PnP-FISTA (Kamilov et al., 2023). PnP-ADMM (Chan et al., 2017). PnP-FISTA and PnP-ADMM correspond to the FISTA and ADMM variants of PnP, both utilizing AWGN denoisers built on DRUNet (Zhang et al., 2022). For diffusion model-based methods, we compare with DPS (Chung et al., 2023) and DDS (Chung et al., 2024), which use pre-trained diffusion models as priors and apply different posterior sampling strategies to address general inverse problems. We use the same pre-trained diffusion model configuration as outlined in the DDS paper. For all baseline methods, we fine-tuned their parameters to maximize the PSNR value. Notably, both the DRUNet denoiser and the diffusion model were trained using the same dataset employed for training our restoration prior. For a fair comparison, the diffusion model pre-trained for DDS and DPS use the same network architecture as our restoration network. All models are trained from scratch on the fastMRI training set, following the architecture settings provided in DDS We also compared with method that also use the deep restoration prior to solve general inverse problem: DRP (Hu et al., 2024c). For DRP, we utilize the same pre-trained restoration network as in ShaRP. However, instead of employing set of degradation priors, DRP uses single fixed prior. For fair comparison, we selected the optimal fixed prior-defined by a fixed and subsampling mask-based on PSNR performance on the validation set, and applied it accordingly.
B.1.2. IMPLEMENTATION OF SELF-SUPERVISED PRIOR FOR CS-MRI
Algorithm 3 Self-Supervised Training of CS-MRI Restoration Network
Require: dataset: p(y:, M,, yj, M2), Restoration model: R- repeat:
y:: Mi, M, ~ p(y: M, y,, M2), e~ (((,00) ming Re (M,' (y + +)) yı 1
until converge
Models training for (Self-Supervised Case). For self-supervised training, the ground truth reference is not available as a label. Instead, as shown in Algorithm 3, work with pairs of subsampled measurements, y and yj, along with their corresponding sampling operators, M, and M,. These paired measurements exhibit significant overlap within the shared auto-calibrattie signal (ACS) region, which increases the weighting of these overlapping k-space regions. Following the approach proposed by SSDEQ (Gan et al., 2023b), we introduce a diagonal weighting matrix W to account for the oversampled regions in the loss function. By incorporating this weighted loss, we are able to train our MMSE restoration operator using incomplete measurements alone. Furthermore, unlike the supervised case where we use the combination of values to form an ensemble, in the self-supervised setting, we construct the ensemble using only eight different sampling masks across the entire dataset.
Inference Using All Restoration Priors (Self-Supervised Case). During inference in the seffssuperviee setting, we utilize all 8 restoration priors corresponding to the different sampling masks. By incorporating the entire ensemble, we fully leverage its capacity to remove the artifacts and enhance reconstruction performance.
Step size and regularization parameter. To ensure fairness, for each problem setting, each method both proposed and baseline-is is fine-tuned for optimal PSNR using 10 slices from a validation set separate from the test set. The same step size y and regularization parameter are then applied consistently across the entire test set.
Baseline details. In the self-supervised setting, we compared ShaRP with two widely used traditional methods: TV (Block et al., 2007) and GRAPPA (Griswold et al.. 2002). both of which address the restoration problem without requiring fully- sampled references. Additionally, we included SPICER (Hu et al., 2024d), a recent sate-of-the-aae self-supervised deep unrolling method designed for MRI reconstruction using only pairs of undersampled measurements. To ensure consistency, we trained the SPICER model on the same amount of paired data used for training our restoration prior in the 8x uniform CS-MRI setting and applied it other CS-MRI configurations.
httpps//githuh..omHHHL haarDDDD
19

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Algorithm 4 Gaussian Deblurring Restoration network training
Require: dataset:p(z, 3). Gaussian blur operator: K, Re(-:a) repeat:
x p(z), e (0,, ((,, !!) ming [|Rg ((1 a) + aKæ;a) - x12
until converge
B.2. Implementation details of SISR tasks
Restoration Model training. We use the same U-Net architecture as the Gaussian deblurring model provided by IB Utilizing the pre-trained checkpoints from their repository, we fine-tune our model accordingly. Specifically, we align with their codebase and configure the model type to OT-ODE to satisfy our MMSE restoration operator assumption.
To create an ensemble of restoration priors, we consider family of degradation operators that are convex combinations of the identity mapping I and the Gaussian blur operator K. The blurring operator K corresponds to convolution with Gaussian blur kernel of size 31 31 and standard deviation 3. Specifically, we define the degradation operator as H, (1 all + a,K, where E [0. 1] controls the degradation level. By varying we generate multiple degradation operators, allowing us to train the restoration network R to handle all these operators, expressed as R(s, H.)) = E [æ]s,, H.). where 8 is the degraded image and æ is the original image.
We select 1,000 different a values from the interval [0. 1), following the a schedule outlined by PSB (Liu et al., 2023). This results in 1, 1,000 different degradation operators H., effectively creating an ensemble of restoration priors during training. The model is trained using the Adam optimizer with learning rate of 5 10-5
Inference with Subset of the Ensemble. During inference, to simplify computation and focus the most effective priors, we use only subset of the supervised trained ensemble. Specifically, we select 6 a values, resulting in 6 restoration priors.
Step size and regularization parameter. To ensure fairness, for each problem setting, each method-botl both proposed and baseline- is fine-tuned for optimal PSNR using images from validation set separate from the test set. The same step size ? and regularization parameter then applied consistently across the entire test set.
Baseline details. We compare ShaRP against several denoiser- and diffuusin model-based methods. For denoiser-basse based approaches, we evaluate DPIR (Zhang et al., 2022), which relies on half half-quadratic splitting (HQS) iterations with DRUNet denoisers. For diffusion model-based methods, we compare with DPS (Chung et al., 2023). DDNM (Wang et al., 2023), and DiffPIR (Zhu et al., 2023). These methods all use the same pre- pre-trained diffusion models as priors, but each employs a distinct posterior sampling strategy to solve general inverse problems. We specifically use the pre-trained diffusion model from DifiPIR. We also compare our approach with DRP (Hu al., 2024c), method that leverages deep restoration priors for solving general inverse problems. For a fair comparison, of degradation priors, DRP uses single fixed prior. For a fair comparison, DRP. However, instead of employing by fixed a based on PSNR performance on the validation set, and applied it selected the optimal fixed
the same pre-trained deblurring network as in ShakP for
accordingly. For all baselines, we fine-tuned their parameters to maximize PSNR performance. Notably, the diffusion model backbone for all diffusion- based baselines was trained on the same dataset used to train our restoration prior.
20

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
C. Additional results for CS-MRI
C.1. Performance of ShaRP for uniform and random subsampling setting
Due to space constraints, we present only the quantitative performance for the 4x uniform subsampling setting in the main paper. In this section, we further evaluate ShaRP's s performance on both uniform random subsampling setting, with two sub-sampling rates (4x and 6x), and three noise levels (a 0.005, 0.01, and 0.015).
4x Uniform
6x Uniform
Noise level
0.005
0.010
0.015
0.005
0.010 =00.11
Metrics
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
Zero-filled
26.93 0.848 26.92 0.847 26.90 0.848 22.62 0.728 22.60 0.726 22.59 0.721
TV
31.17 0.923 31.08 0.921 30.91 0.915 25.00 0.806 24.94 0.803 24.33 0.755
PnP-FISTA
35.88
0.938 31.14 0.894 30.32 0,846 26.30 0.822 25.97 0.786 25.46 0.747
PnP-ADMM
35.76
0.941 32.36 0.878 30.66 0.838 26.13 0.808 25.90 0.776 25.51 0.742
DRP
35.52 0.936 32.32 0.914 30.57 0,901 29.51 0,872 28.52 0.882 28.35 0.876
DPS
32.62
0.888 31.39 0.870
30.29
0.856 30.53
0.862 29.41 0.843
28.63 0.830
DDS
35.21
0.937 35.03
0.935
34.51
0.925 31.02
0.889 30.84
0.888
30.79 0.888
ShaRP 37.59 0.963 35.81 0.951 34.92 0.942 33.42 0.940 32.86 0.932 32.09
0.922
Table 5: Quantitative comparison of ShaRP with several baselines for CS-MRI using uniform masks at undersampling rates of 4 and on fastMRI dataset. The best and second best results are highlighted. Notably, ShaRP outperforms SOTA methods based on denoisers and diffusion models.
4x Random
6x Random
Noise level
σ = 0.005
a 0.010 a= 0.015
T = 0.005 a = 0.010 o a= 0.015
Metries
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
Zero-filled
25.83 0.815 25.81 0.812 25.76 0.807 22.68 0.724 22.67 0.722 22.67 0.719
TV
28.14 0.866 28.06 0.863 27.96 0,859 24.55 0.782 24.33 0.750 24.28 0.736
PnP-FISTA
29.31
0.863 28.40 0.817 27.49 0.799 26.01 0.797 25.63 0.756 24.94 0.717
PaP-ADMM
28.83
0.842 28.39 0.816 27.70 0.786 25.59 0.776 25.19 0.740 24.93 0.728
DRP
29.97
0.880 29.37 0.839 28.31 0.794 26.98 0.866 26.78 0.853 26.49 0.821
DPS
31.72
0.874 30.45 0.857 29.50 0.843 30.32 0.856 29.36 0.824 27.99 0.810
DDS
32.41 0.910 32.37 0.906 32,25 0.901 30.59 0.876 30.35 0.874 30.31 0.879
ShaRP
34.66 0.949 33.57 0.920 33.18 0.931 31.53 0.924 31.46 0.918 31.45 0.914
Table 6: Quantitative comparison of ShaRP with several baselines for CS-MRI using random masks at undersampling rates of 4 and 6 on fastMRI dataset. The best and second best results are highlighted. Notably, ShaRP outperforms SOTA methods based on denoisers and diffusion models.
Table 6 provides quantitative comparison of reconstruction performance across different acceleration factors and noise levels using uniform sub-sampling mask. In all configurations, ShaRP consistently outperforms the baseline methods. The use of a set of restoration operators clearly enhances ShaRP's performance, highlighting the effectiveness of employing multiple operators to maximize the regularization information provided by the restoration model. Figure 6 presents visual reconstructions for two test scenarios, where ShaRP accurately recovers fine brain details, particularly in the zoomed-in regions, while baseline methods tend to oversmooth or introduce artifacts. These results highlight ShaRP's superior ability to manage structured artifacts and preserve fine details, outperforming both denoiser-based and diffusion model-based methods.
21

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
PnP-ADMM
DPS
DDS
ShaRP
Groundtruth
38.75 0.973
29.66 UNNN9
39.46 0.976
45.75 / 0.99
PSNR SSIM
0.050
0.025
0.000
22.23 0.633
27.48 0.864
29.66 0.912
30.23 / 0.920
PSNR SSIM
0.2 0.1
0.
Figure 5: Visual comparison of ShaRP with baseline methods CS-MRI. The top row shows results for 4x random mask with noise 0.005, and the bottom row for 6x random mask with noise 0.015. PSNR and SSIM values are in the top-left corner of each image. Error maps and zoomed-in areas highlight differences. Notably, ShaRP with stochastic priors outperforms state-of-the-art methods using denoiser and diffusion model priors.
PaP-ADMM
DPS
DDS
ShaRP
Groundtruth
27.91 0.787
28.29 / 0.844
30.02 0.875
31.14 / 0.886
PSNR / SSIM
0.2
0.1
0.0
27.04 0.80
27.48 99
27.60 06
30.11 090
PSNR SSIM
0.2
0.1
0.0
Figure 6: Visual comparison of ShaRP with baseline methods CS-MRI for 6x random sampling mask with noise 0.015. PSNR and SSIM values are in the top-left comner of each image. Error maps and zoomed-in areas highlight differences. Notably, ShaRP with stochastic priors outperforms state-of-the-art methods using denoiser and diffusion model priors.
22

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
C.2. Performance of ShaRP with Self-supervised restoration priors
x Random
6x Random
Noise level
0.005
0.010 o 0.015
o 0.005 a 0.010
0.015
Metries
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
PnP-ADMM
28.83
0.842 28.39 0.816 27.70 0.786 25.59 0.776 25.19 0.740 24.93 0.728
ADMM-TV
28.14
0.866 28.06 0.863 27.96 0.859 24.55 0.782 24.33 0.750 24.28 0.736
GRAPPA
28,09
0.792 25.39 0.699 23.94 0.649 25.67 0.737 23.72 0,646 22.51 0,595
SPICER
31.87
0.901 31.67
0.889 31.50
0.887 30.18 0.871 30.05 0.863
30.01 0.860
ShaRpadl 33.87 0.909 33.64 0.900 33.21 0.892 30.87 0.899 30.36 0.890 30.21 0.875
Table 7: PSNR (dB) and SSIM values for ShaRP with a self-supervised pre-trained restoration operator, compared to several baselines for CS-MRI with random undersampling masks at rates of 4 and 6 on the fastMRI dataset. The best and second best results are highlighted. For reference, the highlighted row presents a PnP method using a Gaussian denoiser, which requires fully sampled data for training. Note the excellent performance of ShaRP even using priors trained without fully-sampled ground-trute data.
C.3. Convergence Performance of ShaRP with supervised and self-supervised priors
Figure 7 illustrates the convergence behavior of ShaRP with both supervised and self-supervised learned restoration priors on the test set with an acceleration factor R = 6 and additional noise σ = 0.01. As observed, ShaRP converges stably under both settings. However, the self-supervised prior exhibits a performance gap compared to the supervised one, indicating that does not serve as a perfect MMSE estimator in practice. Despite this, still enables stable convergence as an ensemble of priors for ShaRP, which aligns with our convergence analysis for inexact MMSE estimators presented in Proposition
5xx 10-4
18
5x 165-
36
5x 10-
36.5
x 10r-
34
8
26
26
26
25
4x Uniform (Supervised)
Random (Supervised)
Unilorm (Self-upervised) 300
Random (Self-uueervsed)) 300
Figure 7: Convergence of ShaRP for 4x accelerated MRI reconstruction on the fastMRI dataset. (a)-(b) depict the convergence behavior of ShaRP using restoration operators trained in a supervised manner, while (c)-(d) correspond to those trained in self-supervised manner. The plots illustrate the average distance |æ* x and PSNR relative the ground truth, function of the iteration number, with shaded regions representing the standard deviation. Note the stable convergence of ShaRP with both types of priors.
23

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
C.4. Performance of additional baseline methods matched and mismatched settings
In this section, we highlight an important observation: pre-trained restoration networks typically exhibit poor generalization to mismatched settings. We chose two commonly used methods (SwinIR (Liang et al., 2021) and E2E-VarNe, (Sriram et al., 2020)) for the specific setting of CS-MRI. We trained them the same 8x uniform subsampling setting restoration prior and directly applied them to solve both matched and mismatched problems, as ShaRP did. As shown in the Table 8, the baseline method's performance drops significantly under mismatched conditions, whereas ShaRP maintains stable performance and convergence guarantees. This demonstrates ShaRP's ability to adapt pre-trained restoration models as priors and use to solve problems under mismatched settings, As shown in the Figure 8, due to the mismatched settings, the two baseline methods suffer from over-smoothing, lack important details, and exhibit artifacts, whereas ShaRP still provides high-quality reconstruction performance. This indicates that ShaRP balance data fidelity and the artifact removal capabilities of the prior model, leading to artifact-free reconstruction that preserves important details.
Settings
Uniform
Random 6x Uniform Random 8x Uniform
Metrics
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
SwinIR
24.78 0.849 25.09 0.841 29.55 0.907 27.98 0.819
29.37 0.898
E2r-VerNet 35.40 0.957 33.48 0.945 32.79 0.936 31.02 0.913
32.59 0.919
ShaRP 37.59 0.963 34.66 0.949 33.42 0.940 31.53 0.924 32.37
0.907
Table 8: Quantitative comparison of ShaRP with task-specific baselines trained on the 8x uniform mask. Baselines perform well in matched settings (highlighted in the table) but show a significant drop under mismatched conditions. contrast, ShaRP remains robust, handling both matched and mismatched scenarios effectively.
SwinIR
E2E-VarNet
SHARP
Groundtruth
26.49 0.869
32.19 0.955
33.48 /0944
PSNR SSIM
0.12 0.06
28.15 0.881
32.19 0.923
33.85 0.931
PSNR SSIM
Y
0.10
0.05
0.00
Figure 8: Visual comparison of ShaRP with task-specific baseline methods CS-MRI for 6x random sampling mask with noise 0.015. PSNR and SSIM values are in the top-left corner of each image. Error maps and zoomed-in areas highlight differences. Notably, ShaRP with stochastic priors outperforms state-of-thhe the-trrtrt methods using denoiser and diffusion model priors.
C.5. Additional Evaluation: Non-Cartesian CS-MRI Generalization
In this section, we present an additional evaluation designed to further assess ShaRP's generalization capabilities. This new experimental setting explores Cn-CCaressian sampling for Compressed Sensing MRI (CS-MRI), specifically employing a
24

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
Gaussian sampling mask. A key aspect of this evaluation is that the restoration network, originally trained for 8x uniform Cartesian sampling, was directly applied this unseen sampling scheme without any retraining fine-tuning. The results, summarized in Table 9, provide compelling evidence of ShaRP's robustness to diverse sampling patterns and highlight its strong generalization performance.
Table 9: Performance comparison CS-MRI with 2D Gaussian sampling mask. The restoration network trained on 8x uniform Cartesian sampling and applied directly without retraining or adaptation.
Method
PSNR
SSIM
PnP-ADMM
31.33
0.917
DPS
32.01
0.904
DDS
33.19
0.924
DRP
32.70
0.921
ShaRP
34.01
0.942
25

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
D. A Additional visual results for SISR
In this section, we present additional visual results to numerical comparisons for the SISR task.
D.1. Additional visual results against baselines
As illustrated in Figure and Figure 10, ShaRP outperforms all baseline approaches under both blur kernel settings, achieving higher PSNR and SSIM values. Moreover, we maintain superior data consistency with the measurements while achieving enhanced perceptual quality. The use of an ensemble deblurring priors enables our method to recover fine details at varying corruption levels, contributing to the improved performance.
Bicubic
DPIR
DDNM
DiffIR
ShaRP
Groundtruth
31.71 0.861
32.80/ 0.887
33.63 0.909
35.73 0.938
37.16 0.966
PSNR SSIM
25.90 0.791
27.96 0.8
27.83 0.825
28.67 0.837
29.54 0.900
PSNR / SSIM
Figure 9: Visual comparison of ShaRP with several well-known methods 2x SISR with gaussian blur kernel with 1.5. The quantities in the top-left corner of each image provide PSNR and SSIM values for each method. The squares at the bottom of each image visualize the zoomed area in the image.
Bicubic
DPIR
DDNM
DifPIR
ShaRP
Groundtruth PSNR 2 SSIM
28.71 00878
31.73 0.899
30.31/ 00.667
32.39 00.918
33.54 /..448
29.52 /0.809
28.92 0.8299
31.16 0.875
34.61 0.899
35.41 0.908
PSNR SSTA
Figure 10: Visual comparison of ShaRP with several well-known methods on 2x SISR with gaussian blur kernel with o = 1.25. The quantities in the top-left corner of each image provide PSNR and SSIM values for each method. The squares the bottom of each image visualize the zoomed area the image.
26

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
D.2. Additional visual results against DRP
To further emphasize the necessity and advantages of using an ensemble of deblurring priors, as opposed fixed prior like in DRP (Hu et al., 2024c), provide additional visual comparison results. As shown in Figure 11, ShaRP consistently recovers finer details, resulting in improved PSNR and SSIM scores, along with enhanced perceptual performance.
Bicubic
DRP
ShaRP
Groundtruth
26.30 0.777
29.40
0.864
30.33 0.884
PSNR SSIM
MOTEL
MOTEL
MOTEL
MOTEL
22.84 0.661
25.04 0.780
25.99 0.802
PSNR SSIM
29.90 / 0.795
33.50 / 0.879
34.49/0.900 0.903
PSNR SSIM
Figure 11: Visual comparison of ShaRP with DRP on 2x SISR with gaussian blur kernel with o = 1.5. The quantities in the bottom-left corner of each image provide PSNR and SSIM values for each method. The squares at the bottom each image visualize the zoomed area the image.
27

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
D.3. Additional Comparison against DDRM and DiffIR
To further evaluate ShaRP's 's performance against state-of-the-art diffusion-based methods, we included two additional baselines for comparison: DDRM (Kawar al., 2022) and DiffIR (Xia et al., 2023). The experiment setting is 2 SISR task with gaussian blur kernel with 1.25 ImageNet dataset. For DDRM, utilized the same pre- pre-trained unconditional diffusion backbone as DiffPRR DDNM, and DDS, but followed the sampling procedure outlined in their original paper. For DifiIR, we directly used the provided checkpoint from the authors.
Metries PSNR SSIM LPIPS
DPIR 28.10 0.809 0.305
DDNM
27.53 0.786 0.240
DPS 24.68
0.661 0.395
DifiPIR
28.92
0.852 0.152
DifIR 25.79 0,812 0.180
DDRM
28.20 0.845 0.161
DRP 29.28
0.868 0.207
ShaRP 30.09 0.891 0.179
Table 10: Quantitative comparison of ShaRP with several additional baselines for 2x SISR with gaussian blur kernel with 1.25 on ImageNet dataset. The best and second best results are highlighted. Notably, ShaRP outperforms SOTA methods based on denoisers and diffusion models.
Bicubic 25.37 0.853
DPIR
DDNM
DDRM
29.08 0.900
27.82 0.881
0.881
26.28 0.880
DiffIR
DiffPIR
ShaRP
Groundtruth
25.88 0.864
29.28 0.9
31.06/ 0.929
PSNR / SSIM
Figure 12: Visual comparison of ShaRP with additional baselines on 2x SISR with gaussian blur kernel with 1.25. The quantities in the bottom-left corner of each image provide PSNR and SSIM values for each method. The squares the bottom of each image visualize the zoomed area in the image.
28

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
D.4. Additional Evaluation: SISR with New Gaussian Kernel
To further investigate ShaRP's robustness, we ran an additional Super-Resolution (SISR) experiment using new Gaussian blur kemel with 1.0. This test aimed evaluate if ShaRP can maintain consistently good performance, comparable to its results with the other two kernels detailed in the main manuscript, with this previously unseen degradation. As shown in Table 11, ShaRP continues to provide excellent performance, suggesting its strong generalization capabilities and potential effectiveness across wider range of Gaussian blur levels.
Table 11: SISR performance with Gaussian blur kernel (o 1.0).
Method
PSNR (dB)
SSIM
LPIPS
FID +
DPIR
28.45
0,854
0.247
82.90
DDRM
27.26
0,803
0.209
44.77
DifIPIR
28.37
0,841
0.215
40.59
DRP
28.43
0.853
0.236
75.29
ShaRP
28.70
0.858
0.226
69.75
29

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
E. Additional Experiments
In this section, we include two additional ablation studies to further highlights ShaRP's capability to leverage restoration priors for solving general inverse problems, as well to evaluate its performance under different hyperparameter settings.
E.1. A Abltionn study on using SR prior for CS-MRI task
To demonstrate the flexibility of our approach in integrating diverse restoration models to address general inverse problems, we conducted an additional ablation study using a pre-trained super-resolution network as a prior for solving the CS-MRI problem.
Algorithm 5 MRI Super Resolution network training
Require: dataset:p(x. y). 4x bicubic downsampling operator: K, Ro(-,a)
repeat:
x ~ p(x), e- o ( ((O0, ming || ((1 (1- aD Dz; DD: a) - xz12
until converge
Models training for MRI-SR We use the same U-Net architecture as employed the official implementation DDS for R(;aa.. To create ensemble of restoration priors, consider a family of degradation operators that convex combinations of the identity mapping I and the Gaussian blur operator D. The 4x bicubic downsampling operator D corresponds to bicubic downsample with factor equals to 4. Specifically, we define the degradation operator as Ha = (1 a + aD D. where [0, 1] controls the degradation level. By varying a, we generate multiple degradation operators, allowing us to train the restoration network R to handle all these operators, expressed as R(s, H.) = E [a]ss H.., where 8 the degraded image and æ is the original image.
We select 1,000 different values from the interval [0, 1). following the a schedule outlined by I'SB (Liu et al., 2023). This results in 1, 1,000 different degradation operators H., effectively creating ensemble of restoration priors during training. The model is trained using the Adam optimizer with learning rate of 5 x 10-5
Using MRI-SR model as prior for CS-MRI task. During inference, to simplify computation and focus on the most effective priors, we use only a subset of the ensemble. Specifically, we select 6 a values, resulting in restoration priors.
As shown in Table 12, under the 4x uniform mask setting, employing the pre-trained MRI-SR model as prior allows ShaRP to outperform denoiser- and diffusion- based approaches. However, its performance remains inferior to ShaRP with mismatched CS- MRI-speccccfic prior. In the 4x random mask setting, ShaRP with the pre-trained MRI-SR model as prior continues to surpass PnP-based methods that utilize denoiser prior but performs worse than approaches based on diffusion models. Notably, ShaRP with a mismatched CS-MRI-specific prior consistently delivers the best performance.
Tasks
Metrics
PnP-FISTA
PnP-ADMM
DPS
DDS
ShaRPes
ShaRP SR
PSNR
35.88
35.76
32.62
35.21
37.59
35.91
4x Uniform
SSIM
0.938
0.941
0.888
0.937
0.961
0.943
PSNR
29.31
28.83
31.72
32.41
34.66
30.91
4x Random
SSIM
0.863
0.842
0.874
0.910
0.949
0.905
Table 12: Quantitative comparison of ShaRP against baselines for CS-MRI reconstruction using 8x CS-MRI and 4x super-resolution priors, evaluated on the fastMRI dataset. Results are reported for both uniform and random undersampling masks at a 4x undersampling rate. The best and second best results are highlighted.
30

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
E.2. Ablation Study on the number of restoration priors included in the ensemble
To evaluate the impact of the number of restoration priors, on ShaRP's performance, conducted ablation study in the 4x CS-MRI setting with random sampling masks. We specifically examined how varying b influenced reconstruction performance, offering valuable insights into ShaRP's sensitivity to this parameter and its role in achieving optimal results.
As illustrated in Figure 13 and Figure 14, increasing b, which corresponds to incorporating more restoration priors in the ensemble, generally enhances ShaRP's reconstruction performance.
0.4
Figure 13: Performance comparison of ShaRP's CS-MRI reconstruction at 4x acceleration with varying numbers of restoration priors, b. Left: PSNR vs. b; Right: SSIM vs. b. ShaRP with = 8 consistently achieves superior results, highlighting the performance improvements gained by incorporating more restoration priors into ShaRP.
b=2
b=4 4
b= 6
GroundTruth
0.12
0.06
0.00
Figure 14: Visual comparison of ShaRP with varying amounts of restoration priors, denoted by 6, in the ensemble. The PSNR and SSIM values for each method are shown in the top-left corner of each image. Zoomed-in regions, highlighted as squares the bottom of each image, provide closer look key details. Notably, increasing the number of restoration priors in the ensemble enhances visual performance effectively reducing artifacts and capturing finer details.
31

---

Stochastic Deep Restoration Priors for Imaging Inverse Problems
E.3. Ablation Study on the Influence of Hyperparameter
To evaluate the impact of the hyperparameters as introduced in Section B.1, we conducted ablation study in the 4x CS-MRI setting using random sampling masks, where governs the selection of a specific restoration prior. Specifically, analyzed how varying the values of influenced reconstruction performance. As shown in Figure 15 demonstrates the influence of on performance. A very small a fails to provide sufficient regularization to constrain the solution, while an excessively large a overly restricts the model, leading to a decline in performance. These findings highlight the importance of appropriately tuning a and b to balance flexibility and regularization for optimal results.
PGNR)AI
SGNN
34.
1.5
070
000
6.0
Figure 15: Performance comparison of ShaRP's CS-MRI reconstruction at 4x acceleration with varying a. Left: PSNR vs. a; Right: SSIM vs. a.
32


Reviews:
[{"review_id": "UMP5QKAF0o", "reviewer": "Reviewer_X9oP", "summary": "The paper develops an plug-and-play imaging restoration algorithm that can use MMSE estimators trained to solve an arbitrary inverse problem involving linear forward operators and white gaussian noise (not just denoising), to solve a target linear inverse problem. That is, one can, for example, use a network trained to deblur and denoise images to solve a compressive sensing restoration task. The authors show the algorithm corresponds to taking biased stochastic gradients wrt the true MMSE loss. The proposed method is tested on compressive MRI and and single image super-resolution and is shown to generally outperform existing methods.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for feedback and thoughtful comments on our work.\n\n> 1. *The paper differentiates itself from related self-supervised methods by stating \"Ambient DMs seek to sample from px using DMs trained directly on undersampled measurements. Thus, during inference Ambient DMs assume access to the image prior px, while ShaRP only assumes access to the ensemble of likelihoods of multiple degraded observations.\" However, because p_x was learned directly from undersampled measurements, the assumptions on Ambient GAN and related methods don't seem any more restrictive than those on the current method (which I would argue has also implicitly learned p_x).*\n\nIndeed, both Ambient DMs and ShaRP indeed implicitly learn about p(x) and we will revise that section of the paper to make this point clear. \n\n> 2. *We introduce a novel regularization concept for inverse problems that encourages solutions that produce degraded versions closely resembling real degraded images.\" This seems conceptually similar to the equivariant imaging concept. Can the authors comment on the relationship between the two works. Is there a relationship between Theorem 2 and equivariant imaging?*\n\nThis is a very interesting perspective. Our approach can indeed be seen as finding a fixed point that exhibits equivariance under multiple degradations. The revised paper will include this nice interpretation suggested by the reviewer. \n\n> 3. *May be worth discussing the following paper: Bansal, Arpit, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, and Tom Goldstein. \"Cold diffusion: Inverting arbitrary image transforms without noise.\" Advances in Neural Information Processing Systems 36 (2023): 41259-41282.*\n\nWe will cite and discuss Cold Diffusion in the revised version. Both Cold Diffusion and Ambient Diffusion (mentioned above) point toward a promising direction for extending our method to sampling, which we will highlight as a potential avenue for future work.\n\n> 4. *It might be better to rewrite (1) and (2) with complex-valued forward models, given the focus on MRI.*\n\nWe will fix this per your suggestion.\n \n> 5. *At least in the context of an ensemble of blur-then-deblur operations, ShaRP can be interpreted as masking then restoring a portion of k-space: ShaRP could be viewed as an ensemble of masked (denoising) autoencoders.*\n\nIndeed, when considering an ensemble of blur-then-deblur operations, ShaRP can be seen as masking and subsequently restoring portions of k-space—akin to the mechanism of masked (denoising) autoencoders. In this interpretation, MMSE restoration networks effectively reconstruct the missing null-space signals. We will incorporate this connection into the discussion section of our revised manuscript.\n \n> 6. *A subscript A in $\\nabla(g)$ in Algorithm 1 (i.e., $\\nabla_A(g)$ ) might be useful to remind readers where the forward model comes into play in the reconstruction algorithm. (Or just some additional annotation that g(x)) is the data fidelity term.)*\n\nWe will modify the notation accordingly.\n \n> 7. *What is s' in (12)? Are the losses ell_sup and ell_self wrt \\bar{x}? The notation in this section could use more annotation.*\n\nIn Eq. (12),  s’ refers to an independently subsampled measurement, defined as  s’ = P’M with P’ denoting a separate sampling pattern. We will revise the notation and add clarifying annotations in the revised version.\n\n> 8. *In the MRI restoration task, the trained restoration algorithms are solving problems very similar to the target application; both are solving subsampled MRI, just with different masks. Is the proposed method still effective when the restoration algorithms are solving a very different task? E.g., can I solve compressive MRI by regularizing with an algorithm trained to perform image deblurring or inpainting?*\n\nSection E.1 of our paper explores the scenario suggested by the reviewer by applying a pre-trained super-resolution (SR) model to the compressive sensing MRI problem. Despite the task mismatch, the SR prior still outperformed the Gaussian denoiser prior. We will edit the manuscript to ensure that it is clear this experiment was included.", "claims_and_evidence": "Claims are supported with evidence.", "methods_and_evaluation_criteria": "The methods and eval make sense", "theoretical_claims": "The theory seems correct, though Section 4.3 could use further explanation.", "experimental_designs_or_analyses": "Evaluation criteria are appropriate", "relation_to_broader_scientific_literature": "The paper presents a thorough overview of the existing literature.\n\nThe paper differentiates itself from related self-supervised methods by stating \"Ambient DMs seek to sample from px using DMs trained directly on undersampled measurements. Thus, during inference Ambient DMs assume access to the image prior px, while ShaRP only assumes access to the ensemble of likelihoods of multiple degraded observations.\" \nHowever, because p_x was learned directly from undersampled measurements, the assumptions on Ambient GAN and related methods don't seem any more restrictive than those on the current method (which I would argue has also implicitly learned p_x).\n\n\"We introduce a novel regularization concept for inverse problems that encourages solutions that produce degraded versions closely resembling real degraded images.\" This seems conceptually similar to the equivariant imaging concept. Can the authors comment on the relationship between the two works. Is there a relationship between Theorem 2 and equivariant imaging?", "essential_references_not_discussed": "May be worth discussing the following paper:\nBansal, Arpit, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, and Tom Goldstein. \"Cold diffusion: Inverting arbitrary image transforms without noise.\" Advances in Neural Information Processing Systems 36 (2023): 41259-41282.", "other_strengths_and_weaknesses": "This is a well-written paper that studies an important problem. The proposed method is reasonably novel and apparently effective.", "other_comments_or_suggestions": "It might be better to rewrite (1) and (2) with complex-valued forward models, given the focus on MRI.\n\nAt least in the context of an ensemble of blur-then-deblur operations, ShaRP can be interpretted as masking then restoring a portion of k-space: ShaRP could be viewed as an ensemble of masked (denoising) autoencoders.\n\nA subcript A in $\\nabla g(x)$ in Algorithm 1 (i.e., $\\nabla g_A(x)$) might be useful to remind readers where the forward model comes into play in the reconstruction algorithm. (Or just some additional annotation that g(x)) is the data fidelity term.)\n\nWhat is s' in (12)? Are the losses ell_sup and ell_self wrt \\bar{x}? The notation in this section could use more annotation.", "questions_for_authors": "In the MRI restoration task, the trained restoration algorithms are solving problems very similar to the target application; both are solving subsampled MRI, just with different masks. Is the proposed method still effective when the restoration algorithms are solving a very different task? E.g., can I solve compressive MRI by regularizing with an algorithm trained to perform image deblurring or inpainting?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eZsz19foPq", "reviewer": "Reviewer_dZ9e", "summary": "The paper presents Stochastic Deep Restoration Priors (ShaRP), a framework for imaging inverse problems that leverages an ensemble of pre-trained deep restoration models. ShaRP uses a stochastic gradient descent approach where, in each iteration, a random degradation operator (with added Gaussian noise) is applied to simulate degradations. It minimizes a composite objective combining a data fidelity term with a regularizer derived from MMSE restoration operator score functions.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback.\n> Response to concerns in Claims and Evidence\n\n1. Generalization over configurations: We wanted to highlight that the supplementary material contains evidence supporting the robustness of ShaRP across diverse inverse problems without retraining, particularly under self-supervised conditions from incomplete measurements. Specifically:\nAs detailed in Section C, we conducted experiments in the CS-MRI setting evaluating ShaRP's performance under a range of challenging conditions, including:  (1) Different undersampling rates (4x, 6x), (2) Varied mask types (Cartesian uniform, Cartesian random, and 2D Gaussian), (3) Multiple noise levels (0.005, 0.1, 0.15).\nThese results demonstrate ShaRP's ability to handle diverse inverse problems without retraining, even with self-supervised learning from incomplete data.\n\n2. Convergence Stability and Robustness: Section C.3 of the supplementary material presents an analysis of ShaRP's convergence stability and robustness. We evaluated this using both supervised and self-supervised MMSE restoration priors, providing empirical support for our theoretical guarantees.\n\nPrompted by the reviewer comment, the revised paper will include: \n(1) A summary table that consolidates the key ablation results across different undersampling rates, mask types, and noise levels. This will provide a clearer and more accessible overview.\n(2) Add a brief discussion in the main text highlighting these empirical findings and explicitly linking them to the claim of robustness without retraining.\nWe believe these additions will strengthen our paper by highlighting extensive evidence provided in the paper.\n\n> Response to concerns in Theoretical Claims\n\nPrompted by the reviewer the revised paper will better highlight the following:\n1. Lipschitz continuity and boundedness are needed only for Proposition 1. They are not needed for our main results—Theorem 1 and Theorem 2.\n2. Lipschitz continuity is a standard and widely-used assumption in optimization for establishing convergence rates of gradient-based algorithms (see, for example, Section 1.2.2 in [1]). It is satisfied by a broad class of objective functions, including the least-squares data-fidelity term used in our experiments. The smoothness of MMSE restoration operators is well-known and has been extensively discussed in literature (see, for example, [2]).\n3. Boundedness is a mild assumption, since it is always true for images that have bounded pixel values [0, 255].\n4. Figure 7 in Section C.3 shows empirically convergence of our method in a real-world setting.\n\n[1] Nesterov, Introductory Lectures on Convex Optimization, 2004\n\n[2] Gribonval and Machart, Reconciling “priors” & “priors” without prejudice?, NeurIPS 2013.\n\n>  Response to concerns in Experimental Designs Or Analyses\n\n1. Prompted by your comment, we will include a table summarizing all the relevant information about the experiment to guarantee reproducible results. We will additionally release our code upon acceptance, which should further simplify reproducibility.\n\n2. Section E.2 of the supplementary material shows clear performance gains as ensemble size increases.\n\n> Respond to concerns in Essential References Not Discussed\n\nThank you for pointing to  relevant works such as \"Deep Image Prior,\" \"Noise2Void,\" and \"Self2Self.\" We will cite and discuss these works in our revised manuscript.\n\n> Response to concerns in Other Strengths And Weaknesses\n\nWe appreciate the reviewer's assessment of our paper's originality and contributions. Following your recommendations, the revised manuscript will include summary tables pointing to the relevant results and will improve the discussion on the assumptions (see response to your Comment 2).\n\n> Response to concerns in Other Comments Or Suggestions\n\nThe revised manuscript will explicitly mention that the computational cost of running ShaRP is comparable to those of single-model approaches. This is due to the stochastic nature of our algorithm that uses a $single$ restoration operator in each iteration.\n\n> Response to concerns For Authors\n\n1. The smoothness of h associated with the MMSE restoration operators is well-known (see for example [1] or Appendix B of [2]). The smoothness can also be seen in the derivation presented in Appendix A of our paper. This is a direct consequence of the smoothness of the p(s | H) in eq. (5), since it is a Gaussian convolved with the prior p(x). Note also that since our restoration operator is implemented as a neural network with smooth activation functions like eLU, it will be inherently Lipschitz continuous [2].\n2. (1) The detailed results of ablation studies isolating the effect of the ensemble are available in Section E.2, which is in the supplementary material. (2) There is no computational overhead due to our approach since only one model from the ensemble is utilized for each restoration instance. We will explicitly state this in the revised manuscript.", "claims_and_evidence": "While the paper presents theoretical derivations and experimental results to support its contributions, some claims are not fully substantiated by clear and convincing evidence. For example, the claim that ShaRP can robustly handle diverse inverse problems without retraining—especially under self-supervised conditions from incomplete measurements—is supported by only a limited set of experiments and comparisons. Additionally, the theoretical guarantees regarding convergence and the robustness of the proposed regularizer would benefit from more extensive empirical validation and ablation studies.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria appear to be appropriate for the targeted imaging inverse problems. The selection of tasks like MRI reconstruction and single image super-resolution provides a relevant and practical basis for evaluation.", "theoretical_claims": "The proofs assume a level of smoothness and differentiability for the restoration operators that may not always hold for complex deep networks.\nCertain Lipschitz continuity and boundedness conditions are assumed without full justification.\nThe convergence analysis is presented under idealized conditions, and the impact of real-world deviations from these assumptions is not fully explored.", "experimental_designs_or_analyses": "The experimental design was examined for MRI reconstruction and single-image super-resolution tasks using standard benchmarks and evaluation metrics, with comparisons made against established baselines. However, there are several issues: details on dataset splits, noise levels, and degradation patterns are sometimes insufficient, which could affect reproducibility; baseline methods may not be equally tuned, potentially affecting fairness; and the analysis lacks extensive ablation and sensitivity studies to clearly disentangle the contributions of the ensemble approach and MMSE-based regularizer.", "relation_to_broader_scientific_literature": "It builds upon the extensive literature on plug-and-play priors and regularization by denoising by moving beyond single Gaussian denoisers to an ensemble of restoration models, echoing ideas seen in recent works on deep restoration priors and stochastic denoising regularization. Additionally, it relates closely to diffusion model approaches that leverage score functions for sampling, while providing a novel MMSE-based regularizer that bridges the gap between traditional optimization methods and deep learning.", "essential_references_not_discussed": "While the paper cites a broad range of works on plug‐and‐play priors, denoising-based regularization, and diffusion models, it omits a few related works that could provide additional context for its contributions. For example, \"Deep Image Prior\" demonstrates that the inherent structure of an untrained convolutional network can serve as a powerful image prior, which is relevant to understanding unsupervised restoration strategies. Additionally, more recent self-supervised approaches such as Noise2Void and Self2Self that learn priors from corrupted or incomplete data could further clarify the paper’s context in scenarios without fully sampled measurements.", "other_strengths_and_weaknesses": "The paper is original in its integration of multiple pre-trained restoration models to serve as image priors, moving beyond the traditional reliance on single Gaussian denoisers. This ensemble approach, coupled with a novel MMSE-based regularizer, is a contribution that addresses diverse inverse problems more flexibly. However, some weaknesses include idealized assumptions in the theoretical proofs and a need for more extensive ablation studies and sensitivity analyses to fully validate the empirical findings.", "other_comments_or_suggestions": "A discussion on the practical computational cost of using an ensemble of restoration models, especially compared to single-model baselines, would be beneficial.", "questions_for_authors": "Could you provide a more detailed description of the assumptions underlying your convergence proofs, particularly regarding the smoothness and Lipschitz continuity of the restoration operators? \n\nHave you conducted ablation studies to isolate the impact of the ensemble approach versus the MMSE-based regularizer component? \nWhat is the computational overhead of using an ensemble of restoration models compared to single-model approaches?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hBlj0VTSJW", "reviewer": "Reviewer_DuT7", "summary": "This paper introduces ShaRP, a stochastic regularization for linear inverse problems of the form y=Ax+e. This regulatization relies on MMSE (approximated) restoration machines R(*) for problems of the form s=Hx+n, where H is randomly chosen. By injecting the gradient of this regularization within the iterative recovery algorithm, and choosing at each itreration a differnt H to serve, ShaRP leads to improved recovery results in terms of PSNR, SSIM, (and LPIPS for SISR tests).", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback.\n\n> 1. *It is unclear what ShaRP is after - is it a better MAP estimate? or perhaps an MMSE one? Clearly, this is not a posterior sampler. This question is critical, ..., as they they tend to be somewhat blurry.*\n\nThis is a great point. ShaRP is not designed to produce classical MAP or MMSE estimates, nor is it a posterior sampler. Instead, it optimizes a novel objective function (Eq. (2) using h(x) in Eq. (6)) that balances data fidelity with an implicit regularization learned from an ensemble of priors. While the resulting reconstructions may exhibit characteristics similar to MMSE solutions (e.g., in terms of PSNR and visual smoothness), this is a consequence of the learned objective rather than an explicit design goal. We acknowledge that exploring sampling-based extensions could potentially enhance perceptual performance, and we will include this as a direction for future work.\n\n> 2. *In any case, a directly trained MMSE solver should be included ...*\n\nThank you for the suggestion. For 4× CS-MRI, we added E2E-VarNet as a new baseline, which is a dedicated MMSE solver with data-fidelity constraint that achieves higher PSNR but is task-specific (i.e., [*] needs to be retrained for each task), effectively representing an upper bound on MSE performance.\n\n| Method        | PSNR (σ=0.05) | SSIM (σ=0.05) | PSNR (σ=0.10) | SSIM (σ=0.10) | PSNR (σ=0.15) | SSIM (σ=0.15) |\n|--------------|---------------|---------------|---------------|---------------|---------------|---------------|\n| Zero-filled  | 26.93         | 0.848         | 26.92         | 0.847         | 26.90         | 0.848         |\n| TV           | 31.17         | 0.923         | 31.08         | 0.921         | 30.91         | 0.915         |\n| PnP-FISTA    | 35.88         | 0.938         | 31.14         | 0.894         | 30.32         | 0.846         |\n| PnP-ADMM     | 35.76         | 0.941         | 32.36         | 0.878         | 30.66         | 0.838         |\n| DRP          | 35.52         | 0.936         | 32.32         | 0.914         | 30.57         | 0.901         |\n| DPS          | 32.62         | 0.888         | 31.39         | 0.870         | 30.29         | 0.856         |\n| DDS          | 35.21         | 0.937         | 35.03         | 0.935         | 34.51         | 0.925         |\n| ShaRP        | 37.59         | 0.963         | 35.81         | 0.951         | 34.92         | 0.942         |\n| E2E-VarNet [*] | 38.10       | 0.971         | 36.80         | 0.967         | 35.79         | 0.954         |\n\n\n> 3. *Also, as this work compares various different methods ... perceptual quality evaluation.*\n\nSee our answer to Comment 4 below.\n\n> 4. *Several comments above suggest that the comparisons are not complete and not fair: FID/KID results are missing - does ShaRP aim for high perceptual quality? If not, say so, but then, what is it aiming for? MMSE restoration is missing.*\n\nWe clarify that ShaRP primarily aims for high reconstruction accuracy rather than explicitly targeting perceptual quality. Nevertheless, we agree perceptual quality metrics provide valuable insights (in particular regarding the Perception Distortion tradeoff [3]). Hence, we have included FID alongside PSNR, SSIM, and LPIPS to offer a balanced assessment:\n| Method   | PSNR  | SSIM  | LPIPS | FID   |\n|----------|-------|-------|--------|--------|\n| DPIR     | 27.90 | 0.803 | 0.314  | 89.18  |\n| DPS      | 24.50 | 0.657 | 0.403  | 50.33  |\n| DiffPIR  | 28.59 | 0.834 | 0.172  | 46.12  |\n| DRP      | 28.24 | 0.836 | 0.235  | 64.23  |\n| ShaRP    | 29.28 | 0.872 | 0.209  | 58.79  |\n\n[3] Y. Blau., and T. Michaeli. \"The perception-distortion tradeoff.\" CVPR, 2018.\n\n> 5. *A reference to RED-Diff is very much missing.*\n\nWe will cite RED-Diff in the revised manuscript.\n\n> 6. *The paper is hard to follow in Section 3 - the order of the presentation is flawed ...*\n\nThis is valuable feedback. We will implement the following changes: We will begin by clearly defining all necessary functions and terms, with a specific definition of h(). Following this, we will introduce Algorithm 1. We believe this revised structure will significantly improve the clarity and flow of Section 3.\n\n\n> 7. *A clear definition of the obje3ctive or ShaRP is missing ...*\n\nThe objective function minimized by ShaRP is the composite function consisting of a data fidelity term g and a novel regularizer h specified in eq. (6). See also our response to Comment 1.\n\n> 8. *In equation 3, why not have sigma be random as well? ...*\n\nWhile ShaRP generates a random noise at each iteration, it does so using fixed sigma. Using random sigma is an interesting idea that we could explore in the future.\n\n> 9. *Equation 6 is very hard to follow, and esspecially so since you do not show the chosen h() beforehand. It would be helpfull to show how the actual h() is generalized to lead to (6).*\n\nWe will revise the paper to better explain Equation (6).", "claims_and_evidence": "The claims made throughout the paper are well-supported.", "methods_and_evaluation_criteria": "It is unclear what ShaRP is after - is it a better MAP estimate? or perhaps an MMSE one? Clearly, this is not a posterior sampler. This question is critical, because the tables in the results' section suggest that we are after an MMSE solution with the best PSNR, and the visual results (Figure 2 and 3) support this belief, as they they tend to be somewhat blurry. \n\nIn any case, a directly trained MMSE solver should be included in the comparisons, as it is nothing but a special case of the R operator that has been learned. \n\nAlso, as this work compares various different methods that strive for different goals (e.g. DPS and DDNM as posterior samplers), this must be accompanied with perceptual quality evaluation.", "theoretical_claims": "The theoretical part of this paper is solid and beautiful.", "experimental_designs_or_analyses": "Several comments above suggest that the comparisons are not complete and not fair:\n- FID/KID results are missing - does ShaRP aim for high perceptual quality? If not, say so, but then, what is it aiming for?\n- MMSE restoration is missing", "relation_to_broader_scientific_literature": "OK in general", "essential_references_not_discussed": "A reference to RED-Diff is very much missing. See here: https://openreview.net/forum?id=1YO4EE3SPB", "other_strengths_and_weaknesses": "Strengths:\n- Beautiful theoretical work\n- Lovely idea that extends beyond RED and DRP.\n\nWeaknesses:\n- The paper is hard to follow in Section 3 - the order of the presentation is flawed, as it starts by referring to Algorithm 1, never define h(*), then jumps to talk about the gradient of f(*) - it is nearly impossible to follow. \n- A clear definition of the obje3ctive or ShaRP is missing, and this could give better context to the results obtained.", "other_comments_or_suggestions": "In equation 3, why not have sigma be random as well? In this case you get closer to RED-Diff mentioned above. \n\nEquation 6 is very hard to follow, and esspecially so since you do not show the chosen h(*) beforehand. It would be helpfull to show how the actual h(*) is generalized to lead to (6).", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MuSZN5lSMh", "reviewer": "Reviewer_w5bu", "summary": "This paper introduces stochastic deep restoration priors, a framework leveraging an ensemble of pre-trained restoration models as priors for solving imaging inverse problems. The authors claim they minimizes a regularizer based on degraded observation likelihoods, generalizing denoiser-based methods like RED and SNORE. Experiments on MRI reconstruction and super-resolution demonstrate SOTA performance. The main contributions include theoretical convergence guarantees, adaptation to self-supervised training without fully sampled data, and improved handling of structured artifacts.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback.\n\n>1. *For concerns in Methods And Evaluation Criteria:*\n\nPrompted by your comment, we ran additional experiments that we will include in the supplementary material of the paper. The new setting considers non-Cartesian sampling for CS-MRI, using a 2D Gaussian sampling mask. Importantly, the restoration network trained for the 8× uniform Cartesian sampling was applied directly without retraining or adaptation. Our results, summarized below, provide additional evidence of ShaRP's robustness to diverse sampling patterns, highlighting its strong generalization ability.\n\n| Method     | PSNR  | SSIM  |\n|------------|-------|-------|\n| PnP-ADMM   | 31.33 | 0.917 |\n| DPS        | 32.01 | 0.904 |\n| DDS        | 33.19 | 0.924 |\n| DRP        | 32.70 | 0.921 |\n| ShaRP      | 34.01 | 0.942 |\n\nIt would certainly be interesting to explore additional natural distortions in SISR tasks, such as motion blur, in future work. We will mention this revised manuscript.\n\n>2. *For concerns in Theoretical Claims:*\n\nThanks for your suggestion. We will include an expanded discussion about the two technical assumptions. Lipschitz continuity is a standard and widely-used assumption in optimization, needed for establishing convergence rates of gradient-based algorithms (see, for example, Section 1.2.2 in [1]). It is satisfied by a broad class of objective functions, including the least-squares data-fidelity term used in our experiments, as well as neural networks (e.g., Appendix B in [2]). Boundedness is a mild assumption, since it is always true for images that have bounded pixel values [0, 255]. \n\n[1] Nesterov, Introductory Lectures on Convex Optimization, 2004\n[2] Hurault et al. Gradient step denoiser for convergent plug-and-play. ICLR, 2022.\n\n>3. *..., but no ablation study quantifies how ShaRP’s gains scale with ensemble size b.*\n\nSection E.2 of the supplementary material presents an ablation study quantifying performance gains with varying ensemble size (b). Computationally, ShaRP remains efficient because it uses only one restoration model from the ensemble at each inference step, thus having comparable computational complexity to single-model methods. The revised manuscript will ensure that this study is clearly referenced in the main manuscript for visibility.\n\n>4. *SISR uses only two Gaussian blurs, which is not enough.*\n\nThanks for your suggestion. Prompted by your comment, we ran an additional SISR experiment using a new Gaussian kernel (σ = 1.0). As shown in the table below, we can achieve consistently good performance as the other two kernels in the manuscript. Note how ShaRP still provides excellent performance, suggesting that it would work with more Gaussian blurs.\n\n| Method    | PSNR  | SSIM  | LPIPS | FID   |\n|-----------|-------|-------|--------|--------|\n| DPIR      | 28.45 | 0.854 | 0.247  | 82.90  |\n| DDRM      | 27.26 | 0.803 | 0.209  | 44.77  |\n| DiffPIR   | 28.37 | 0.841 | 0.215  | 40.59  |\n| DRP       | 28.43 | 0.853 | 0.236  | 75.29  |\n| ShaRP     | 28.70 | 0.858 | 0.226  | 69.75  |\n\n>5. *Perceptual quality requires human evaluation.*\n\nPrompted by your comment, we introduced FID and LPIPS as perceptual metrics. Please refer to the rebuttal comment (7) to Reviewer DuT7 for the table.  The revised manuscript will mention that true perceptual quality requires human evaluation.\n\n>6. *The two papers shown below handle similar tasks: [1] Osmosis, ECCV, 2024. [2] DreamClean, ICLR, 2024.*\n\nWe will cite these papers in the revised manuscript.\n\n>7 *Technical terms (e.g., \"restoration operator\") are inconsistently defined.*\n\nWe will review the use of “restoration operator” across to make sure it is consistent. Restoration operator in our paper refers to an operator that computes MMSE solution of eq. (3).\n\n>8. *Supplement should provide experiments with other degradation types.*\n\nWe will include results reported in our response to your Comment 1 to the supplementary material.\n\n>9. *How does ShaRP’s performance/complexity scale with ensemble size?*\n\nPlease refer to our response to your Comment 2.\n\n>10. *Could ShaRP adapt to settings without even partially sampled data?*\n\nWe are not entirely sure what the reviewer means by “without even partially sampled data.” If the reviewer is referring to the setting where there are no measurements at all, we can still run our method, but it was not conceived for this setting. Indeed, it would be very interesting to extend our methodology to “unconditional generation” or “sampling”, i.e., generation of images from the prior specified using a set of restoration operators. This will be a great direction for future work.\n\n>11 *How did inexact restoration operators impact convergence in practice? Does the bias term remain stable?*\n\nSee Figure 7 in Section C.3, which shows stable convergence even with inexact (self-supervised) MMSE restoration operators. Overall, we did not observe any stability issues with ShaRP.", "claims_and_evidence": "The authors claim that restoration priors outperform Gaussian denoisers is supported by experiments showing higher PSNR/SSIM on MRI and SISR tasks. Theoretical justification links ShaRP to a sound regularizer, and convergence analysis under idealized assumptions provides a partial foundation. Empirical gains over diffusion models are clear.", "methods_and_evaluation_criteria": "ShaRP builds on PnP/RED but modifies via restoration operator ensembles. Evaluation uses standard datasets, i.e., fastMRI and ImageNet. However, for MRI, validation on uniform/random sampling subsets is reasonable, but real-world non-Cartesian sampling is not tested. SISR results focus on synthetic blur kernels, but robustness to natural distortions (e.g., motion blur) is unclear.", "theoretical_claims": "In Appendix, Theorem 1 is correct, and Theorem 2 relies on ideal assumptions: Lipschitz continuity and bounded bias. While practical convergence is shown empirically, the gap between theory and practice needs discussion.", "experimental_designs_or_analyses": "1. MRI experiments are thorough, but no ablation study quantifies how ShaRP’s gains scale with ensemble size b.\n2. SISR uses only two Gaussian blurs, which is not enough.\n3. Perceptual quality requires human evaluation.", "relation_to_broader_scientific_literature": "ShaRP aligns with recent trends in learning-based priors, e.g., denoisers and diffusion, but extends them via restoration ensembles. Comparisons to PnP/RED and diffusion methods (DPS, DDS) are suitable.", "essential_references_not_discussed": "The two papers shown below handle similar tasks: \n[1] Osmosis: Rgbd diffusion prior for underwater image restoration. ECCV, 2024.\n[2] DreamClean: Restoring Clean Image Using Deep Diffusion Prior. ICLR, 2024.", "other_strengths_and_weaknesses": "Strengths:\n1. Theoretically grounded regularization using MMSE restoration ensembles.\n2. Practical advantages in self-supervised MRI where prior denoiser training requires full data.\n3. Clear empirical gains over denoiser-/diffusion-based baselines.\n\nWeaknesses:\n1. Limited diversity in tested degradations.\n2. Computational costs of stochastic restoration steps are not analyzed.", "other_comments_or_suggestions": "1. Technical terms (e.g., \"restoration operator\") are inconsistently defined\n2. Supplement should provide experiments with other degradation types..", "questions_for_authors": "1. How does ShaRP’s performance/complexity scale with ensemble size?\n2. Could ShaRP adapt to settings without even partially sampled data?\n3. How did inexact restoration operators impact convergence in practice? Does the bias term remain stable?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers provided a detailed assessment of the paper and found it to provide good contributions to the domain of inverse problems in imaging in terms of originality and theoretical analysis. There are some minor concerns on the experiments and several recent related methods have not been cited. The authors provided rebuttals and engaged in discussions with the reviewers. The reviewers unanimously concluded that the paper can be accepted at the conference and the AC agrees.
Decision: Accept (poster)