Plug-and-Play Posterior Sampling under Mismatched Measurement and Prior Models
Authors: Marien Renaud
Jiaming Liu
Valentin De Bortoli
Andres Almansa
Ulugbek Kamilov
Venue: ICLR 2024 poster (2024)
Keywords: Inverse problems
plug-and-play priors
posterior sampling
unadjusted Langevin algorithm
Primary area: optimization
TL;DR: We present a theoretical framework for analyzing the sensitivity of plug-and-play unadjusted Langevin algorithm to mismatched models

Abstract: Posterior sampling has been shown to be a powerful Bayesian approach for solving imaging inverse problems. The recent plug-and-play unadjusted Langevin algorithm (PnP-ULA) has emerged as a promising method for Monte Carlo sampling and minimum mean squared error (MMSE) estimation by combining physical measurement models with deep-learning priors specified using image denoisers. However, the intricate relationship between the sampling distribution of PnP-ULA and the mismatched data-fidelity and denoiser has not been theoretically analyzed. We address this gap by proposing a posterior-$L_2$ pseudometric and using it to quantify an explicit error bound for PnP-ULA under mismatched posterior distribution. We numerically validate our theory on several inverse problems such as sampling from Gaussian mixture models and image deblurring. Our results suggest that the sensitivity of the sampling distribution of PnP-ULA to a mismatch in the measurement model and the denoiser can be precisely characterized.

Published as a conference paper at ICLR 2024
PLUG-AND-PLAY POSTERIOR SAMPLING UNDER MIS- MATCHED MEASUREMENT AND PRIOR MODELS
Marien Renaud"
Jiaming Liu
Valentin de Bortoli ENS, CNRS, PSI. University Paris, 75005, FRANCE
Washington University in St. Louis St. Louis, MO 63130, USA
Washington University in St. Louis St. Louis, MO 63130, USA
Andrés Almansa
Ulugbek Kamilov
CNRS, Université Paris Cité Paris, 75006, FRANCE
Washington University in St. Louis St. Louis, MO 63130, USA
ABSTRACT
Posterior sampling has been shown to be powerful Bayesian approach for solv- ing imaging inverse problems. The recent plug-and-play unadjusted Langevin algorithm (PnP-ULA) has emerged as a promising method for Monte Carlo sam- pling and minimum mean squared error (MMSE) estimation by combining phys- ical measurement models with deep-learning priors specified using image denois- ers. However, the intricate relationship between the sampling distribution of PnP- ULA and the mismatched data-fidelity and denoiser has not been theoretically an- alyzed. We address this gap by proposing a posterior-L2 pseudometric and using it to quantify an explicit error bound for PnP-ULA under mismatched posterior dis- tribution. We numerically validate our theory on several inverse problems such as sampling from Gaussian mixture models and image deblurring, Our results sug- gest that the sensitivity of the sampling distribution of PnP-ULA to mismatch the measurement model and the denoiser can be precisely characterized.
INTRODUCTION
Many imaging problems can be formulated as inverse problems seeking to recover high-quality images from their low-quality observations. Such problems arise across the fields of biomedical imaging (McCann et al., 2017a), computer vision (Pizlo, 2001), and computational imaging (Ongie et al., 2020). Since imaging inverse problems are generally ill-posed, it is common to apply prior models on the desired images. There has been significant progress in developing Deep Learning (DL) based image priors, where a deep model is trained to directly map degraded observations to images (McCann et al., 2017b; Jin et al., 2017; Li et al., 2020).
Model-based DL (MBDL) is an alternative to traditional DL that explicitly uses knowledge of the forward model by integrating DL denoisers as implicit priors into model-based optimization algorithms (Venkatakrishnan
al., 2013; Romano et al., 2017). It has been generally observed that learned denoisers are essential for achieving the state-of-the--an results in many imaging con- texts (Metzler al., 2018; Ulondu-Mendes et al., 2023; Ryu al., 2019; Hurault et al., 2022; Wu et al., 2020). However, most prior work in the area has focused on methods that can only produce point estimates without any quantification of the reconstruction uncertainty (Belhasin et al., 2023), which can be essential in critical applications such as healthcare or security (Liu et al., 2023).
In recent years, the exploration of strategies for sampling from the posterior probability has emerged as focal point in the field of inverse problem in imaging (Pereyra et al., 2015; Bouman & Buzzard, 2023: Chung et al., 2023; Song et al., 2022). This pursuit has given rise plethora of techniques, encompassing well-established methods such as Gibbs sampling (Coeurdoux et al., 2023), the Un- adjusted Langevin Algorithm (ULA) (Roberts & Tweedie, 1996; Durmus et al., 2018), and more contemporary innovations like conditional diffusion models (Chung et al., 2023; Kazerouni et al., 2022; Kawar et al., 2022; Song et al., 2023), and Schrödinger bridges (Shi et al., 2022).
"Corresponding author; anee. renaud@mathh u-bordeaux..f fr

---

Published as a conference paper at ICLR 2024
Among these sampling methods, ULA, characterized by its Markov chain- based framework, has gained prominence owing to its ease of implementation and recent versions (Cai et al., 2023; Klatzer et al., 2023; Ehrhardt al., 2023). Plug-and-Plae and-Play Play-ULA ULA (PnP-ULA) (Laumont et al., 2022) is spe- cific variant that incorporates the prior knowledge into the dynamics of the Markov chain through denoiser. While this technique stands out for its simplicity and its ability to approximate the pos- terior law effectively, it is not immune to challenges, including computational time and distribution shifts between the mathematical formulations and practical experiments. These distribution shifts arise due to several factors. First, there is a distribution shift in the prior distribution (Jalal et al., 2021: Chung & Ye, 2022), attributed to the approximation of the minimum mean squared error (MMSE) denoiser. Second, distribution shift emerges in the data-fidelity term due to the inherent uncertainty in the forward model (Wirgin, 2004; Guerquin-Kern al., 2012; Blanke et al., 2020). The impact of these shifts on the efficacy of the sampling method presents an intriguing gap in the current theoretical understanding.
Contributions. (a) Bayesian posterior sampling relies two operators: data-fidelity term and denoising term. This paper stresses that the case of mismatched operators, errors do not accu- mulate indefinitely. Moreover, with mismatched operators, the shift in the sampling distribution can be quantified by a unified formulation, as presented in Theorem 1. (b) Furthermore, we provide a generalized convergence result for PnP-ULA (Laumont et al., 2022) in Corollary 1.2. A new proof strategy based on Girsanov theory allows us to relax assumptions on the denoiser accuracy. These insights are substantiated by a series of experiments conducted on both 2D Gaussian Mixture prior and image deblurring scenarios.
2 BACKGROUND
Inverse Problem. Many problems can be formulated as an inverse problem involving the estimation of unknown vector E Rd from its degraded observation = Ax + n. where A E Rmxxd is measurement operator and n ~ NN00 o is usually the Gaussian noise.
Posterior Sampling. When the estimation task is ill-posed, it becomes essential to include ad- ditional assumptions on the unknown æ in the estimation process. In the Bayesian framework, one utilize p(x) as the prior to regularize such estimation problems, and samples from the posterior distribution p(zly)- The relationship is then established formally using Bayes's 's rule p(zly) p(y ())()), where p(ylan) (z)) denoted as the likelihood function.
In this paper, we focus on the task of sampling the posterior distribution based on Langevin stochas- tic differential equation (SDE) (æt) (Roberts Tweedie, 1996; Durmus al., 2018) as
da:; V logp(a. y + V2dz, = V logp(u/<::) V p(x,) v2dz,,
(1)
where (ze)LER+ is d-dimensional Wiener process. A posterior sampling approach produces mul- tiple solutions for the same degradation (more details in Appendix C.I). When p(x(y) is proper and smooth, with V pog (y) Lipschitz continuous, it has been proven that the stochastic process defined in equation has a unique strong solution which admits the posterior p(aly) as unique sta- tionary distribution (Roberts & Tweedie, 1996). In practice, an Euler-Maruyama discretization of Equation define the Unadjusted Langevin algorithm (ULA) Markov chain for all l E N as
F6+1 2% 8V logp(y)=*) + 8V logp(z>)) + 2882++1.
where 2ą ~ 1) and 8 > is a step size controlling trade-off between asymptotic accuracy and convergence speed (Dalalyan, 2017). The likelihood score V log P(y) can be computed using the measurement model'. However, the prior score log)(z) cannot computed explicitly and needs to be approximate.
Score approximation using PnP Priors. Score approximation a key problem in machine learn- ing which can be solved in various ways such Moreau- Moreau-Yosida envelope (Durmus et al., 2018), normalizing flows (Cai et al., 2023) or score- matching (Nichol & Dhariwal, 2021). 1). In this work, we approximate to the prior score V loggp(w> (x) through MMSE denoiser Dr denoted
(*
xPp] (x]
In this paper, we will assume that the mcasurement model known, even though it be inexact.
2

---

Published as a conference paper at ICLR 2024
where x = x++ with p(z),, is generally intractable in practice, one can approximate it by deep neural network (DNN) denoiser D, * Dr This DNN denoiser is trained by minimizing the mean squared error (MSE) loss (Xu et al., 2020). It gives a link between the intractable prior distribution and the MMSE denoiser which be approximated through Tweedie's formula (Efron, 2011)
c2 -=(, (x) Since the exact MMSE denoiser D
V logp(æ)) V (z) (a) x) ~ (z)
-))
(4)
The right- hendd-handaa side of equation is then plugged in the ULA Markov chain to obtain the PnP-ULA
0k+1 2b + be (xx) + v2
(5)
where the deterministic term of the process b. corresponds to the drift function defined as
1 be (x) V Vlogplyla) + (D.(z) x) (Is(x) -x),
with HI is the orthogonal projection the convex-compact S. This term is added for theoretical purposes (ensures that the Markov chain is bounded) but it is rarely activated in practice. The drift (ar) of this Markov chain (5) also corresponds the approximation of the posterior score V Vlogp(< yy)
Despite all the approximations made by PnP-ULA, it has been shown by (Laumont et al., 2022) that, under certain relevant assumptions, this Markov chain (5) possesses a unique invariant measure Re, and converges exponentially fast it. This means that T.. is the sampling distribution limit of the Markov chain (5). In practice, significant number of step (hundred of thousands) is computed to ensure the convergence. So, we will refer to Ne,6 as the sampling distribution. This has an impact on the computational time required by PnP-ULA which exceeds that of alternative methods, such as diffusion model or flow matching model (Delbracio & Milanfar, 2023). Note that the difference between (x) and p(æ yy has been quantified in total variation (TV) distance (Laumont et al., 2022, Proposition 6).
PNP-ULA SENSITIVITY ANALYSIS
In this section we study the impact of drift shift on the invariant distribution of the PnP-ULA Markov chain. Such shift can be observed in both denoiser shift or forward model shift. Most of the introduced assumptions are reminiscent of previous work on PnP-ULA analysis (Laumont et al., 2022).
With two different drifts by and b², we define the two corresponding Markov chains for i {1,2)
क æt + (b v282
(6)
Hence, the variables subject to modification in the expression of bj are only the forward model A and the denoiser D
1 (D)()) x)) + (Is( x),
(x 20 V
1
With || the Euclidean norm on Rd The forward model A' and the denoiser D can be viewed as parameters of the PnP-ULA Markov chain. Our goal is to study the sensitivity of the sampling dis- tribution "². these parameters. For simplicity in this paper, will designate by the reference drift and b2 as the mismatched drift. The proposed analysis of a drift shift is based on the following assumptions.
Assumption 1. The prior distributions (xx) and p² (x), denoted as target and mismatched priors, have a finite second moment, Vi {1,2), Sg. ||==||pp' (x dx < toc.
This assumption
reasonable assumption since many images have bounded pixel values, for example [0, 255) or [0, 1).
Assumption 2. The forward model has a bounded density, Vy E R" prm,supment meRd p(y x)) < t00. Moreover, the forward model smooth with Lipschitz gradient, p(vl.) E C' (Rd, 10, +ool) and there exists I. > 0 such that, Vy Rdd V log (p(y).)) is I.-lipschitz.
3

---

Published as a conference paper at ICLR 2024
Assumption 2 is true if the forward problem is linear, which will be the in applications.
Assumption 3. There exists > 0, M 2 0 such that for any E]0, cul I1, 02 E R
(x) D. (x2)) & M x1 Ø2
Assumption holds if the activation functions of the DNN denoiser are Lipschitz (e.g., ReLU). The constant M is independent of e, when the denoisers are blind denoisers.
Assumption 4. There exists m E IR such that for any æ Ø2 IR4, Vi {1,2}
log x2) - V log p" (3/ 22 x1) S - ml -m| - x 11?
Note that if Assumption is satisfied with the forward model A is not invertible, such as deblurring, then m < 0. Assumption 2 holds, then Assumption holds with m L. However, it is possible that m -L which leads to better convergence rates for PnP-ULA. To ensure the stability of PnP-ULA in the case of m < 0, the projection on S has been added.
then the likelihood x log pl x)) m-concave.
We introduce metrics which will be used to quantify the difference between the sampling distri- butions 7/,8 and x².5- The first one is the TV distance which quantifies the point-wise distance between the densities of the probability distributions. The TV distance is the distance induced by
the TV norm defined for a probability density by
J f(z))d Rd
| = sup IIII $1
The second one is the Wasserstein distance which is the cost of optimal transport from one distribu- tion to the other (Villani et al., 2009). Formally, The Wasserstein-1 distance between two distribution Ti, IT2 be defined :
/
W1 = inf p))(mLmm))
| - 222ldu((11, x22,
(8)
Where [(m1, x2) denoted all transport plans having Tı and 22 as marginals.
4 MAIN RESULT
To present our theoretical analysis, we first define a new pseudometric between two functions taking values in Rd
Definition 4.1 (Posterior-Lz pseudometric). The posterior-Ly pseudometric between fi and f2 Rd Rd defined by:
(2), Ex
The properties of the posterior- La pseudometric studied in Supplement A. For simplicity, shall call it posterior- L2. This metric can be computed in practice because T. is sampling distribu- tion of PnP-ULA run with the drift b..
We can now state our main result on PnP-ULA sensitivity to drift shift.
Theorem 1. Let Assumptions 1-4 hold true. There exists Ao, Bo, AI, Bi RRt, such that for
5 5|
|| - 2.olio I (b,, b.) Bod Wi m5 < Ai (dr ()) $ + 0
With 8 = (L M+1 4) The proof is provided in Supplement D.
Theorem establishes that a drift shift implics a bounded shift in the sampling distribution. Both the total variation (TV) and Wasserstein distance bounds consist of two different terms. The first
4

---

Published as a conference paper at ICLR 2024
term depends on the posterior-Ly distance d1 (b) b2), which represents the drift shift error. The second term is a discretization error term that depends on the step-size The Wasserstein distance bound seems suboptimal, since it has been derived directly from the TV bound. A similar bound in Wasserstein distance, analogous to the TV-distance could potentially be demonstrated. However, leave this for future work. Constants Ao, Bo, A1, B1 have polynomial dimension dependence (Appendix D.1.6).
The pseudometric di also appears in the analysis of diffusion models. For example in (Chen al., 2023; Benton et al.. 2023; Conforti et al., 2023) the convergence bounds are expressed in terms of Jo V log (xx)) (t, x 2)]l ( (xx) where q is the density of the forward process and sy the approximated score. This quantity also corresponds to the difference of the drifts between the ideal backward diffusion process and the approximated backward diffusion used in practice. Similar tools, namely Girsanov theory, are used in both our analysis and the error bounds of diffusion models.
Our analysis is beckward-compatible with the previous theoretical results (Laumont et al., 2022, Proposition 6) on PnP-ULA. In addition, our result provides reformulation of (Laumont et al., 2022, Proposition 6) if denoisers D are close to the exact MMSE denoiser D; in the infinite norm. However, it is worth to note that our result is more general because assumptions are made on the quality of denoiser D' Another key difference lies in the posterior-Ly L2 pseudometric, which is can in our experiments (see Section 5).
relevant to characterize the Wasserstein distance,
4.1 IMPLICATIONS AND CONSEQUENCES
In this section, will present three consequences of Theorem in different application cases. All these results are presented in TV -ddisttacc for simplicity, but can also be easily derived in Wasser- stein distance. A demonstration of these results can be found in Supplement E.
4.1.1 DENOISER SHIFT
Theorem has an interesting consequence in case of a denoiser shift. The posterior-Ly L2 appears to be a relevant metric to compare different denoisers efficiencies to provide high-quality sampling.
Corollary 1.1. Let Assumptions 1-4 hold true. There exists A2, B2 E Rt, such that V8 EjO, 5) and for all denoisers D and D
| TÊ, S ADd D2) B2
Similar Theorem 1, there are two terms: the first one quantifying the denoiser shift, and the second one is the discretization eтrоr. The proof of this corollary can be found in Supplement E. provides a quantification of the sensitivity of the invariant law of PnP-ULA the denoiser, which is a key component of the process. This can be viewed as the process's 's sensitivity to regularization, or in a Bayesian paradigm, to prior knowledge.
4.1.2 REFORMULATION OF PNP-ULA CONVERGENCE GUARANTIES
Another consequence is the reformulation of PnP-ULA's previous convergence result (Laumont al., 2022, Proposition 6). which previously required the denoiser be close the exact MMSE denoiser D. (see equation 3). We have demonstrated a similar convergence result without this as- sumption.
More precisely, by naming (-ly) the posterior distribution with P. (see equation 2) as a prior distribution, the following result holds.
Corollary 1.2. Let Assumptions 1-5 hold true. Let such that 2X(L+ M+1 min (m, 0)) and e S Eg- There exists C3 > such that for Re > 0 such that B(0, Rc) there exist A3, B3, E R such that vš o]
Ipp. (-1v) - slrv S Agdi D?? + B3d4 + C.R-.
demonstration can be found in Supplement E.2. This result requires another technical assumption 5 which is nothing more than some regularity the exact MMSE denoiser D
5

---

Published as a conference paper at ICLR 2024
300
3.5
14
Puderaan disaAio MIL FHOSE
Posteriond: distance (di)
Wasserstein distance (W:)
Wasserster aslance (W) mon
Pusterin-ly didance (1) meet, D-
Posteriar- distance (ds)
Uwssenteel dicacc 1W))
Wasserstein distance
5 10
Denoiser parameter (c)
Number of parameters of the denolser
Figure 1: Illustration of Theorem I's bound by visualizing the strong correlation between the Wasserstein distance between sampling distributions and posterior-Ly L2 distance between denois- Left plot: Distances, for the GMM experiment in 2D, computed between sampling generated by mismatch denoisers and the exact MMSE denoiser. Note how the posterior-Ly L2 is more correlated to the Wasserstein distance than the prior-L2. Right plot: Distances, for the gray-scale images experi- ment, compute between DrCNN denoisers with 5 x 10 weights and other DrCNN denoisers with fewer weights. Note how the posterior- L2 and the Wasserstein distance highly correlated with correlation r 0.9909 in average and r > 0.97 for each image.
4.1.3 FORWARD MODEL SHIFT
Another consequence of Theorem I is in case of forward model mismatch (Wirgin, 2004; Blanke et al., 2020), a high-stakes subject especially in medical imaging (Guerquin-Kern et al., 2012). It implies that if the shift of the forward model is limited, then the shift on the sampling distribution is limited.
Corollary 1.3. Let Assumptions 1-4 hold true. There exist A3, B3 RR, such as V5 EJO,
|| 2.,ll & AsIAA' - A + B5
A demonstration can be found in E.3. The spectral norm is used here, but this result holds true for any norm the matrix space because all norms are equivalent in Enite-dimensional space. Thus the stability of PnP-ULA with respect with the measurement model has been proved.
5 NUMERICAL EXPERIMENTS
Our main result in Theorem provides error bounds for the distance between two sampling dis- tributions Th,6 and n²,6- a function of the corresponding posterior-Lg. We provide numerical validations of this bound by exploring the behavior of the Wasserstein distance between sampling distributions and the corresponding posterior L2 pseudometric defined Eq 4.1. We use the correla- tion between these two distances validate our theoretical results and show that the posterior-L2 pseudometric can characterize the Wasserstein distance.
The three experiments illustrate the usefulness of the three different corollaries in the previous sec- tion. Our first experiment (section 5.1) illustrates sampling distribution error (Corollary 1.2) for GMM 2D. The second experiment (section 5.2) illustrates denoiser shift (Corollary 1.1) for an gray-scale image deblurring. The third example in section 5.3 illustrates forward model shift (Corollary 1.3) on color image deblurring.
More results of PnP-ULA on images can be found in Supplement C. More technical details about experiments can found in Supplement B. The code used in these experiments can be found in PnP ULA posterior law sensivity code.
5.1 DENOISER SHIFT ON GAUSSIAN MIXTURE MODEL IN 2D
In general, computing the exact MMSE denoiser D" is challenging task. However, we assume that the prior distribution p(x) follows Gaussian Mixture Model (GMM), then closed-form pression for D* becomes available. This allows us to evaluate Corollary 1.2 in simplified scenario involving 2D GMM. We emphasize the ability of the posterior-L.g pseudometric between denoisers

---

Published as a conference paper at ICLR 2024
MMSE denoiser
Denoiser, c
Denoiser, c-
Denoiser, c
8
8
8
Illustration of denoisers with 0.05 used for the 2D Gaussian Mixture experiment. The prior distribution, a Gaussian Mixture, is represented in light blue. Denoisers, D. R² + R², represented by there outputs (in dark blue) on set of imputs (in orange) linked together (by orange lines). Rightmost: Exact MMSE denoiser. Leftmost: Three mismatch denoisers with various e parameter.
Figure 2:
explain variations the Wasserstein distance between sampling distributions. Notably, denoisers are trained based on the prior distribution, which might lead one to expect that the prior-L2 norm would provide more accurate measure. However, we will illustrate that this is not the case.
The algorithm is run for a denoising problem (A = 1z. o² = 1) and the observation = (0, With PnP-ULA parameters: c 0.05, 0.05 and N 100000. The initialization the Markov chain is taken at æ 0. Samples, s* (si NJ. generated by the exact MMSE denoiser D
provide a reference sampling distribution π* Σ deg.
We introduce novel class of denoising operators referred to as mismatched denoisers, denoted as DE These operators are defined to be equal to the exact MMSE denoiser when the horizental coordinate 21 exceeds threshold and 0 otherwise.
(1 x2) = { (21,32)) if >
(10)
otherwise
It becomes evident that lime-+-oo Dr D and conversely lim.++ DE = 0. In Figure 2. we visualize denoisers used the experiment. A total of 50 distinct mismatched denoisers are system- atically generated, spanning the parameter range E [-5,5]. Concomitant with each of these de- noisers Df, corresponding sample s ke(,)) and sampling distribution nT Σ deg is generated.
In Figure 1, present the Wasserstein distance, denoted Wi(m"x=*) between the refer- ence sampling distribution n' and the mismatched sampling distribution T.. Concurrently, we compute the prior-L2 pseudometric between denoisers, N E (xx) (2xx,) with (I=)ke(L,N] sample of the prior distribution. Furthermore, we display the posterior-Lz pseudo-
metric, di (Dr, D V EN [Dt 12 The relation between these metrics is explained in Corollary 1.2. Upon closer examination of Figure 1, becomes evident that the posterior- Le pseudometric exhibits stronger correlation with the Wasserstein distance, r = 0.98, when compared to the prior-I-2 pseudometric, 0.89. This experiment shows the relevance of the posterior- L2 pseudometric (at the basis of Theorem 1) to reflect the accuracy of a posterior sampling method.
5.2 DENOISER SHIFT ON GRAY-SCALE IMAGES
In practical applications, the learned denoising model does not equate to the exact MMSE denoiser. Furthermore, the testing distribution rarely aligns perfectly with the training distribution, giving rise to distributional mismatch that can be understood In this context, it becomes imperative to understand the sensitivity of the invariant distribution of PnP-ULA concerning the denoiser. This sensitivity is elucidated Equation 1.1.
form of incorrect training for the denoiser.
In order to validate empirically this result, the deblurring task is addressed using a uniform blur kernel with dimensions of x 9 and noise level of 255 This degradation is applied natural

---

Published as a conference paper at ICLR 2024
Ground Truth
Observation
10a welghts
10 weights
x 10 weights
PSNRYSSI
21. 500055
240.0 74
21.005
23 AULD
PENRYSSIM
22. 22.64/052
28 22.2200.0
21.2210.50
31 46.46.55 50
Figure 3: Illustration of MMSE estimators computed by PnP-ULA run 10y steps with various DrCNN denoisers. The quantities in the top-left corner of each image provide PSNR and SSIM values for each reconstructed images. Denoisers have a different number of weights, but are all trained in the same way. Note that shift between the reference denoiser (5 x 10 weights) and mismatched denoisers using less weights (103 or 10 weights) implies shift in the MMSE estimator quality.
images (from CBSD68 dataset (Martin et al., 2001)) of size 256 x 256 in grayscale. A total of 17 distinct DrCNN denoising models (Ryu et al., 2019) were trained, each varying in the number of layers, ranging from a single layer to a maximum of 17 layers. PnP-ULA was runned for a duration of 10s steps using each of these denoising models across 15 different images (see Supplement B). Parameters are chosen following the recommendation of (Laumont et al., 2022). The initialization of PnP- InA saved as samples, with one image being saved every 10 steps in the sampling process.
performed using the observation vector y. and total of N 1000 images were
We denote by st (sk)) the samples generated using the denoiser D with i layers and r' N Ex= 54 the corresponding sampling distribution. nt our reference sampling dis- tribution. Figure illustrates the Wasserstein distance between sampling distributions denoted as WI(T T²), in comparison to the posterior-Lz between denoisers, defined as d1 dD(D! D:)
V V D (s - (1( Results on images are depicted in Figure 3 to provide em- pirical evidence that a more powerful denoiser results a more precise reconstruction.
The figures display both the mean and the range of results across the 15 images. For each image (note that constants of Theorem are problem-specific)) we compute the correlation, and the average correlation is computed be 0.9909, with the correlation between the two distances for each image r >> 0.97.
It important note that the Wasserstein distance does tend zero because of bias in this scenario. A sample of 1000 images in space of dimension 256 x 256 = 65, 536 is insufficient.
Therefore, two samples from the same distribution should not be expected to have Wasserstein distance of zero.
Evidently, the two distances exhibit a significant correlation, showing link between these two distances. Consequently, the objective of this experiment is achieved. Furthermore. it is apparent that the samples generated by the denoiser with only 10 weights closely resemble those produced by the denoiser 5 x 10 weights. In this context, it appears relevant to train a denoiser with only 5 layers (10$ weights), as easier to train and less computationally intensive to deploy.
5.3 FORWARD MODEL SHIFT ON COLOR IMAGES
Theorem has another implication when dealing with uncertainty the forward model. This arises, for instance, in medical imaging when the exact parameterization of the measuring instrument is not
8

---

Published as a conference paper at ICLR 2024
Gonund
Wasserscch Distance
Figure 4: Illustration of the PnP-ULA stability to a mismatch forward model. Lefimost six plots: MMSE estimators computed with PnP-ULA run on 30, 000 steps on Gaussian blur of standard deviation ø1. Rightmost: Evolution of the Wasserstein distance between sampling distributions computed with mismatched blur kernel, or [0 and sampling with the exact forward model
o' = 3. Note that the image reconstruction quality improves as gets closer to a² = 3, used to degrade the image. In addition, the case of Gaussian blur, the pseudometric given by (a a²1 which justifies qualitatively the observed linear decrease of the Wasserstein distance.
well-defined (Guerquin-Kenl Kern et al., 2012). Corollary 1.3 elaborates on how the sampling distribution sensitive to shifts the forward model.
In our experimental setup, we address the deblurring inverse problem using the CelebA validation set, which consists exclusively images of women's faces resized to RGB dimensions 256 x 256 pixels. A Gaussian blur kernel is applied to the images, with standard deviation σı being subject to modification. The denoiser is implemented using the DRUNet architecture (Zhang et al., 2021) and has been trained on the CelebA dataset (Liu al., 2015). Other parameters of PnP-ULA chosen following the recommendation of (Laumont et al., 2022).
The image is initially degraded using Gaussian blur kernel with standard deviation of o² 3. This reference sampling distribution is denoted n² Multiple Markov chains are then computed, each spanning 30,
000 steps, under the assumption of mismatched forward models with varying standard deviations, 01 E [0, 3.. At every 30 steps of these chains, we select N = 1, 000 samples, resulting in distinct sampling distributions Toy We subsequently compute the Wasserstein distance, denoted as ( ). ) to quantify the discrepancy between the reference sampling distribution ' and the mismatched sampling distribution Taz
This procedure is applied to two images presented in Figure 4. The distance between the blur matri- ces with Gaussian kernel is directly the difference between the standard deviation of these kernels. The observed trend aligns precisely with the expected behavior: as the assumed standard deviation of the blur kernel, denoted as σ1, approaches 3, the exact kernel of blur, the Wasserstein distance Wi (m= To, ) decrease and the quality of the reconstruction improves. It's notable that when 01 O, we essentially have a denoising scenario with minimal noise, resulting reconstructed image that closely resembles the observed image. This experiment clearly illustrates the corollary 1.3 in context of a forward model mismatch.
CONCLUSION
In conclusion, our comprehensive analysis of the PnP-ULA algorithm has provided us with insights into the intricacies of this posterior sampling technique. Through rigorous examination, we'v suc- cessfully unified the understanding of denoiser shifts and variations in the forward model, encap- sulating these phenomena error bounds are expressed in terms of posterior-Lz pseudometric, which we show to be more relevant than the previously used bound between denoisers (Laumont et al., 2022). Future work will investigate how to extend our results to other Langevin based dynamics (Klatzer al., 2023), with a particular focus on annealing and exploring the broader applications of our results in addressing various inverse problems.
singular and novel result, denoted as Theorem
Importantly, our

---

Published as a conference paper at ICLR 2024
7 REPRODUCIBILITY STATEMENT
Source code used in our experiments can be found in PnP ULA posterior law sensivity code. It contains a README.md file that explains step by step how to rum the algorithm and replicate the results of the paper. In Section 5 it is precisely detailed how all the hyper-parameters are chosen and, for each experiment, which dataset is used. As for the theoretical results presented Section 4, complete proofs are given in the Supplements.
ACKNOWLEDGEMENTS
This paper is partially based upon work supported by the NSF CAREER award under grants CCF-2043134. This study has been carried out with financial support from the French Research Agency through the PostProdLEEA grant MANR-19-CE23-0022 Shoushtari for providing pre-trained DRUNet denoisers.
We also want to thank Shirin
REFERENCES
Alper Aksac, Douglas| Demetrick, Tansel Ozyer, and Reda Alhajj. Brecahad: dataset for breast cancer histopathological annotation and diagnosis. BMC Research Notes, 2019.
Omer Belhasin, Yaniv Romano, Daniel Freedman, Ehud Rivlin, and Michael Elad. Principal un- certainty quantification with spatial correlation for image restoration problems. arXiv preprint 1rXi::2305.10124, 2023.
Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Linear convergence bounds for diffusion models via stochastic localization, 2023.
Stephanie E Blanke, Bernadette N Hahn, and Anne Wald. Inverse problems with inexact for- ward operator: iterative regularization and application in dynamic imaging. Inverse Problems, 36(12):124001, oct 2020. doi: 10.
URL https://dds doi. org/
10.1088 1361-1111111 4220 12442/abba abb5el.
Valentin De Bortoli and Alain Durmus. Convergence of diffusions and their discretizations: from continuous to discrete processes and back, 2020.
Charles A. Bouman and Gregery Buzzard. Generative plug and play: Posterior sampling for inverse problems, 2023.
Ziruo Cai, Junqi Tang, Subhadip Mukherjee, Jinglai Li, Carola Bibiane Schönlieb, and Xiaoqun Zhang. Nf-ula: Langevin monte carlo with normalizing flow prior for imaging inverse problems. arXiv preprint arXiv:2304.083422 2023.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference Machine Learning, pp. 4735-4763 PMLR, 2023.
Hyungjin Chung and Jong Chul Ye. Score-based diffusion models for accelerated mri, 2022.
Hyungjin Chung, Jeongsol Kim, Michael T. Mccann, Mare L. Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems, 2023.
Florentin Coeurdoux, Nicolas Dobigeon, and Pierre Chainais. Plug-and-play split gibbs sampler: embedding deep generative priors in bayesian inference, 2023.
Noa Cohen, Hila Manor, Yuval Bahat, and Tomer Michaeli. From posterior sampling to meaningful diversity in image restoration, 2023.
Giovanni Conforti, Alain Durmus, and Marta Gentiloni Silveri. Score diffusion models without early stopping: finite fisher information is all you need, 2023.
10

---

Published as a conference paper at ICLR 2024
Arnak S Dalalyan. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(3):651- 676, 2017.
Mauricio Delbracio and Peyman Milanfar. Inversion by direct iteration: An alternative to denoising diffusion for image restoration, 2023.
Alain Durmus, Eric Moulines, and Marcelo Pereyra. Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau. SIAM Journal on Imaging Sciences, 11 (1):473-506, 2018.
Andreas Eberle. Reflection couplings and contraction rates for diffusions. Probability theory and related fields, 166:851-886, 2016.
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Asso- ciation, 106(496)::602-1114 2011. doi: 10.1 2..119//aaa..211.mmm. 111111.... URL https://do/ org/10 11 98/9119880000 / jasa 2011. tm1 1181. PMID: 22505788.
Matthias J. Ehrhardt, Lorenz Kuger, and Carola-Bibiane Schönlieh. Proximal langevin sampling with inexact proximal mapping, 2023.
M. Guerquin- Kem, I.. Lejeune, K. P. Pruessmann, and M. Unser. Realistic analytical phantoms for parallel magnetic resonance imaging. JEEE Transactions on Medical Imaging. 31(3):626-636, 2012. doi: 2...1009/MM.2211221/444
Samuel Hurault, Arthur Leclaire, and Nicolas Papadakis. Proximal denoiser for convergent plug- and-play optimization with nonconvex regularization. In International Conference on Machine Learning, pp. 9483-9505. PMLR, 2022.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G. Dimakis, and Jonathan I. Tamir. Robust compressed sensing mri with deep generative priors, 2021.
Kyong Hwan Jin, Michael T McCann, Emmanuel Froustey, and Michael Unser. Deep convolutional neural network for inverse problems imaging. IEEE Transactions on Image Processing, 26(9): $509-45222 2017.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila, Training generative adversarial networks with limited data, 2020.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising Diffusion Restoration Models. In ICLR Workshop on Deep Generative Models for Highly Structured Data, volume 2020-Decem, jan 2022.
Amirhossein Kazerouni, Ehsan Khodapanah Aghdam, Moein Heidari, Reza Azad, Mohsen Fayyaz, llker Hacihaliloglu, and Dorit Merhof. Diffusion models for medical image analysis: compre- hensive survey. arXiv preprint arXiv:221 20t::2211.07804, 2022.
Teresa Klatzer, Paul Dobson, Yoann Altmann, Marcelo Pereyra, Jesús María Sanz-Serna, and Kon- stantinos C. Zygalakis. Accelerated bayesian imaging by relaxed proximal-point langevin sam- pling. 2023.
Rémi Laumont, Valentin De Bortoli, Andrés Almansa, Julie Delon, Alain Durmus, and Marcelo Pereyra. Bayesian imaging using plug & play priors: when langevin meets tweedie. SIAM Journal Imaging Sciences, 15(2))701-737, 2022.
Housen Li, Johannes Schwab, Stephan Antholzer, and Markus Haltmeier. Nett: solving inverse problems with deep neural networks. Inverse Problems, 36(6):065005, jun 2020. doi: 10.1088/ 6360-6622//b66557. URL https://///.. http:/xxx..d.oooo///o.o
doi wrg/ 1088/103811110 6420/ ab 6d57.
Jiaming Liu, Rushil Anirudh, Jayaraman J. Thiagarajan, Stewart He, K. Aditya Mohan. Ulugbek S. Kamilov, and Hyojin Kim. Dolce: A model-based probabilistic diffusion framework for limited- angle CT reconstruction. In Proc. IEEE Inf. Conf. Comp. Vis. (ICCV), 2023.
11

---

Published as a conference paper at ICLR 2024
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
D. Martin, C. Fowlkes, D. Tal, and Malik. A database human segmented natural images and its application evaluating segmentation algorithms and measuring ecological statisties. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume 2, doi: 10. 109/ICCV.2001 .376555.
pp. 416-423 vol.2, 2001.
Michael T. McCann, Kyong Hwan Jin, and Michael Unser. Convolutional neural networks for inverse problems in imaging: A review. IEEE Signal Processing Magazine, 34(6):85-95, nov 2017a. doi: 10.1 1011000mm..00.../1/.. 2017.22 2739299.
URL
Michael T McCann, Kyong Hwan Jin, and Michael Unser. Convolutional neural networks for inverse problems in imaging: review. IEEE Signal Processing Magazine, 34(6):85- 2017b.
Christopher A. Metzler, Philip Schniter, Ashok Veeraraghavan, and Richard G. Baraniuk, prDeep: Robust phase retrieval with a flexible deep network. In Proc. 36th Int. Conf. Mach. Learn., pp- 3501-3510, Stockholmsmässan, Stockholm Sweden, Jul. 10-15 2018.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffuusion probabilistic models. In International Conference on Machine Learning, pp. 8162-8171. PMLR, 2021.
Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1(1):39-56, 2020.
Marcelo Pereyra, Philip Schniter, Emilie Chouzenoux, Jean-Christophe Pesquet, Jean-Yves Tourneret, Alfred Hero, and Steve McLaughlin. A Survey of Stochastic Simulation and Opti- mization Methods in Signal Processing. IEEE Journal on Selected Topics in Signal Processing, 10(2):224-244 may 2015. ISSN 19324553. doi: 1..110//SSSP2205224496666
Zygmunt Pizlo. Perception viewed as an inverse problem. Vision Research, 41(24):314-3311,, 2001. ISSN 0042-6989. doi: http:///o.ooggo..././//000222200 // http/./ccieee www.sssceneeeirecttttc
URL https:
Gareth O. Roberts and Richard L. Tweedie. Exponential convergence of langevin distributions and their discrete approximations. Bernoulli, 2(4):341-363, 1996. ISSN 13507265. URL http:
/tlee/3310 //wwww jstor. . org/ www.eeaabeeeeeeeeeeeeae 331 8418
Yaniv Romano, Michael Elad, and Peyman Milanfar. The little engine that could: Regularization by denoising (red). SIAM Journal on Imaging Sciences,
2017.
Ernest K. Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Plug- and-Play Methods Provably Converge with Properly Trained Denoisers. In (ICML.) International Conference on Machine Learning, may 2019.
Yuyang Shi, Valentin De Bortoli, George Deligiannidis, and Amaud Doucet. Conditional simulation using diffusion schrödinger bridges, 2022.
Sh Liptser-AN Shiryayev. Statistics of Random Processes. Springer- Verlag, New York, 1977.
Jiaming Song. Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-Guut Diffusion Models for Inverse Problems. In (ICLR) International Conference on Learning Representa- fions, 2023. URL https::/ httpps://operreview ntt.://wwweee/eeeeeeeeeeee.eeeee forummid=Sa Btt.r//roo.....LLgggMa...CCCCCC gIMMAMRKKhhttps: //youtu. be beriuu.bbDDDDDaa heee//DAAqqqMM,,
Yang Song. Liyue Shen, Xing Lei, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations, 2022. URL https::p//.c /twp://veee//eeeeeee///ee
httt:/oormmmid-vaCCYy. JuGI.
Maciej Sypetkowski, Morteza Rezanejad, Saber Saberian, Oren Kraus, John Urbanik, James Taylor, Ben Mabey, Mason Victors, Jason Yosinski, Alborz Rezazadeh Sereshkeh. Imran Haque, and Berton Earnshaw. Rxrx1: A dataset for evaluating experimental batch correction methods, 2023.
12

---

Published as a conference paper at ICLR 2024
Cristiano Glondu-Mendee Loic Denis, Charles-Alban Deledalle, and Florence Tupin. Robustness to spatially- speckle in Plug-and-Play PoISAR despeckling. working paper or preprint, 2023. URL https::/tellceeeeee wwweeecccoo-iiiiiieeee hal. science wwweeeennehaaaleeeee httpe//hhaac11111110
Singanallur V Venkatakrishnan, Charles A Bouman, and Brendt Wohlberg. Plug-and-play priors for model based reconstruction. In 2013 IEEE global conference on signal and information process- ing. pp. 945-948. IEEE, 2013.
Cédric Villani al. Optimal transport: old and new, volume 338. Springer, 2009.
Armand Wirgin. The inverse crime, 2004.
Zihui Wu, Yu Sun, Alex Matlock, Jiaming Liu, Lei Tian, and Ulugbek Kamilov. Simba: Scalable inversion in optical tomography using deep denoising priors. IEEE Journal of Selected Topics in Signal Processing, 14(6):1163-1177) 2020.
Xiaojian Xu, Yu Sun, Jiaming Liu, Brendt Wohlberg, and Ulugbek S Kamilov. Provable convergence of pluy-and-play priors with mmse denoisers. IEEE Signal Processing Letters, 17:1280-1284, 2020.
Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Lue Van Gool, and Radu Timofte. Plug-and- image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):6360-633 6376, 2021.
13

---

Published as a conference paper at ICLR 2024
SUPPLEMENTARY MATERIAL
Our unified analysis of PnP-ULA based on stochastic equation theory. In Supplement A, we first analyse the properties of the posterior-Lz pseudometric. In Supplement B, we include additional technical details on experiments. In Supplement C. more result of PnP-ULA on various images are presented. In Supplement D, we demonstrated the Theorem 1. In Supplement E, we derive from the main result the different corollaries.
A POSTERIOR-L, PSEUDOMETRIC
We define the distance which has been naturally introduce 4.1, the posterior-L, L2 pseudometrie:
di (b) b2) /Exmats (116 (2))))
It is clear that this function of (b). b2) is positive, symmetric and is verifies the triangular inequality. However it is not distance, because the separability is not to be ensured. If di (6., 62) = 0, then b. 62 on the domain where ,2,5 Thus, d1 will be named posterior- pseudometric.
However, in our case of evaluation it will have the same behavior distance. Let's suppose that Th. equals p(-()) (no approximation errors). But the support p(-1z) is the support of the prior p(x) because of the Gaussian noise. Thus, the two functions are equal their support are included on the support of the distribution of data. This is totally acceptable because the area of interest the space of images, i.e. the support of p(æ).
We the notation di for this metric emphasize that it involves integrating the L2 norm over the reference sampling distribution T,,8. While it is possible to perform integration over the mismatched sampling distribution ~ the expectation only on n1,s to make it easier to compute in practice. In this work, the convention has been made that 6/ is closer to p. (-ly) than b, (for instance, with the infinite norm on the space of images). This situation occurs if D. is trained better than D2 or if the measurement model A more relevant than Ag. With this convention, we have observed in practice that the distance computed with T more correlated the Wasserstein distance between the samples than the distance computed with x²,5- This empirical evidence reinforces our choice.
taking the minimum of these two values. We have chosen to retain
B ADDITIONAL TECHNICAL DETAILS
This section presents several technical details that were omitted from the main paper for space. There are three parts for each of the experiments.
B. TECHNICAL DETAILS OF GMM IN 2D
Following the experimental setup of Laumont et al. (2022), a regularization weight a > 0 is added to the process. So, the Markov chain, which is computed, is defined by 5 with a drift:
1
b. (x) logp(yy<>)
(D. x) + (IIs(x) x).
This parameter allows for the balancing of weights between the prior score and the data-fidelity score. This a parameter useful in practice to obtain better convergence. For GMM in 2D, = 0.3. For experiment on images, a =
The prior is a Gaussian Mixture Model
p(x) = Σ w.N((;; Ex), 1=1
With w, 2 0, E. w; = 1 weights between the Gaussian and N (x:; p4 E,) the Gaussian distribu- tion of mean {,; and covariance matrix Σ, evaluate in
14

---

Published as a conference paper at ICLR 2024
Figure 5: Illustration of the images used in the gray-scale images experiment 5.2
In this case, the posterior distribution has closed form
p(a = Σ Na
With
S E
A o²
m.
Σ
A
E ph + Av) y)
wi exp
1 m
8.m,
T E, Mc
xT
0;
20
p(y)
(2x)a
,,, AX?
Similarly the exact MMSE denoiser D has closed form
(T( Ep (z)(())) DP (.)+(a))
With
n; (E,""
Ia)) -1 (Σ,-) #i + =)
(-2 (p4 xx)TT ( cIa) cla) (M:
=²) -ui=)))
G 22 det (e,+
exp
2
0.5
In our experiment, parameters of the GMM are p = 2, p1 = 11 = 0. Σ
=
0.15
00155
0.5
Σg =
0.5 2 w1 = wy = 0.5.
15

---

Published as a conference paper at ICLR 2024
PV
Figure 6: Illustration of images used in the color images experiment 5.3
B.2 TECHNICAL DETAILS FOR DENOISER SHIFT ON GRAY-SCALE IMAGES
The 15 images used for the gray-scale images experiment 5.2 can be found the Figure 5. These images been took from classical validation crop if necessary to be of size 256 256 and normalized within the range of 0 to 1.
A total of 17 distinet DrCNN denoising models (Ryu al., 2019) trained, each varying in the number of layers, ranging from single layer to a maximum 17 layers. Each of these denoiser is trained on the CBSD68 dataset composed of 400 natural images of size 256 256 for 50 epochs and a learning rate of 1.10-3 The noise level is fixed at 253 and the Lipschitz constant of the network not constraint. It has been test constraint the Lipschitz constant but the network with this constraint perform worst. The computational time required for training one denoiser an NVIDIA GeForce RTX 2080 GPU was approximately 2 hours. So a total of 34 hours of computation was needed to train the gray-scale image denoisers.
5
The computational time required for one PnP-ULA sampling on an NVIDIA GeForce RTX 2080 GPU was approximately 40 seconds per image. This equates to approximately 3 hours of compu- tational time for the entire experimental procedure. The PnP-ULA run on in-distribution image (from CBSD68 dataset) and out-of-distribution images (from BreCaHAD dataset (Aksac et al., 2019) and RxRx1 (Sypetkowski et al., 2023), resized to be 256*256 in gray-scale).
B.3 TECHNICAL DETAILS FOR DENOISER SHIFT ON COLOR IMAGES
pretrained denoiser with the DRUNet architecture (Zhang et al., 2021) on CelebA dataset was used. This denoiser was trained with images from CelebA (only women faces) resized to be 256+256 and with a noise level choose uniformly in the range [0, 255
75
The computational time required for PnP-ULA sampling of 30, 000 steps GeForce RTX 2080 GPU was approximately minutes per image. This procedure is applied to set of 10 images presented in Figure 6. This equates to approximately 15 hours of computational time for the entire experimental procedure. The PnP-ULA is applied on in-distribution images (woman faces from CelebA dataset) and on out-of-distribution dataset (from BreCaHAD dataset, RxRx1 dataset and MetFaces dataset (Karras et al., 2020), resized to be 256#256 color images).
NVIDIA
In section 5.3 only presented result of this experiment two images but we can compute the mean of the Wasserstein distance W (1(To, ?T)) between a sampling with mismatched measure- ment model 01 [0,3[ and the exact measurement model σ 3. This result is presented in Table B.3.
Standard deviation of blur kernel Wasserstein distance
0
2.5
640
642
585
471
341
16

---

Published as a conference paper at ICLR 2024
devittion
128-262559--330
2A8-26552200-00 220--607-5M-777 228--264-22--A7
1588-22 SAGO Med
PO AOPOONA
MARHOR
PONT DWWOO
PARLOR EXPIO
Figure 7: Result of PnP-ULA on different gray scale images for 100, 000 steps with DnCNN. The five top images are in-distribution and the bottom two images are out-of-distribution images.
One can see the same conclusion on this table, the Wasserstein distance decrease when σ - 3, showing experimentally the result of Corollary 1.3.
C PNP-ULA RESULTS ON IMAGES
In this section, we provide more result of the PnP-ULA sampling algorithm on gray-scale and color images.
General setting Parameters choose for running the PnP-ULA have been taken following the anal- ysis of (Laumont et al., 2022) with a L,
255 À
2 - and å
31 - C has
been choose to take a projection on the image space C' [0, 1j because it gives a little better result in practice than the advice of (Laumont et al., 2022).
The PnP-ULA algorithm run for deblurring with a uniform blur 9 9 and noise level of The convergence of the algorithm can be observed in number of step of order 10, 000
100, 000.
Results gcay-scale images The DnCNN denoiser is trained CBSD68 dataset (natural im-
ages of size 256 * 256 on gray-scale normalized between 0 and 1) with 17 layers and 50 epochs of training.
17

---

Published as a conference paper at ICLR 2024
In Figure 7, the result various images can be seen. The image is well reconstruct with the MMSE estimator, especially, looking at the Structural Similarity Index Measure (SSIM), the gap between the observation and the reconstruct image is huge. The algorithm is able reconstruct more fine structure. By looking at the sample themselves, it clear that they have a worst quality, this is due to the stochastic term which imply the sample residual noise which disappear in the MMSE. By looking at the Standard deviation of the sample, one can remark that it is close to the error map between the MMSE estimator and the ground truth. The outstanding result is that PnP-ULA gives still good result on out-of-dise of--o--itiibbtiion data. The MMSE is still relevant but the uncertainty map given by the Standard deviation is not anymore relevant for that kind of data.
One can also remark artefacts on the reconstruction in the roof area of the second image. In fact for this image, if the Markov chain is running for more steps, these artefacts get worse. There an instability of the algorithm. This might be explain by the fact that the denoiser is not able to constraint the Markov chain for some frequencies who are zeros of the blur kernel (so not constraint also by the data fidelity term). So in these direction in the frequency domain, the Markov chain is free and still discover the space leading artefacts like aliasing.
Results on color scale The DRUNet denoiser is trained on RGB woman faces images of size 256 256 from the dataset celabA. The studied inverse problem and the parameter of the algorithm are the same than for gray scale images. The only modify parameter is the number of step, take at 30, 000, for faster computation.
We have similar result than previously. However the algorithm performance is more visible on the SSIM metric. In this case, the algorithm is also less powerful on out-of-distribution data. Especially on the image of cells, the reconstruction is not good. But the PnP-ULA still gives good results on other dittrf-distribution images, which are very different from woman face. One remark that the convergence is also a bit weaker. Especially in the different between the MMSE estimator and the ground-truth, artefacts as described before are more visible.
C.I. DO WE CAPTURE MULTIMODALITY WITH PNP-ULA? ?
The problem of capturing the multimodality of the posterior distribution is the key problem of sam- pling methods (Cohen et al., 2023). Because the inverse problem is ill-posed, multiple proper solu- tions can correspond to a unique observation. With multiple samples of the posterior law p(-ly), we aim to capture the different possibility of reconstruction.
Looking at the eyebrow in Figure 9, we can see that different modes are discovered during the Markov chain process with different number of eyebrows. The posterior distribution seems to be well discovered because this detail ambiguous in the blurred image implying multimodality of the posterior distribution.
D DEMONSTRATION OF THEOREM
The proof is based on decomposition of || - 2 llrv detailed in Equation 11. Then, each term controlled independently by the exponential convergence of the Markov Chain (Fquation 23) and by consequence of Girsanov's theorem (Equation 15). Finally, the demonstration in Wasserstein distance deduced from the inequality in the T'-nnomm Note that the bound in Wasserstein distance might be demonstrated with similar proof as the one in the TV-norm, leading to a more refindd inequality.
D.1 CONTROL OF || +iI
Definition D.1 (The V-norm for distributions). The V-norm (for V Rd + [1, +o00 of a distribu- tion is defined by
sup // f(z)dT(x))
|
II//V|
The TV-oorm is the V-norm with V = For every distribution Rd, ||]||r & ||]]
18

---

Published as a conference paper at ICLR 2024
Oued
Dhhen Nii and
IN Mmn.---... M-0-10
MARHOE 2010--9-11
PARKOD JANARHOL
154--22 1T00 M-044
MAR-34 45550-444
018-17 11/0 M-0
Figure 8: Result of PnP-ULA on different color images for 30, 000 steps with Drunet. Top four images are in-distribution and bottom fouor images out-of-diitribution images.
D.1.1 FIRST DECOMPOSITION
The Markov kernel of the Markov chain is defined by, for i E [1,2]
(2T) / 1a (1x 5b) + /25z) exp
|| 2
R (2, A)
dz.
Ri (2x A) is the probability that the Markov Chain in æ go in the set A at next iteration. The Markov Kernel is the transition matrix of the Markov Chain the number of state finite. Here the space of state is Rdd leading to this continuous definition.
By definition of the invariant law, because the Markov chain (æ ken converges in law to n/ we gets VN E N, x
[R:
= T
The first step of the bounding of || NHollrV is to remark that, VN E N, Va E Rd:
|| 22alluv $ (R),)) rsollor + (R,,)) 5z (R2 1ay + 118m (R - - llvv.
(11)
19

---

Published as a conference paper at ICLR 2024
Ground Truth
Sample
Sample 20000
PSNR 28.54, SSIM 0.76
PSNR 29.85, SSIM 0.79
Osservation PSNR 22.44, SSIM 0.66
Sample 75000 29.92, SSIM
MMSE PSNR 33.84, SSIM 0.94
Figure 9: Patches of different sample from one PnP-ULA Markov chain image Simpson
oz (R is the probability density of the N iterate of the Markov chain which begin in (so with the distribution 5z).
We will first bound separately |1 (Ri,,) - -T and - (R2 Mlrv, then deduce a bound on the target quantity.
-
1.1.2 BOUND OF [|6 (R 3) slTV
By (Laumont et al., 2022, Proposition 5), Va E Idd, there exists 01, 02 E R+ and pl, P2 EJO, 1[, such that, for i {1,2)
6 (R) - +clllrv S ( - TelllV S appi (r²(x)+ + / (2(
with the V-norm defined in Definition D.I.
D.1.3 BOUND OF || [R - Sz (R? lTv
We will suppose that 5 = and control the quantity defined for k E IN by
| )k - 8. (R? km
)km ||
We can make the decomposition
| (R__) jkm 5z (n,
Σ
oa R [kk 1)m _ (RRE (
(R²
(R² pim lTv.
TV
1=0
(12)
We introduce the continuous Markov processes, (#)) for E {1,2), defined to be the strong solution of:
dēc = (t, (z)) + 22db'
(13)
20

---

Published as a conference paper at ICLR 2024
With z dæs (t, (2)))e Ei=o (t( and bj continuous Wiener process. For simplicity of notation, in the following computation we will write (z() instead of (1, (3t))
From this definition, Øis oa for k E [0, m 1]. And one can remark, that mó 1.
Moment control First we control the moment of at. The moment bound, Vj IN', with for i E {0, 1) is defined by
PE [|xx|=]) sup (Sy=o_eell>>11 +E(0,+00)
my = max
sup tEf0,+oo|
with (P)) the Markov kernels associated with process 13.
For (xx ||=]| By (Laumont et al., 2022, Lemma 17), with ő = [1 (L+ M+1 +)
XE e [0, 1[. such that 3 C2 2 0, such that vi (1,2), v8 eJ0, 8), V E Rd, Vk N
P;W, - C,W,
8 P:W, - C
E Ex P!
|||||
S (x))
||| ||||||]
s (x)
x~ Pi
sup 1e|0,+20f
(Ex |||
I (x).
Note that (ar) is independent of 6. It has been shown that: AM₁ 0 independent of such as vo EjO, 0]:
my (n)
(14)
Control of (R 1m (R? ( )m1l/v
For any i E (1,2), we have
|| [ [R] 1mth
de (R? 1 || 6 Po 1fllv.
""
with (Pi) the Markov kernels associated with process 13.
Using (Laumont et al., 2022, Lemma 19), which is direct consequence of the Girsanov's theorem (Shiryayev, 1977, Theorem 7.7), we get the bound
11 1 /2 (1 -( di
6 [ )m (
di
(15)
Estimation of v So (112((())
- (2( dt
For / E [0, 1]. thank to the form of be, a L has the same law that a
can deduce from the previous property an estimation of the error
IE (16. i2 (=a)))))) EE (10(( (² ((1])10)))))
(16)
E (1)) -()))))))((())))
((())) (1bb 12--666 63(0) all) LE -16
+ (16b i (8) 6.(()) ((1))l)
(17)
21

---

Published as a conference paper at ICLR 2024
The first term be control by the fact that by and b2 are La M+1 + L Lipschitz, which immediately follows from the Assumptions I and 3
E [10] (a))) ((( E (10)(()) -=)((())))))) (110))))
E (()b (=))) (+ - (2(
E (I)( (())_ [b]( [
S F (a?() (m (=1b) + (2 H1O) ((())11))
- 245E (Iā²) - x
by is Lipschitz, so there exists D. S 0, such that Va E Rd, Vi E (1,2),
(x)) - D2 + |==1)
(18)
By the Itō's isometry, Equation 18 and Equation 14, we get
11 [
(1/-" db!112
E [ a [11)) s E
by (((ddull?
+ 2E
1311
S (2d + 2D. + Mz)2(
The previous inequality gives that
- b( - E (110(())))) (10)
63( - < (2d+ M2)2(2)))o
(19)
Now we control the second term of the error estimation
E (a))) - (()))) + E (10)() - b8 (=
< E (11! (atii) + E (1((()))1 + (116( + E (169 ( [
2D. (2 + || + |1aa)))))
< 4D, (1 + (x))
Thus the second part of the error estimation is bounded by
E (I(((()))
(+))) + E (# - 62( ([(=)) < 4D. (1+ (W))
(20)
Using Equation 16, Equation 19 and Equation 20 obtain the error estimation
E (ab! - (3( E (# (1a) 6, ([(()11)01)) s &DD.LG(())VO,
With G(æ)) (2d 2D. (2)) (L) (x))
Hence, we get an estimation of the integral Equation 15
W..' e (10A(at))-2(())))))) de V..00 (117(()))1) 62 [2(0tt10)]22) di S (8)t
(21)
22

---

Published as a conference paper at ICLR 2024
Control of [|6 1m 6 (R, >rVllv
Now we will deduce from the previous paragraph bound on (|0,, e, 1. 1 Equation 21 and that ă Li:e J8 has the same law that 2 3J
-m- dz (R2 1 )"IIT. Using
L. E (16L(*
(R),
8m RR )m
I V2
dt + V16D,L,G((001
S 2
- |11 TE - di + (16De
$ 2
E ((())))) V1DD,LC)++))...)EEE))))
m
Hence, we denote Do = v16DeLz and it gives
[1 1m - 60 (R ( )r S V2
+ DovG(z)) (22)
m
Control of | ykm oz RR2
Jon ||v
Using Laumont et al. (2022, Proposition 10), there exist Ag 2 and PC E [0, 1[ such that for any x, y E Rd. N E N and E (1,2):
|| (RC )N
1 S AcPe (V²(x) (3)))
For f :Rd R measurable such Vo Rd. lf(x)) S V(z).. This result combine with (Laumont et al., 2022, Lemma 17), shows that there exists B. < 0 such that Vo e Rd, VN E N, Vm E IN and Vi E [1,2}:
oze
RC [
- [f] V2
(23)
Using Equation 22 and Equation 23, we have )m
8m RR)
RR²
)m
RR [f]] = 60 (R) 1 ) 1m--(1 (R² )m
(RR² )
π T (5)
mm
(R²
fim[f]
Tt m
[f]
(())
8m [R. )m (R² c
)m)
f.m
BaPe
fm V(x)
< Bape im (xx)ll0
1 1m (R²2 c,
)m)
11v
m-1
S /2Baro"V
(((+))))) Dov G(z) V
m
In the last inequality, the Markov chain x has be initialized with the distribution 8n. In the follow- ing, we will denote at,e the Markov chain defined in 13 with the initial distribution dω RR. 1 )² By using the decomposition in Equation 12 and the previous computation, we get
k-1
|10 (121 ) jk - (122 )
km
1 (R. )0- (1R! -(R? mm) (R2 1 jomlly
1=0
1 m-1
$
/2BapC V(x)
Σ m
(10 - 111
1=0
1=0
DDBapp im 8 V(I) p(----1) (V(x))
23

---

Published as a conference paper at ICLR 2024
By the Jensen inequality, the inequality E(X) VE(X2) and the moment control (Equation 14), there exists g(x) < too such that
Ex- 1 (VG(x) E 30----1) (Vied+2Da 20, + M2W2(x))) M,)((x)
Rx 10-3--12
(V(2d (v - M
Ex ya-5-11m ( --
1 11 2,----...000222--000000044604110
+ (A
+ MMM (
2M2 4M2(1 M,W4 1
g(x))
with g(x))* 4d + 81 + M2)( + 4D.(1 + M2 + M22((=))) + 42, +
2M, (2(2)) + 4M2(11 (z))(
By summing the previous inequality, have
Σ De im 8 VZ)) 1im--1mm (VG(x)) 1
8
1-Po
((x) 8
1-
We denote D1 Lo) We have prove that
1-p2
DoBare im 8
10m--111m (VG(x) S D1St.
(24)
1=0
Then we study the convergence of the term m Er-ote E (10 - (x= We denote y ~ Th.S random variable. By similar computation than in paragraph D.1.3, we have
m
(100((0))1-))111)))) - b22 (m) (a- (1-3)1)))11 ((1b)(y)) -(()))1)))
m
M (1000/111111.01111111 -113)
Moreover, using Equation 23:
E (1 (1---(()))1)) yl1²
F
||| [ (1 2 mt (1 (ttas--)mmt))) fm)
< [| 1)m x/12) $ E (10)))-)-1)4tt))
(1))
S E - y1p) 1
!
de de ( (k- 17m ((--)-----
(k--- (²
2M, W1 BaPc
24

---

Published as a conference paper at ICLR 2024
By using the two previous computations, obtain
b2 (= E (1)b2(y) ((()))19))
16DL + ( a 2-2- Dm-- (2(
1=0
16DL, Ba
(6-1--1
(L+
VPC VPC
Then looking at the accumulation of these errors, we have
(1
-60(2) V ((1b) 62 -
m
1=0
4 B, V(1 (M (x)
mm (1- Pc Pc
VT VPC
4 M/ BaB
(L)
V1 /P-VPC
We denote D2 A.IDLAM,.BB. y VI-PP
(1+ M1 (W we have demonstrated that the
accumulation of error is controlled by
(1
3BB.C 2m V(x)
1
E
(10 -6((=1((k5--1mmm) -
|16]
(1
s D2kpc
1mm
1=0
(25)
Hence, we have
k-
V2B.V (xx)
/2Bare (()) ((y))
Eymete ()lb)(y) - (2()
1 10
0
VB Vx)
VE (1bb (1())
1 o
By denoting D3 -2B, ((x)
we get
1-p8
k-1
im 22Bacc
(P)) & Day D Eyunty (1bb(()) - 12(
(26)
1=0
Combining Equation 24, Equation 25 and Equation 26, we get
(k-1)m
[m (R. ) 5m 5RR
| S D, D22Pc + Day Ey (lbb(y) - (y)
(27)
D.1.4 DEMONSTRATION OF UVV-DISTANCE BOUND OF THEOREM 1
We go back in the initial decomposition (Equation 11). By taking max(a1, oz) E Rt and p= max(p1, P2) eJ0, 1[ in Equation D.1.2, and using Equation 27, for ls E N and m we get
||T' -.lllrV S 110x ) - =!lllTV (1.., -6x (12
yom lTv + 110 (RE - T2.6ITVV
amB (2V2(0) + Rd
(²( (do) Dj D2kip (--1)
Day Eywmt ((b
25

---

Published as a conference paper at ICLR 2024
By taking, h + to0, have for 8 1 < <
|| - lcl S Eymat (1] (y) b²()) + D
The parameter a (appear inside the constants) is free in the last inequality. We choose a: gives that there exist two constants Ao, Bo 2 0 defined by
A. = D3
BB D1
such as for d 1 8
(L+ M+1 + -
|| -,l I Agd (b)) Boo+.
With the posterior-I- defined 4.1. This demonstrated the Theorem
D.1.5 DEMONSTRATION OF W WI-DISTANCE BOUND OF THEOREM I Let and y be two random variable of laws Th, and y 22.5
By the definition of the Wasserstein distance (Equation 8) and the Cauchy-Schward inequality
WI x2,5) S E|| x y
s [[|| y >x|4x
S [[[[|||| + I|vl)
[[[[x|] |1 1 1/2
S / - 11x111 11/2
S / ||x11 T.,slliv.
1/22
Using Cauchy-Schward incquality and the moment bound (Equation 14)
W1 2 s E[[| x y'll
< 2 M2 || |T. n2 1/2
< 2/ M2 //22 (Ad (b-, (2) + Boot) $
s 2 Ao (ds (b_, (b),))))) + 2 Mr Buos
By the first part of Theorem 1, the total variation distance between the two invariant distributions be bound. We denote the constant A 2/ Ao and B1 2 2//Mv/B3 The following theorem have been prove, which shows the second part of the Theorem 1.
D.1.6 ANALYSIS THE CONSTANT Ao. Bo. A AND B₁
In this section, we analyse the dependency of the constant Ag and Bo. first of all by construction, these constants are independent of å EJ0, 8). with 8 = (L + M+1 } Also these constant are dependent of the inverse problem observation y, in fact they depend of the targeted posterior distribution (through the constant Ba, pc).
Study of Ag The definition of Ag is
V2B
Ag = D3
1 PC
Where B,, and PC are the constant defined in the Equation 23. Bortoli & Durmus (2020) has proved that PC: is independent of the dimension and moreover this rate is closed to the optimal rate thank
26

---

Published as a conference paper at ICLR 2024
to Eberle (2016, Lemma 2). However Bortoli & Durmus (2020) has also prove that the constant B, has a polynomial dependent on the dimension. So there exists 2 and C,- > such that
Ba d00
C,d²
(28)
Thus Ag has a polynomial dependency on the dimension
V2C, d.
Ao
(29)
d++000 - PC
Study of Bo
The definition of Bo
Bo D1 B,
1 PC
(1 pt) ((dd 81 M2) + 4D,(1 M2(1 M2) + 4D, 2M2 4M2(1 + Ma)))
V16D.TaB.).
Thanks to the dependency of Ba the dimension in Equation 28, By has the dependency
Bo
d+++0 (1 (1- (4+ (4
+ M2)) &
(30)
Recall that M1 and M2 are the moment upper-bound, La Lipschitz constant of by and De = La + [0]]l. These quantity has no dependency on the dimension.
Study of A
The definition of A₁ is
A = 24 Ap-
Using Equation 29, A has the dimension dependence
V2C,
A do+o0
2/M2
P
Study of B1
The definition of Bj is
2v Mzv B
Using Equation 30, Bj has the following dimension dependence
B d+0000 2/ (1 -1 + Ma)) d8
D.1.7 ADDITIONAL RESULTS
With a similar demonstration than before, we can obtain bound on the MMSE estimator or the standard deviation of the Markov chain instead of the distribution. In fact, in practice, mainly look at the MMSE (mean of the Markov chain) and the standard deviation of the Markov chain (as confidence map).
27

---

Published as a conference paper at ICLR 2024
MMSE error bound
With the same and y random variable than above
||[[x] Elylll
= E[|x -yl
s 22 M2] _e, - 2,, 117
The expectation of T2,5 is the MMSE estimator of the inverse problem with the prior defined by the first denoiser. If we name, for i E {1, MMSE been prove (with As A1, B3 B1). Proposition 1.1. It As: Bs R+, such for 5 Ej0,ō]:
F x [x]. The following proposition has
MMSSEI @MMSE.2l & As (di (b). BB0
(31)
Standard deviation error bound
To control the difference between variance of the samples
Var[x] - Var[y]]
= x|| |[[ || + ||
[ | [] Ellly | + [ ||[
E[ |x| || 1x -||xx y [|] + |[||]l)
(v 2M 2 2M2 MMM MI =
We denote As = (v2M4 2M2MM V and Bg Bo. the following proposition has been proved.
Proposition 1.2. There exist As, Bg E R+ such that for 8 E]
Var[m Vari & As (d1 b.)) $ + Boos.
(32)
E COROLLARY DEMONSTRATIONS
E. DEMONSTRATION OF COROLLARY 1.1
the forward problem the same for the two Markov chain, the only shift is on the denoiser. Then, the distance between the two drift is
di d1 dbbb VExwat, (II D2(X))P) 1 dD D.).
Hence have
| rbl I Au di dj D..) + B
So the result 1.1 has been demonstrated with A₂ = Ao and BN = Bo.
E.2 DEMONSTRATION OF COROLLARY 1.2
If D! D the exact MMSE denoiser. Some more assumption are needed on the prior. We defi,e
|02 21113)
) [ (*))
||2
x612 dā.
ge (m12222) = p² (x1) exp
2c
2c
Assumption 5. For any e >> O, there exists K. 2 0 such that Vo E Rd
f ||
/ á' g' (1'( ()) ( K..
And
f (1
11) < +00.
Rd
28

---

Published as a conference paper at ICLR 2024
Using Assumptions 1-5 and (Laumont et al., 2022, Proposition 6). applied with the exact MMSE denoiser (MR = and R = +00). If ((x) = 1 + | for é I co. and À > such that 2M(Ly + - - min (m, 0)) I and 8 = 3/ (Ly + L + x) There cxists D4 2 0 such that for
Rc > 0 such that B(, Rc) C S, there exists D3,c 2 0 such that Vo ē]
| pe(-b))lvv s Di + Do,ovō.
Due to 1lrv $ llv, we have
1 -e---llllll S D.Rle + Do.ovō.
Using the triangular inequality and Corollary 1.1, for 0 < 8 < min (8, 1)
(pe(-lv) Nē,allTv S || - -T, + | Pe Pel-
S A2d1 (D,, D..) B2d + R, + Ds.cV8
- A2d1 (D D2) + (B2 + Ds,c)o+= + --R
Finally, the corollary 1.2 has been demonstrated with Ag = A1, B3 = =B2 + D2,c) and C3 = D1-
E.3 DEMONSTRATION OF COROLLARY 1.3
In this case, the denoiser is fix but there is a shift on the forward problem.
D D2
V logg' p (3)l) + V logr"(yl)) Because of the forward model 2, the likelihood has the form
Vx (ogp'(y/z) -V y (x)11
σ2 A (y A.æ)).
202
Hence, we have
(x) - (z(z)) V x (zgzx - V logpp -/2 (A A1x) Ab A2x))
0² ((A1 A2) A - A2)x)
A1
Moreover, we can develop
A A1
A/
Af A
A5Ai A1 AT A ATA,- (Ai A2) A + AA A2).
So, with the operator norm, note || ||. on matrix associated with the Euclidean norm, the following inequality holds
1
(x) - (x))1
T² ||4 A2) - (Ai A²) A (A A2) |
< || Aall (11=1 + (1 + 2| [=
Hence, we have
[b)( - -b,00)b))0)11 (x)
< ||A A2ll (1y1l (11 || + ||a
< 2 A - A₂ 2 (1 2 + (Il A + || 111 |0: 112)
If A₁ and A₁ are in a bound domain. There exists M such that M N max (1) A1l. ||A21)), it gives:
[6] (z))) < A| Aoll2 (1ly|1 + 4
Hence, we have
di (b)
v E Ex (x)bl(x) [[(xx())
v 2| 21 Azll² (11/12 2 X (1|x|12)
< 2211A1 - -A2ll (ll/) + 2M / /EX (1
29

---

Published as a conference paper at ICLR 2024
Using the moment bound (Equation 14), / Ex (1x(()) S VM2.
Then Theorem 1 gives, that v5 c]
||= - I Aodi 62) Boo
Therefore with A4 = -2 (l|vll + 2M (M) 2 0 and B4 = Bo, corollary 1.3 has been proved.
30


Reviews:
[{"review_id": "AayKm0FhGo", "reviewer": "Reviewer_eQV2", "summary": "The authors study the influence of model mismatch on the invariant distribution of a certain model-based posterior sampler based on the unadjusted Langevin Algorithm. The model (a forward operator in some imaging application) and the prior (incorporated via a denoiser) factor in through the drift term. The authors prove that the distribution drift is controlled by the total size of the model mismatch (in the forward model and the denoiser).", "questions": "- Under \"Contributions\" you say that you \"provide a more explicit re-evaluation of the previous convergence results...\", but I am not sure what this means.\n\n- In \"we focus on the task of sampling the posterior distribution to reconstruct various solutions...\", what is meant by \"various solutions\"?\n\n- In \"... Markov chain can be naturally obtained from an Euler-Maruyama discretisation by reformulating the process ...\", what is meant by \"reformulating\"?\n\n- Before equation (7), Wasserstein norm should be Wasserstein metric (or distance); before (7), TV distance should be TV norm (which is what is defined in (7)). (Also: why are Rd vectors bold in (8) and not in (7)?)\n\n- \"pseudometric between two functions in Rd\" -> taking values in Rd\n\n- In Corollary 1.3 which norm is || A^1 - A^2 ||?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 2, "strengths": "It is great that the paper proves a theoretical result about a relevant topic where most work is highly empirical. The setting is very clear and the derivations seem sound (although I have not checked in great detail.) The authors work under fairly general assumptions and also illustrate their bounds empirically.", "weaknesses": "- The contribution is somewhat incremental given all the preparatory work in Laumont et al. 2022. Model mismatch is certainly a relevant topic and it is nice to have a paper about it so this is a somewhat subjective statement relative to papers I've reviewed for ICLR this year. \n\n- The prose is waffly, with too much hyperbole. An example: \"In this section, our focal point resides in the investigation of the profound impact of a drift shift on the invariant distribution of the PnP-ULA Markov chain\" which could be \"In this section we study the impact of drift shift on...\".\n\n- There are also numerous typos and broken sentences, especially in the appendices.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HK4LoueqbS", "reviewer": "Reviewer_Uuqz", "summary": "This paper considers sensitivity analysis of posterior sampling in inverse problems using diffusion models. It analyzes the effects of mismatches to the drift function on the stationary distribution of Langevin sampling.The mismatch can arise due to uncertainty in the forward operator and due to the denoiser not being exactly the MMSE denoiser.\n\nThe main result is that the stationary distributions differ proportional to a pseudometric that depends on the drift mismatch.", "questions": "Listed in the weaknesses section", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The considered problem is relevant, especially in medical imaging, where we want algorithms to be robust to mismatch in the forward model.\n\nInverse problems using diffusion models is also an active area of research and the proposed results could be relevant.", "weaknesses": "- The main result in Theorem 1 shows that the TV between the stationary distributions of two Markov chains that have different drift functions can be bounded in terms of the proposed ``posterior-$L_2$ pseudometric''. This pseudometric is defined in terms of the expectation of the difference between the two drift functions when samples are drawn from the stationary distribution of one of the drifts. It's not clear at all how this pseudo metric behaves, and whether it is sufficiently small for two drifts that are close. (the $\\epsilon$ used in the results is only for the mollification level present in the denoiser, and has nothing to do with the closesness of the drifts themselves). \n\n- It is also not clear how different the two stationary distributions are when compared to the continuous stationary distribution. This can be very different due to discretization error, etc.\n\n- There is very little comparison to existing results in the literature. Other than saying that their results are backwards compatible with Laumont et al 2022, the authors do not state what benefits / drawbacks their results face.\n\n- The paper considers Langevin sampling, which is known to not mix very well -- most theoretical results in the literature consider ODE / SDE solvers for an Ornstein–Uhlenbeck process.\n\n- The upper bounds in Theorem 1 are specified in terms of $A_0, A_1, B_0, B_1$, without any mention on the dimension dependence of these quantities.\n\n- Some statements are unsubstantiated. In the contributions section, the authors claim \"This paper stresses that in the case of mismatched operators, there are no error accumulations.\" I don't see why this would we be true.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yD7JybYEpK", "reviewer": "Reviewer_rc8y", "summary": "The paper studies the error bound of the plug-and-play unadjusted Langevin algorithm (PnP-ULA) under mismatched posterior distribution owing to mismatched data fidelity and/or denoiser. After rigorously deriving their main theoretical results, they provide some numerical experiments with simple settings to further support their theory.", "questions": "See above\n\n____\nAfter rebuttal: the authors addressed my concerns, thus I raise my score.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Sections 1–4 are generally clearly written. The readers can get what the authors try to convey without diving into the mathematical details.\n- The quantification of the error bound for PnP-ULA under a mismatched posterior distribution is of theoretical importance.", "weaknesses": "- The section associated with the numerical experiment is hard to dig out. Particularly, it's not easy to understand how and why the proposed setting can be adopted to validate the theoretical corollary.\n- As claimed by the authors, \"our results can be seen as a PnP counterpart of existing results on diffusion models.\", which therefore weakens the novelty of this paper.\n- It seems like the theoretical results drawn rely on \"oracle\" information that is unavailable in practice. So the practical use of this theoretical tool is largely unclear for me.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tApjb562mq", "reviewer": "Reviewer_WmLx", "summary": "The paper proposes a theory of the plug-and-play unadjusted Langevin algorithm (PnP-ULA) to solve inverse problems, that builds and improves upon a prior work [1]. Specifically, the main theorem quantifies the distributional error of PnP-ULA under prior shift and the likelihood shift, which are both practical questions in the field of inverse imaging. Numerical experiments including the analytical GMM experiment and the image deblurring experiment solidify the correctness of the theorem.", "questions": "1. In the forward model shift experiment on color image deblurring, what happens if one takes $\\sigma > 3$, and taking to the extreme, when one uses a uniform blur kernel?\n\n2. For image deblurring, what happens when you have an anisotropic blur kernel, but you use an isotropic kernel for inference?\n\n3. Two different versions of references are given for [1]\n\n4. It is probably better to cite [2] rather than [3] for score-matching (pg. 2)\n\n\n\n\n\n**References**\n\n[1] Chung, Hyungjin, et al. \"Diffusion posterior sampling for general noisy inverse problems.\" ICLR 2023.\n\n[2] Vincent, Pascal. \"A connection between score matching and denoising autoencoders.\" Neural computation 23.7 (2011): 1661-1674.\n\n[3] Dhariwal, Prafulla, and Alexander Nichol. \"Diffusion models beat gans on image synthesis.\" NeurIPS 2022", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, concise, and clear.\n\n2. The theory given in the paper is solid, with numerical experiments building support for the proposed theorem. The theory is practical and non-vacuous, as mismatch in the prior or in the likelihood happens (at least to a minimal amount) on virtually every application that you can think of.\n\n3. The subject of the paper is well-suited for the conference, given the popularity of diffusion models on solving inverse problems.", "weaknesses": "1. The prior mismatch model, and the likelihood mismatch model, are not too realistic.\n\n1-1. For the prior mismatch, it would be more interesting if one could see the effect when the underlying training distributions are different. For example, in the context of medical imaging, [5,6] demonstrated that diffusion models are particularly robust under prior distribution shifts. A discussion would be useful.\n\nNote that I understand why the authors opted for the number of parameters for a DNN when they assumed a mismatch from the perfect MMSE estimator. However, given the current landscape of ML/generative AI, this situation would be easily solvable by more compute, whereas solving the data distribution shift is a much harder and realistic problem.\n\n2. The authors cite [1] as an example of a mismatched imaging forward model. Correct me if I am wrong, but as far as I understand, when using unconditional denoisers as in PnP-ULA, [1] uses the exact forward operator that were used to generate the measurement. I believe references such as [2-4] would be more relevant.\n\n\n\n**References**\n\n[1] Güngör, Alper, et al. \"Adaptive diffusion priors for accelerated MRI reconstruction.\" Medical Image Analysis (2023): 102872.\n\n[2] Wirgin, Armand. \"The inverse crime.\" 2004.\n\n[3] Guerquin-Kern, Matthieu, et al. \"Realistic analytical phantoms for parallel magnetic resonance imaging.\" IEEE Transactions on Medical Imaging 31.3 (2011): 626-636.\n\n[4] Blanke, Stephanie E., Bernadette N. Hahn, and Anne Wald. \"Inverse problems with inexact forward operator: iterative regularization and application in dynamic imaging.\" Inverse Problems 36.12 (2020): 124001.\n\n[5] Jalal, Ajil, et al. \"Robust compressed sensing mri with deep generative priors.\" NeurIPS 2021.\n\n[6] Chung, Hyungjin, and Jong Chul Ye. \"Score-based diffusion models for accelerated MRI.\" Medical image analysis 80 (2022): 102479.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)