Variational Diffusion Posterior Sampling with Midpoint Guidance
Authors: Badr MOUFAD
Yazid Janati
Lisa Bedin
Alain Oliviero Durmus
randal douc
Eric Moulines
Jimmy Olsson
Venue: ICLR 2025 Oral (2025)
Keywords: Diffusion models
Inverse problems
posterior sampling
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Diffusion models have recently shown considerable potential in solving Bayesian inverse problems when used as priors. However, sampling from the resulting denoising posterior distributions remains a challenge as it involves intractable terms. To tackle this issue, state-of-the-art approaches formulate the problem as that of sampling from a surrogate diffusion model targeting the posterior and decompose its scores into two terms: the prior score and an intractable guidance term. While the former is replaced by the pre-trained score of the considered diffusion model, the guidance term has to be estimated. In this paper, we propose a novel approach that utilises a decomposition of the transitions which, in contrast to previous methods, allows a trade-off between the complexity of the intractable guidance term and that of the prior transitions. We validate the proposed approach through extensive experiments on linear and nonlinear inverse problems, including challenging cases with latent diffusion models as priors, and demonstrate its effectiveness in reconstructing electrocardiogram (ECG) from partial measurements for accurate cardiac diagnosis.

Published as a conference paper at ICLR 2025
VARIATIONAL DIFFUSION POSTERIOR SAMPLING WITH MIDPOINT GUIDANCE
Badr Moufad Yazid Janati Lisa Bedin
Alain Durmus' Randal Douc Eric Moulines Jimmy Olsson
Ecole polytechnique Télécom SudParis 3KTH Royal Institute of Technology
ABSTRACT
Diffusion models have recently shown considerable potential in solving Bayesian inverse problems when used as priors. However, sampling from the resulting denoising posterior distributions remains challenge it involves intractable terms. To tackle this issue, state-of-the-ar approaches formulate the problem as that of sampling from a surrogate diffusion model targeting the posterior and decompose its scores into two terms: the prior score and an intractable guidance term. While the former is replaced by the pre-trained score of the considered diffusion model, the guidance term has to be estimated. In this paper, we propose novel approach that utilises a decomposition of the transitions which, in contrast to previous methods, allows trade-off between the complexity of the intractable guidance term and that of the prior transitions. We validate the proposed approach through extensive experiments on linear and nonlinear inverse problems, including challenging cases with latent diffusion models as priors. We then demonstrate its applicability various modalities and its promising impact on public health by tackling cardiovascular disease diagnosis through the reconstruction of incomplete electrocardiograms. The code is publicly available https: / //github. com/ aanijaantttiiaitsaa
INTRODUCTION
Inverse problems aim to reconstruct signals from incomplete and noisy observations and are prevalent across various fields. In signal and image processing. common examples include signal deconvo- lution, image restoration, and tomographic image reconstruction (Stuart, 2010; Idier, 2013). Other applications extend to protein backbone motif scaffolding (Watson et al., 2023) and urban mobility modeling (Jiang et al., 2023). Due to the presence of noise and the inherent complexity of the measurement process, these problems are typically ill-posed, meaning they have an infinite number of possible solutions. While many of these solutions may fit the observed data, only few align with the true underlying signal. Bayesian inverse problems provide a principled framework for addressing the challenges of signal reconstruction by incorporating prior knowledge, enabling more plausible and meaningful solutions. The a priori knowledge about the signal x to be reconstructed is captured in the prior distribution g(x), while the information about the observation y is encoded in the likelihood function pxy(x). Given these components, solving the inverse problem boils down to sampling from the posterior distribution (xx) x ((ylx))((x)) which integrates both prior knowledge and observational data.
The choice of the prior distribution crucial for achieving accurate reconstructions. If the goal is to reconstruct high-resolution data, the prior must be able to model data with similar fine detail and complexity. Recently, denoising diffusion models (DDMs) (Sohl-Dickstein al., 2015: Song & Ermon, 2019; Ho et al., 2020; Song et al., 2021b) have emerged as powerful approach in this context. These models can generate highly realistic and detailed reconstructions and are becoming increasingly popular as priors. Using DDM, sample X, being approximately distributed according to given data distribution 9 of interest can be generated by iteratively denoising an initial sample
authors contributed equally. Correspondence: {fi rstname secondname@po wtwpaeeeeppoopooooooemmmm ytehhhiciggo ytedhnique.eddu

---

Published as a conference paper at ICLR 2025
X, from standard Gaussian distribution. The denoising process (Xx) consists of the iterative refinement of X3 for k E [1.n] based on a parametric approximation of the score V log qk. where (x(xx)) [ ((xx)))x(((xx Xo) dxg, with (ejo(-(xo) being Gaussian noising transition.
Denoising Diffusion Models form a highly powerful class of prior distributions, but their use intro- duces significant challenges in posterior sampling. Specifically, since DDM prior does not admit an explicit and tractable density, conventional Markov chain Monte Carlo (MCMC) methods, such as the Metropolis-Hastings algorithm and its variants (Besag, 1994; Neal, 201 cannot be applied in general. Furthermore, gradient-based MCMC methods often prove inefficient, as they tend to get trapped in local modes of the posterior.
The problem of sampling from the posterior in Bayesian inverse problems with DDM priors has recently been addressed in several papers (Kadkhodaic & Simoncelli, 2020; Song et al., 2021b; Kawar et al., 2021; 2022; Lugmayr et al., 2022; Ho et al., 2022; Chung et al., 2023; Song et al., 2023a). These methods typically modify the denoising process to account for the observation y. One principled approach for accurate posterior sampling involves skewing the denoising process at each stage k using the score V log TR. where T& is defined analogously to gg+ with g replaced by the posterior и. This guides the denoising process in a manner that is consistent with the observation. Notably, the posterior score V log Ty& decomposes into two components: the prior score and an additional intractable term commonly referred to as guidance. Previous works leverage pre-trained DDMs for the prior score, while providing various training-free approximations for the guidance term (Ho et al., 2022; Chung et al., 2023; Song et al., 2023a; Finzi al., 2023). This framework enables solving a wide range of inverse problems for given prior. However, despite the notable success of these methods, the approximations often introduce significant errors, leading to inaccurate reconstructions when the posterior distribution is highly multimodal, the measurement process is strongly nonlinear, or the data is heavily contaminated by noise.
Our contribution. We begin with the observation that the posterior denoising step, which trans- forms a sample X into sample Xg. does not necessarily require the conditional scores V log T or V log ky. Instead, we demonstrate that this step can be decomposed into two intermediate phases: first, denoising X into an intermediate midpoint state Xi., where l < k, and then noising back to obtain Xs. unconditionally on the observation y. This decomposition introduces an additional degree of freedom, as it only requires the estimation of the guidance term at the intermediate step & rather than step k: + 1. Building on this insight, we introduce MIDPOINT GUIDANCE POSTERIOR SAMPLING (MGPS), a novel diffusion posterior sampling scheme that explicitly leverages this approach. Our algorithm develops principled approximation of the denoising transition by utilizing a Gaussian variational approximation, combined with the guidance approximation proposed by Chung et al. (2023) at the intermediate steps la- The strong empirical performance of our method is demonstrated through an extensive set of experiments. In particular, we validate our approach on Gaussian mixture toy example, as well various linear and nonlinear image reconstruction tasks inpaininpi,, super-resolution, phase retrieval, deblurring, JPEG dequantization, high-dynamic sange-using both pixel-space and latent diffusion models (LDM). Finally, we demonstrate the versatil- ity of our approach by applying it to cardiovascular diagnosis using reconstructed electrocardiograms, showing that MGPS achieves significant improvements over competing methods.
POSTERIOR SAMPLING WITH DDM PRIOR
Problem setup. In this paper, we focus on the approximate sampling from a density of the form
t T(x)= == p(y x /x))///
12.1)
where p(yl-): : Rd + R is a neg-ntttiiee and differentiable likelihood that can be evaluated pointwise, q is prior distribution, and 2 := p(y|()qq(>)) dx is the normalizing constant. We are interested in the case where a DDM p for the prior has been pre-trained based on i.i.d. observations from q. As by-product, we also have parametric approximations ($)R_ of the scores (V log qx )i where the marginals (==) are defined as 9, (xx) == J' ((xo) 4:jo (Xx (X)) dxo with (xjo(x)) (xx) (x)) := N(xx; VagXu, 2% I2) and 2/2 == ag. Typically, the sequence (aw)) is a decreasing sequence with ag and a, approximately equals zero for large enough.
These score approximations enable the generation of new samples from g according to one of the DDM sampling schemes (Song& & Ermon, 2019; Ho et al., 2020; Song al., 2021 a;;; Karras
2

---

Published as a conference paper at ICLR 2025
et al., 2022). All these approaches boil down to simulating Markov chain (Xx)k=n backwards in time starting from a standard Gaussian P., == N(Os,Ia and following the Gaussian transitions
Paje+11 (x& x1)+
(xx; m I-+1 (xx+1) + where the mean functions m k can be are fixed and pre-defined variances. The backward transitions are designed in such a way that the marginal law of X approximates y3 see Appendix A.
derived from the learned DDM, and Ukje+11
for more details on the form of the backward transitions.
In our problem setup, we assume we only have access to the approximated scores of the prior and no observation from neither the posterior π nor the prior q. This setup encompasses of solving an inverse problem without training conditional model from scratch based on paired signal/observation dataset; requirement typically imposed by conditional DDM frameworks (Song et al., 2021b; Batzolis et al., 2021; Tashiro et al., 2021; Saharia et al., 2022). On the other hand, we require access to p(yl.). This setup includes many applications in Bayesian inverse problems, e.g., image restoration, motif scaffolding in protein design (Trippe et al., 2023; Wu et al., 2023), and trajectory control in traffic-simulation frameworks (Jiang et al., 2023).
Conditional score. Since we have access to DDM model for q. natural approach to pling from is to define a DDM approximation based on the pre-trained score approximations for gu- Following the DDM approach, the basic idea here is sample sequentially from the marginals T& (xx)) == f x (xo) (x& xb) dxp by relying on approximations of the conditional scores (V log Th R The latter can be expressed in terms of the unconditional scores by using
Ty (xx) x f
p(y| (o(y|x)))(xx) (x))(xx)))))))))))))))))))))(X (x))))) (xxs Xu) dxu
(2.2)
where after defining the backward transition kernel (o)x((oo (xc+) = ((x0) (x& (x)) qu (xx)) yields
(xx)) (x())x)))(xxxs) where px(()(x))= = / ply (x (xx) dxo.
(2.3)
then follows that
V log (Xe) V log de (xx) + V (
While we have a parametric approximation of the first term on the r.h.s., the second term is the bottleneck as it involves the integration of p(yl-) against the conditional distribution Loju (-1x)).
Existing approaches. To circumvent this computational bottleneck, previous works have proposed approximations that involve replacing Ooja (xx) in (2.3) with either a Dirac delta mass (Chung et al., 2023) or a Gaussian approximation (Ho et al., 2022; Song et al., 2023a)
Oojs (xolxx) R (xoo m6 (xx), va)
(2.4)
where m is parametric approximation of maje (xx: := f xo 9 ( (xs) dxg and Voje a tun- ing parameter. Using Tweedie's formula (Robbins, 1956), it holds that mo (Xx) (-X; you V log qr (xx))//e, which suggests the approximation m (xx) == (-x: + Vaus& (xx) /)/uk- As for the variance parameter ko Ho et al. (2022) suggest setting Vojk = Uk/OK, while Song et al. (2023a) use dojk Uk. When p(yl-) is the likelihood of linear model, i.e., p(y(x) := N(;; Ax, of for A Rddxd and By > can be exactly integrated against the Gaussian ap- proximation (2.4), yielding p& (y(x)) x My; A TT + Pojk AAT). Chung et al. (2023), on the other hand, use the pointwise approximation, yielding pk (y[xx) ss p (xx)). We denote by pu (yl-) the resulting approximation stemming from any of these methods. Approximate samples from the posterior distribution are then drawn by simulating a backward Markov chain (Xx) where X,, Pn and then, given X X is obtained via the update
X = Nx + wh+1 (X log ( (x)))+1), where X₂ ~ (
(2.5)
and Wh+1 is a method-dependeet weighting function. For instance, Chung et al. (2023) use wk+1 (X++1) <ož/ /lly Am (Xx+1)ll, with E [0, 1]. We note that the update (2.5) in general involves a vector-Jacobian product, e.g., considering the approximation suggested by Chung et al. (2023), Vx, ( Vx, m (Vx, poy p(y(X))) (Xx) (000)))mm (xe) which incurs an additional computational cost compared to unconditional diffusion step.
3

---

Published as a conference paper at ICLR 2025
3 THE MGPS ALGORITHM
In this section we propose a novel scheme for the approximate inference of x. We start by presenting midpoint decomposition of the backward transition that allows us to trade adherence to the prior backward dynamics for improved guidance approximation.
We preface our description of the MGPS algorithm with some additional notations. First, consider the DDPM forward process with transitions given by 4xlj (x1](x//xx) == N N(xx; /as aaa//a (Rz/ayXy, (1 aak a /ay La)
for all (j, k)) E [[0, n?] such that j < k. We also define the joint law of the forward process
72
To: (Xom) == T (x0)
II A=U
(K+1
(3.1)
when initialized with the posterior (2.1) of interest. Note that with these definitions, T.. (given by (2.3)) is the marginal of Tu.n (Xu:n) w.r.t. Xg. The time-reversed decomposition of (3.1) writes
n-1
Toen (Xo:n) == nT (x,)
II k==
Th Teja+1 (x||x++1),
(3.2)
where, more generally, for all (i,5)= E [0, nI? such that <
"T T (x|1,)) T. (x, (x)))) (x))))
(3.3)
where the marginals are given by (2.2), the conditional density of X; given X, = x, and y. Using the decomposition (3.2), an exact draw Xo from (2.1) is obtained by first drawing Xn, from n. and then sim- ulating recursively X from the transitions x (+)x++) for all k. However, according to (3.3), such transitions in- volve the intractable likelihood pk (y (x.) which is hard approximate accurately when k is large.
Xo
X
X2
X
Midpoint decomposition. The transition (3.3) has components: the conditional density (x),), and the prior transition density (il (x,(xx(x,). The approximation ()())) p (x:)) of Chung al. (2023) is ac- curate when i is small, whereas the Gaussian approxima- tion of quy (x,) x2) proposed by Ho et al. (2020) is accu- rate when the difference li jl is small. Thus, in order to combine the best of both worlds, we introduce a de- composition that transfers the problem of sampling from T to that of sampling from " "(2]K+1' i.e., drawing Xt, given X and y, where 4 < k. The parameter l balances the approximation errors of the two conditional densities in (3.3); see Figure 1. In order to make this construction, define, for each E [0, kl]. the bridge kernel Xx), which is Gaussian; see Appendix A.I. By
Figure I: arrows indicate different conditional densi- ties that need to be approximated for a given choice of the midpoint Le. The longer arrow, the more difficult it to approximate the corresponding conditional density. By placing the la midway between zero and k the shortest arrows are obtained.
For each color, the different solid
Q Geje( (xxx|xx)) convention, 4k|R, Xxxx x1) = 8x..
Qeje,t+1 (xg xx,xx+1]
Lemma 3.1. For all k E [1, 1] and la E [0, k]] it holds that
/ Q Qe Xx, xx+1 e (x6+1) dx
Tejei (X54)
(3.4)
The proof is found in Appendix A.3. Lead by the decomposition provided by Lemma 3.1, we define
# (xx) xx+1)= Uxl62,A+1 (x. (Xe,, x
1 Ee (xe, (x++1) dx
(3.5)
where a 1 (xe, (x*+1) pl, (y|xe, Ip 1 (Xe, (x++1 with pl, (y|xe,) p(ylmg (xx.))
and p 1 (xea (x+1) = 4e 10,6+1 (Xe, | (xx+1), X6+1). (xx+)) Finally, for any sequence f = (Ex)) satisfying lo k. we define surrogate model for Wa:n and the posterior (2.1)
n-1
#6(x0) / - 1f66m (Xu:n) dxim (3.6)
Hon (Xo:n) = Rr (Xn)
II 1==0
# (xx (x442,
4

---

Published as a conference paper at ICLR 2025
where R, (x,) (x,;; Oa. Id) and 1 (1)xa) (xxg):== Ampnixi (x)) (similarly to Ho al., 2020; Song et al., 2021a). Since the bridge kernel .k+ is Gaussian, it can be easily sampled. Hence, we only need to focus on the approximate sampling from " Moreover, since the decomposition (3.4) is valid for any l,. it can be chosen to better balance the approximation errors, as discussed above.
Choice of the sequence (fx))=1- The accuracy of the surrogate model (3.6) ultimately depends on the design of the sequence (+x) In fact, for a given note that decrease in 4 ensures that the approximation př, (y(xx,) of ps, (y(x,,) becomes more accurate. For instance, setting lx eliminates any error of the approximate like- lihood (since (y(l).)
p(yl-)); how-
ever, decreasing la can cause the distribution Toje-+1 (-Ix++1) to become strongly multimodal, especially in the early stages of the diffusion pro- cess, making the DDPM Gaussian approxima- tion less accurate. Conversely, setting la closer to ht similar to the DPS approach of Chung et al. (2023), improves the accuracy of the approxima- tion of (ea/k+1. but impairs the approximation p?, (xx) of pe, (y xt,). The choice of & is therefore subject to particular trade-off, which we illustrate using the following Gaussian toy example. When considering the parameterization 6g = Lnn] with 17 E [0, 1]. we show that the W distance between т and o f(1) reaches its minimum for 7) around 0.5, which confirms that the additional flexibility obtained by introducing the midpoint can provide surrogate model (3.6) with reduced approximation error.
0.4
Wafa 2ig 510)
Frequency
0.2
0.0
0.2
01.5
0.8
0.6
0.7
0.8
7
Figure 2: Left: average W3 with 90%%%0%%-- quantile range. Right: distribution of the minimizing vy"..
Example 3.2. Consider a linear inverse problem Y AX + 7 with Gaussian noise and prior q N m, Σ), where m E TRd and Σ E Raxd is positive definite. this setting, we have access to the exact denoiser Mojk in a closed form, which allows to take m molk- Moreover, since the transition (3.5) admits an explicit expression in this case, we can quantify the approximation error of (3.6) w.r.l. for any sequence (6x) Note that in this case the true backward transitions qecle+1 are Gaussian and have the same mean as the Gaussian DDPM transitions, but differ in their covariance. All necessary derivations and computational details can be found in Appendix B. We consider sequences {(T) == (CT)) with f% = [nk, 7 E [0, 1). and compute the Wasserstein-2 2 (1)) function of n the basis of 500 samples (A, m, Σ) for d = 100. The
distance W2(n, No left panel of Figure 2 displays the average error Wz( #o =((n)) ) as a function of n,. while the right panel displays the associated distribution of := ar gemi W2 mo =11))) Figure 2 illustrates that the smallest approximation error for the surrogate model (3.6) is reached at intermediate values close to 7 = 0.5 rather than 17 1. which corresponds to a DPS-like approximation.
Variational approximation. To sample from th.n in (3.6), we focus on simulating approximately and recursively (over k) the Markov chain with transition densities (3.5). Idcally, for tt n, we simply sample from a then, recursively, assuming that we have access to an approximate sample X from # # the next state X₁ is drawn from #f *b|4+1 (+)X&+1). However, as * (+)x++1) is intractable, we propose using the Gaussian approximation that we specify next.
In the following, let & E [1, and Ea E [0, be fixed. For 4p (Fre Pee)) E (R2)2, consider the Gaussian variational distribution
X Mjk++ (xx]Xx++1 / (+ Xx+1 )A k++1 (xfe) dxex
(3.7)
1
21 (x=)):=N(x)); Mec. diag(e PPc) ))-
(3.8)
In definition (3.8) the exponential function is applied element-wise to the vector Pr. and diag(e
the diagonal matrix with diagonal entries e.t., Based on the family { 60 E (R2))2 and an approximate sample X4+1 from 7f we then seek the best fitting parameter that minimizes the upper bound on the backward KL divergence obtained using the data-proe inequality
KL kjk+1 (+ (-1X+-1) # (+xx++))) S KL(2), 1++1 ||
#0 (-
Lx(sp; X
Here the gradient of the upper bound is given by
V (xp: -E log Pi, (yl + ddigg(e [Z)] + +V VL(L 18+1 plyke 1- (-|x+++)))
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 MIDPOINT GUIDANCE POSTERIOR SAMPLING
1: Input: (4x)==1 with e and & 1; number M of gradient steps.
2: Xn (Oa, Ia), X, Xn
to do 4: Pe. - v.r (aa41///a)))
3: for k = n
(Xitt)) )+ + (1 Xk+1
1-0811
ullett
1-
(1-a++1 jan. ((-
5:
Pec - 2 log
1-ak+1
6:
for j 1 M do
7:
Z~ Z~N(0s, Ia)
(i,, Pa.) + OptimizerStep( Vo Z; Xx+1); Pex Pie)
8:
9:
end for
Zee Zk Lid. NOa: Ia)
10:
Xen t - + diag(e (cx 7Z,,
12:
X 1062,,t11 (0 (.18ex, Xk-1) (See (A.5))
end for
14:
Xo m
where Z ~ N(O,, Lạ) is independent of X The first term is dealt with using the reparameterization trick (Kingma& & Welling, 2014) and can approximated using a Monte Carlo estimator based on single sample Zk+1; we denote this estimate by Vi Zk+1; Xk+1). The second term is the gradient the KL divergence between two Gaussian distributions and can thus be computed in closed form. Similarly to the previous approaches of Ho et al. (2022); Chung et al. (2023); Song et al. (2023a), our gradient estimator involves a vector-Jacobios product of the denoising network.
We now provide summary of the MGPS algorithm, whose pseudocode is given in Algorithm 1. Given (=x) MGPS proceeds by simulating a Markov chain (X+) starting from Xn ~ (Oa, La). Recursively, given the state Xx+1, the state X is obtained by
1. minimizing Cg(3 (x-1) by performing M stochastic gradient steps using the gradient estimator V.Lx (: Xx+1). yielding parameter (Xx+1),
2. sampling Xe, P 11 where we drop the dependence on X in 4 and then X
Oe/fe. X
1alk4411 the actual law of Xee given XK-1 1 is not a Gaussian distribution as we need marginalize over that of 4h due to the randomness in the stochastic gradients.
Remark 3.3. While conditionally on stt we have that Xc. XP
well-chosen initialization for the Gaussian variational approximation parameters crucial for achieving accurate fitting with few optimization steps, ensuring the overall runtime of MGPS remains the previous step, competitive with existing methods. Given Xegss sampled from A 1 choose the initial parameter PP so that
X 1..+11x00 (x+) = Ge. (xg/= (Kiar)) Xx+1).
(3.9)
This initialization is motivated by noting that 90,10,A+1 i (m k+1 (X++1) 1), Xx+1), the DDPM approx- imation of ( (-IXx=1), where m a supposed denoiser for the posterior, a reasonable candidate for the initialization. As is intractable, we replace it with m f+:1 (fee..) ), our best cur- rent guess. Finally, in Appendix C.2 we devise warm-start approach that improves the initialization during the early steps of the algorithm. This has proved advantageous in challenging problems.
Related works. We now discuss existing works that have similarities with algorithm, MGPS. In essence, our method tries to reduce the approximation enror incurred by DPS-like approximations. This has also been the focus of many works in the literature, which we review below.
When p(yl-) 1) is likelihood associated to linear inverse problem with Gaussian noise, Finzi et al. (2023); Stevens et al. (2023); Boys al. (2023) leverage the fact that the Gaussian projection

---

Published as a conference paper at ICLR 2025
Rergle
Sangle
Sunple 3
Sanple
Oiaercation
Geveettrah
Sucell
Seople
Sangle a
Sengle
Figure 3: MGPS sample images for half mask (left), expand task, Gaussian blur and motion blur (right) on the ImageNet dataset.
of 40jo (+xx) minimizing the forward KL divergence can be approximated using the pre-trained denoisers; see Meng et al. (2021, Theorem 1). In the considered Gaussian setting, the likelihood p(yl-) can be exactly integrated against the estimated Gaussian projection. However, this involves a matrix inversion that may be prohibitively expensive. Boys et al. (2023) circumvent the latter by using a diagonal upproximation which involves the computation of d vector-Jacobian products. For general likelihoods p(yl.), Song et al. (2023b) use the Gaussian approximations of Ho et al. (2022); Song et al. (2023a) to estimate p' (yl.) for non-linear likelihoods p(yl-) using vanilla Monte Carlo estimate. Zhang et al. (2023) also consider the surrogate transitions (3.5) with l k, and, given an approximate sample Xx+1 from 4 estimate the next sample X5 maximizing x + T *klk+1 (x& (x&+1) using gradient ascent. They also consider improved estimates of the likelihood Pk: (yl-)) running a few diffusion steps. However, it is shown (Zhang et al., 2023, last subtable in Table 2) that this brings minor improvements at the expense of sharp increase in computational cost, which is mainly due to backpropagation over the denoisers. See also Appendix C.3 for discussion on how our method relates to the DPS (Chung al., 2023) in the case where Ea =
Finally, a second line of work considers the distribution path (ft))==m where *xx()) x p(y) Mojo (xx))q, for ls E [[0, n 1] and म. (0,,,, Ia). This path bridges the Gaussian distribution and the posterior of interest. Furthermore, if one is able to accurately sample from 4x+1, then these samples can be used initialize a sampler targeting the next distribution fT.- As these distributions are expected to be close, the sampling from fhe can also be expected to be accurate. Repeating this process yields approximate samples from π regardless of the approximation error in the likelihoods. This approach is pursued by Rozet & Louppe (2023), who combines the update (2.5) with Langevin dynamies targeting 7t.- Wu et al. (2023) use instead sequential Monte Carlo to recursively build empirical approximations of each Ry. by evolving sample of N particles.
4 EXPERIMENTS
We now evaluate our algorithm on three different problems and compare it with several competitors. We begin in benchmarking our method on toy Gaussian-mixtur targets and image experiments with both pixel space and latent diffusion models. We review the latter in Appendix A.2. Finally, we perform inpainting experiments on ECG data. For experiments based on pixel-space diffusion models, MGPS is benchmarked against state-of-the art methods in the literature: DPS (Chung et al., 2023), PGDM (Song et al., 2023a), DDNM (Wang et al., 2023), DIFFPIR (Zhu et al., 2023), and REDDIFF (Mardani al., 2024). Regarding experiments using latent diffusion models, we compare against PSLD (Rout et al., 2024) and RESAMPLE (Song et al., 2024). Full details on the hyperparameters can be found in Appendix D.1. Our code for reproducing all the experiments is publicly !vailable.'
Gaussian mixture. We first evaluate the accuracy of our method and the impact of the hyperparam- eters on toy linear inverse problem with Gaussian mixture (GM) as q and Gaussian likelihood p(y(x) := Ny: Ax, La where A E Rd xd and σy = 0.05. We repeat the experiment of Cardoso et al. (2023, App B.3). where the prior is a GM with 25 well-separated strata. For this specific example, both the posterior and the denoiser moj& are available in closed form. In particular, the posterior can be shown to GM. The full details are provided Appendix D.3. We consider the settings (20, 1) and (200, 1) for the dimensions (d, dy). For each method, we reconstruct 1000
Code available https: : / /ttpttttbb//////////// com/ hamm/yzziaa i jjanat /mgps
7

---

Published as a conference paper at ICLR 2025
200
Table 1: 95 % confidence interval for the SW the GM experiment.
7.5-
SW(m. 2to))
5.0
d= 20, dy
200, dy =
MGPS
2.11 + 0.30
3.66
0.53
2.5
DPS
8.93 0.49
9.15 + 0.41
PGDM
2.44-.30
5.23 1 0.38
0.6
DDNM
4.24
7)
#Gradient steps
DIFPPIR
4.14
8.43 + 0.92
Figure 4: Left: SW function of y with & Innc- Right: SW as a function of the number of gradient steps, for specific choice of (fx)
REDDIFT
6.70
8.35 + 0.39
samples and then compute the sliced Wasserstein (SW) distance exact samples from the posterior distribution. We repeat this procedure 100 times with randomly drawn matrices A, and consider the resulting averaged SW distance. In every replication, the observation is generated Y AX oy Z where Z ~ N (0dy Ia,) and X ~ q. The results are reported in Table 1. It can be observed that MGPS with r7 0.75 outperforms all baselines in both settings. Although we have tuned the parameters of DPS it still exhibits considerable instability and often diverges. To account for these instabilities we set the SW to when diverges.
Ablations. Next, we perform an ablation study on the total number of gradient steps and the choice of the sequence ((=) The right-hand plot in Figure 4 shows the average SW distance as function of the number of gradient steps per denoising step when using é = |3k/4J. For d = 20, we observe a monotonic decrease of the SW distance. For d = 200, the SW distance decreases and then stabilizes after 10 gradient steps, indicating that in this case, MGPS performs well without requiring many additional gradient steps. We also report the average SW as a function of 17 E [0, 1] in the left-hand plot of Figure 4, following the same sequence choices in Example 3.2, i.e,, f [np]]. In both
settings, the best SW is achieved at v = 0.75. In dimension d = 200, the SW at = 0.75 is nearly twice good as at 1) 1, which corresponds a DPS-like approximation. This demonstrates that the introduced trade-off leads to non-negligeable performance gains.
Images. We evaluate our algorithm on a wide range of linear and nonlinear inverse problems with noise level σy 0.05. As linear problems we consider: image inpainting with a box mask of shape 150 x 150 covering the center of the image and half mask covering its right-hand side; image Super Resolution (SR) with factors x4 and 16; image deblurring with Gaussian blur; motion blur with 61 x 61 kernel size. We use the same experimental setup as Chung et al. (2023, Section 4) for the last two tasks. Regarding the nonlinear inverse problems on which we benchmark our algorithm, we consider: JPEG dequantization with quality factor 2%; phase retrieval with oversampling factor 2; non-uniform deblurring emulated via the forward model of Tran et al. (2021), 1); high dynamic range (HDR) following Mardani et al. (2024, Section 5.2). Since the JPEG operator is not differentiable. we instead use the differentiable JPEG framework from Shin & Song (2017). We note that only DPS and REDDIFF apply nonlinear problems.
Datasets and evaluation. We test our algorithm on the 256 x 256 versions of the FF#Q (Karras al., 2019) and ImageNet (Deng et al., 2009) datasets using publicly available pre-trained DDMs. For FFHQ, we use the pixel-space DDM of Choi et al. (2021) and the LDM of Rombach et al. (2022) with the VQ4 autoencoder. For ImageNet, to computational constraints, we first evaluate MGPS and competitors on all tasks using 50 random images per dataset. We then focus on the 5 most challenging tasks, evaluating MGPS and the top competitors on Ik images in Table 7. For all tasks, we report the LPIPS (Zhang et al., 2018) between the reference image and the reconstruction, averaged over 50 images. For the five tasks tested on 1k images, we also report the FID in Table 7. Although pixel-wise metrics like PSNR and SSIM are less informative for multimodal posterior distributions, they are available in Tables 7 to 10. For all the considered competitors, we use the hyperparameters proposed in their official implementations available, otherwise we manually tune them to achieve the best reconstructions (implementation details in Appendix D.1). For phase retrieval, due the task's inherent instability, we follow the approach of Chung et al. (2023) by selecting the best reconstruction out of four, further discussion on this can be found in Chung et al. (2023, Appendix C.6).
use the model by Dhariwal & Nichol (2021). Due
8

---

Published as a conference paper at ICLR 2025
Table Mean LPIPS for various linear and nonlinear imaging tasks on the FFHQ and Tma Net 256 x 256 datasets with By 0.05. Lower metrics are better.
FFRQ
ImageNat
Task
PGDM
DDNM
DIFFPIR
REDDIFF
DPS
SR (x:4)
0.09
0.09
0:33
0.14
0.13
0.36
0.30
0.41
0.78
0.34
0.36
0.56
SR (Rx(xx
0.26
0.24
0.44
0.30
0.28
0.51
0.53
0.50
0.60
0.70
0.63
0.83
Box inpaining
0.10
0.17
0.12
0.22
0:34
0.29
0.28
Hall mask
0.20
0.24
0.26
0.22
0.23
0.28
0.29
0.44
0.38
0.38
0.35
0.44
Gaussian Deblut
0.15
0.16
0.87
0.19
0.12
0.23
0.32
0.35
1.00
0.45
0.29
0.54
Motion Deblur
0,13
0.16
0.21
0.22
0.39
0.40
(QF
L.0
0.32
0.63
Phace retrival
0.11
0.46
0.25
0.47
0.62
0.60
Nonlinear dehlur
0.23
D.52
0.66
11,4
0.38
0.77
High dynamic runge
0.07
0.85
Results. We report the results in Table for FFHQ and ImageNet with DDM prior, and Table for FFHQ with LDM prior. These reveal that MGPS consistently outperforms the competing algorithms across both linear and nonlinear problems. Notably, on nonlinear tasks, MGPS achieves up to twofold reduction in LPIPS compared to the other methods with DDM prior. It effectively manages the additional nonlincarities introduced by using LDMs and surpasses the tfae-off-the-art as demonstrated in Table 3. Reconstructions obtained with MGPS are displayed in Figure 3 and Figure Further examples and comparisons with other competitors are provided in Appendix D.6. seen that MGPS provides high quality reconstructions even for the most challenging examples. For the DDM prior, the results are obtained using the warm-start approach, see Algorithm 3, and setting the
la [k/2] on all the tasks based on the FFHO dataset. As for the ImageNet dataset, same configuration on all the tasks, except for Gaussian deblur and motion deblur. For these tasks we found that using lg [ 1x2 [n/2| [=/2) improves performance. We use similar strategy for the LDM prior. In Appendix D.2.1, we measure the runtime and GPU memory requirements for each algorithm. The memory requirement of our algorithm is the same that of DPS and PGDM, but the runtime of MGPS with 300 is slightly larger. With fewer diffusion steps, the runtime halved compared to DPS and PGDM and comparable that of DIFFPIR, DDNM and REDDIFF. Still, MGPS remains competitive and consistently outperforms the baselines, particularly on nonlinear tasks. See Table 8 and for detailed results using n E (50, 100} diffusion steps.
Obscaation
Goandtruh
Sumple
Table 3: Mean LPIPS with LDM FFHQ.
Lower metrics are better.
Task
MGPS
RESAMPLE
PSLD
Morion derber
(x4)
SR (x
0.30
0.36
0.35
Box impuinting
0.16
0.22
0.26
Half mask
0.25
0.30
0.31
Caussian Deblur
maaic
JPEG (QF - 2)
0.20
0.26
Phose retrieval
0.34
0.41
Nonlinear deblur
0.26
High dynamic range
Figure 5: MGPS samples with LDM on FFHC dataset.
ECG. We now explore posterior sampling algorithms beyond image tasks show their versatility and public health impact. Cardiovascular discases cause one-third of global deaths, and better detection can improve management. Wearables like smartwatches can enhance diagnosis by capturing brief symptom episodes, particularly for conditions like paroxysmal atrial fibrillation (AF), which may go undetected during routine medical visits. However, they offer only a partial ECG view (Lead I instead of 12 leads) and recent study showed the Apple Watch detected AF in only 34 of 90 episodes (Seshadri et al., 2020). To address this limitation, we propose using posterior sampling algorithms to reconstruct incomplete electrocardiograms (ECG). An ECG is an electrical recording of the heart's activity in which the signals generated by the heartbeats are recorded in order to diagnose various cardiac conditions such as cardiac arrhythmias and conduction abnormalities. Unlike static images, ECGs constitute complex sene-series data consisting of 12 electrical signals acquired using 10 electrodes, 4 of which attached to the limbs and record the 'limb leads', while the others record the pprecordial leads' around the heart. We study two conditional generation problems in ECGs. The first forecasting missing-block (MB) reconstruction problem, where half of the ECG is

---

Published as a conference paper at ICLR 2025
reconstructed from the other; see fig. 8. This task evaluates the algorithm's ability to capture temporal information to predict a coherent signal. The second problem is an inpainting or missing-leads (ML) reconstruction, where the entire ECG is reconstructed from the lead I: see fig. 6. The question is whether we can capture the subtle information contained in lead and reconstruct a coherent ECG with the same diagnosis the real ECG. This task is challenging because lead I, being acquired with limb electrodes far from the heart, may contain very subtle features related to specific cardiac conditions. We train state-space diffusion model (Goel al., 2022) generate ECGs using the 20k training ECGs from the XLB dataset ( (Wagner et al., 2020), and benchmark the posterior sampling algorithm on the 2k test ECGs; see appendix D.5. We report the Mean Absolute Error (MAE) and the Root Mean Squared Error (RMSE) between ground-truth and reconstructions in Table 4. We demonstrate that a diffusion model trained to generate ECGs can serve as a prior to solve imputation tasks without additional training or fine-tuning, yielding superior results to a diffusion model trained conditionally on the MB task (Alcaraz & Strodthoff, 2022). The rationale for this result is discussed in Appendix D.5.2. We report in Table 5 the balanced accuracy of diagnosing Left Bundle Branch Block (LBBB), Right Bundle Branch Block (RBBB), Atrial Fibrillation (AF), and Sinus Bradycardia (SB) using the downstream classifier proposed in Strodthoff al. (2020) (see appendix D.5.1) applied to both ground-truth and to reconstructed samples from lead I. See appendix D.5.2 for sample figures. MGPS outperforms all other posterior sampling algorithms with just 50 diffusion steps.
Table 4: MAE and RMSE for missing block task on the PTB-XL dataset.
Metrie
MGPS
DPS
PGDM
DDNM
DIPePIR
REDDIN
TRAINEDDUY
MAE
0.111 3--3
0.117 + 4e-3
0,118 + 2e-3
0.103-
2--3
0,115 + 2--3
0.171 + 3e-3
0.116 + 2e-3
RMSE
0.225 4e-3
0.232 + 4e-3
0.233 4e-3
0.224
4e-3
0.233 4e-3
0.287 Se-3
0.266 3e-3
aVR
VO
Table 5:
Balanced acc. downstream diag- nosis from ECG reconstructed from lead I.
aVF
Method
RBBB
LBBB
AF
SB
11I
aVL
V2
v5
MGPS
0.81
0.92
0.94
0.66
MGPS
0.90
0.93
0.92
0.66
DPS
0.54
0.84
0.79
0.50
PGDM
0.65
0.87
0.88
0.55
DDNM
0.71
0,83
0.86
0.59
DIRFPIR
0.57
0.80
0.77
0.53
REDDIFF
0.73
0.86
0.88
0.60
Figure 6: 10s ECG reconstruction from lead I. Ground truth in blue, 10%-90% quantile range in green, random sample in orange.
Grund-truth
0.99
0.98
0.94
0.70
5 CONCLUSION
We have introduced MGPS, a novel posterior sampling algorithm designed to solve general inverse problems using both diffusion models and latent diffusion models as prior. Our approach is based on trading off the approximation error in the prior backward dynamics for a more accurate approximation of the guidance term. This strategy has proven to be effective in variety of numerical experiments across various tasks. The results show that MGPS consistently performs competitively and often even superior to state-of-the-art methods.
Limitations and future directions. Although the proposed method is promising, it has certain limitations that invite further exploration. First, a detailed analysis of how the algorithm' 's performance depends on the choice of the intermediate time steps (f4))=) remains a challenging but critical subject for future research. Although our image and ECG experiments suggest that setting l = [k/2] leads to significant performance gains on most tasks, we have observed that using an adaptive sequence 4 = [nn J. where 7/,- increases as l decreases, further enhances results on certain tasks, such as Gaussian and motion deblurring, as well as when using latent diffusion models. A second direction for improvement lies in refining the approximation of p (yl-) which we believe could lead to overall algorithmic improvements. Specifically, devising approximation &% (yl-) such that V log pf. (yl-) does not require a vector-Jacobian product could significantly reduce the algorithm's 's runtime. Lastly, see potential in using the two-stage approximation introduced in warm-start strategy (see Algorithm 3) at every diffusion step. Although this technique is promising, it currently leads to instabilities l decreases. Understanding and resolving these instabilities is also of key interest.
10

---

Published as a conference paper at ICLR 2025
Acknowledgements. The work is supported by the Swedish Research Council, project 2024-05680. The work of Eric Moulines has been partly funded by the European Union LERC-2022-SGG-OCEA. 101071601). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. We would like to thank IHU-LIRYC for the computing power made available to us. The work of Y.J. and B.M. has been supported by Technology Innovation Institute (TII), project Fed2Learn.
REFERENCES
Juan Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. Transactions Machine Learning Research, 2022. ISSN 2835-8856. URL https:: /ttp////wwww.eee/eeeeeeaeeeeeeee
7ApW.
Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based conditional ecg generation with structured state space models. Computers Biology and Medicine, 163: 10:10001,, 2023. ISSN 0010-4825. doi: http:://bborry//llooooooccommooo
URL https: //www.
sciencedirect. wwwccc//ccccoooe htt.a///cccooaaaa.e.e//////////0
Georgios Batzolis, Jan Stanczuk, Carola-Bibiane Schönlieb, and Christian Etmann. Conditional image generation with score-based diffusion models. arXiv preprint 1r 1316606, 2021.
J.E. Besag, Comments on "Representations of knowledge in complex systems" by U. Grenander and M. Miller. J. Roy. Statist. Soc. Ser. B, 56:591-592 1994.
Christopher M. Bishop. Pullern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg, 2006. ISBN 0387310738.
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and O Deniz Aky- ildiz. Tweedie moment projected diffusions for inverse problems. arXiv preprini arXiv:2310.06721, 2023.
Gabriel Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte Carlo guided diffusion for Bayesian linear inverse problems. arXiv preprint 2xXi:2330..07988 2023.
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Condi- tioning method for denoising diffusion probabilistic models. in 2021 ieee. In CVF international conference on computer vision (ICCV), volume 1, pp. 2, 2021.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mecann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023. URL https: /oop//peeeeeeeeee/ee/eeeeeeeeeee net/ forum?
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-877 8794, 2021.
Mare Anton Finzi, Anudhyan Boral, Andrew Gordon Wilson, Fei Sha, and Leonardo Zepeda-Núñezz User-defined event sampling and uncertainty quantification in diffusion models for physical dynamical systems. In International Conference on Machine Learning, pp. 10136-10152. PMLR, 2023.
Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with state-space models. International Conference on Machine Learning (ICML), 2022.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In The International Conference on Learning Representations (ICI.), 2022.
11

---

Published as a conference paper at ICLR 2025
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6883 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646, 2022.
Jérôme Idier. Bayesian approach to inverse problems. John Wiley Sons, 2013.
Chiyu Jiang, Andre Cormman, Cheolho Park, Benjamin Sapp, Yin Zhou, Dragomir Anguelov, et al. Motiondiffuser: Controllable multi-agent motion prediction using diffusion. Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9644-9553, 2023.
Zahra Kadkhodaie and Eero P Simoncelli. Solving linear inverse problems using the prior implicit in a denoiser. arXiv preprint arXiv:2007.13640 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4411 2019.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in Neural Information Processing Systems, 011226655-667777 2022.
Bahjat Kawar, Gregory Vaksman, and Michael Elad. Snips: Solving noisy inverse problems stochas- tically. Advances in Neural Information Processing Systems, 34:21757-21768 2021.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593-23606 2022.
Diederik Kingma and Max Welling. Auto-cncoding variational Bayes. In International Conference Learning Representations, 2014.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In The 3rd International Conference for Learning Representations, San Diego, 2015, 2015.
Anji Liu, Mathias Niepert, and Guy Van den Broeck. Image inpainting via tractable steering of 2023.
diffusion models. arXiv preprint
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11461-11471, 2022.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. variational perspective on solving inverse problems with diffusion models. In The Twelfth International Conference Learning Representations, 2024. URL https: /pwprevieeeeeeeeeeeeeeeeeeeeeeee
Chenlin Meng. Yang Song, Wenzhe Li, and Stefano Ermon. Estimating high order gradients of the data distribution by denoising. Advances in Neural Information Processing Systems, 34: 25359-255 25369, 2021.
R.M. Neal. MCMC using Hamiltonian dynamics. Handbook of Markov chain Monte Carlo, 2011.
Ingram Olkin and Friedrich Pukelsheim. The distance between two random vectors with given dispersion matrices. Linear Algebra and its Applications, 48:257-263, 1982.
Herbert E. Robbins. An empirical bayes approach to statistics. In Proceedings of the Third Berkeley Symposium Mathematical Statistics and Probability, Volume 1: Contributions the Theory of Statistics, 1956. URL https: api. semant escholar. org/ wttg/orrppeaooooooooooeee (CopppsI::220. 22:6166640 6188144411 1.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 10684-10693 10695, 2022.
12

---

Published as a conference paper at ICLR 2025
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
François Rozet and Gilles Louppe. Score-based data assimilation. Advances in Neural Information Processing Systems, 36:40521 2023.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713-4726, 2022.
Dhruv R. Seshadri, Barbara Bittel, Dalton Browsky, Penny Houghtaling, Colin K. Drum- mond, Milind Y. Desai, and A. Mare Gillinov. Accuracy of apple watch for de- tection of atrial fibrillation. Circulation, 141(8):702-703, 2020. doi: 10.1161/ CIRCULATIONAHA.II 19404 URL https: htpps://wwww.t http///w.aaaaaaa www.aaaajournals.rrrr..oo org/ /ab///s//////////////// 10.1611CIRCULLTTONNAA 119, 044126.
Richard Shin and Dawn Song. Jpeg-resistant adversarial images. In NIPS 2017 workshop machine learning and computer security, volume pp. 8, 2017.
Jascha Sohl-Dicke Dicksteein Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamies. In International Conference Machine Learning, pp. 2256-2265. PMLR, 2015.
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. In The Twelfth International Conference on Learning Representations, 2024. URL https: httpt:tst//p/eeeeeeee /oppereeeeveveeeeeeeeeeeeeveeeee ew net/ forum? jddjBhdRQOUhN
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2021a. URL httppp::///ppeeeee...... net/
Corum?id=SSe giillliiaarr....
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guidee diffusion models for inverse problems. In International Conference on Learning Representations, 2023a.
URL www.thtpp////......m /ttp::////ppereiieew net wwt.eeeeeeeeetteteeeeeeeeeeee 9 Baaiiddd__aaMMM_-- SMRKK.
Jiaming Song. Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning, pp. 32483-32222 (283--2448.. PMLR, 2023b.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021b.
Tristan SW Stevens. Hans van Gorp, Faik C Meral, Junseob Shin, Jason Yu, Jean-Luc Robert, and Ruud JG van Sloun. Removing structured noise with diffusion models. arXiv preprint arXiv:2302.052900 2023.
Nils Strodthoff, Patrick Wagner, Tobias Schaeffter, and Wojeiech Samek. Deep learning for eeg analysis: Benchmarks and insights from ptb-xl. arXiv preprint 2004. 13701, 2020.
Andrew M Stuart. Inverse problems: Bayesian perspective. Acta numerica, 19:451-559, 2010.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffu- sion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804-244 16, 2021.
Phong Tran. Anh Tuan Tran, Quynh Phung, and Minh Hoai. Explore image deblurring via encoded blur kernel space. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1956-1 1965, 2021.
13

---

Published as a conference paper at ICLR 2025
Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif- scaffolding problem. In The Eleventh International Conference Learning Representations, 2023.
URL https:/ /wwpneeeeveeeeeeeeeeeeeeeeeeeeee eew net / forum? i d= (d=xXxxxMMM
Tim Van Erven and Peter Harremos. Rényi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory, 60(7):377-33800, 2014.
Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima Lunze, Wojeiech Samek, and Tobias Schaeffter. Ptb-xl, large publicly available electrocardiography dataset. Scientific Data, 7:154, 05 2020, doi:
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In The Eleventh International Conference Learning Representations, 2023.
URL settps://openreweee
net forum? id=mRi -ee-mmiieeMMNT
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew Borst, Robert Ragotte, Lukas Milles, et De novo design of protein structure and function with rfdiffusion. Nature, (2097976):1089-1110 2023.
Luhuan Wu, Brian L. Trippe, Christian A Naesseth, John Patrick Cunningham, and David Blei. Practical and asymptotically exact conditional sampling in diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL htttps://openrevieet dddeenggllaeee ZeRv.
Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, and Shiyu Chang. Towards coherent image inpainting using denoising diffusion implicit models. In International Conference on Machine Learning, pp. 41 41193. PMLR, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration. In Proceedings of the IEFF/CVE Conference on Computer Vision and Pattern Recognition, pp. 1219-1229, 2023.
14

---

Published as a conference paper at ICLR 2025
A BACKGROUND ON DENOISING DIFFUSION MODELS
A.1 DENOISING DIFFUSION PROBABILISTIC MODELS
In this section, we provide further background on DDMs based on the DDPM framework (Ho et al., 2020; Dhariwal & Nichol, 2021; Song et al., 2021a). We rely on definitions provided the main text.
DDPMs define generative models for y relying only parametric approximations (m of the denoisers (mo These approximate denoisers are usually defined through the parameterization
m6j (x+) = (x, vI orey (x+))/
(A.1)
and trained by minimizing the denoising loss
T
Σ
W [||x - <V + vI - oxEx)]1
(A.2)
w.r.t. the neural network parameter 8, where (G)T_ are i.idd. standard normal vectors, X, q, and (w:)T= are some nonnegative weights. Having trained the denoisers, the generative model for is defined follows. Let (4x (4x)==o be increasing sequence of time steps in [0,7]] with to 0. We assume that in is large enough so that 4t,, (xr,. density of multivariate standard normal distribution. For convenience, we assign the index & to any quantity depending on t&; e.g., we denote 9t, by 9д- Consider the backward decomposition
((xo) Xt,, xxo) dxn is approximately the Jo.
n-1
12
Jown (Xo:n) q(xo) II 1K+ (x 9n (xx)
II
9:l0+1 1 (x xx++)
of the forward process initialized at q, where gkjk (Xx (26+1) oc qu (xx))x 11k (xK+1 (x)). Next, for (j. k.) E [[, n such that j < < k. define
(X), /or/a;( -as x, /
as/a))
- Xb.
(A.3)
1 1 -aa/aa
(1 at a (1 aaa/aa ax/ass)
vejjk
(A.4)
1 -an/aj
Then the bridge kernel is
Aejj. (X (x, Xx) = 4el (xxxxx))) (xx(x;)
= =x(x;; mejjA (xj. xx). uejj
(A.5)
Using the bridge kernel, lonn is approximated using the variational approximation
Po.. (Xu.)) =p (xx.)
II
PhIK+1 (x+1),
k=
where pg (xolx1) - (xo: m (x1), Noj2 Ia), 2oj1 being a tunable parameter, and
Ph|--1 (xx (x++1) == 4.j, (xx m (x++1),
E [1, n- 1].
(A.6)
When = T', the denoising objective (A.2) corresponds the KL divergence KL pb..) for specific choice of weights (- In practice, a DDPM is trained using the objective (A.2) with large but inference is usually much smaller.
A.2 LATENT DIFFUSION MODELS
Latent diffusion models (LDM) (Rombach et al., 2022) define DDM in latent space. Let E: Rdd + R be an encoder function and Rp - R a decoder function. We assume that these functions satisfy DjE,g & g, where, for instance, &gg denotes the law of the random variable E(X) where X q. A LDM approximating g is given by D.pg. where r is diffusion model trained on
15

---

Published as a conference paper at ICLR 2025
samples from Egg. Finally, when solving inverse problems with LDMs, we assume that the target distribution is instead
T(x) p(y)x)))(,g(xx).
Let X ~ n, D ~ DjEng. E ~ ~ Egg. and Z ~ म. where #(z) x p(y(D:(2))) For any bounded function f on IRdd we have, following the definition of æ, that
B|((X) Epp(((D)((D) EIo(yP(E)(((()) (L((D( Elp(y|((E))
(yp(y)D)
Hence (Xw = Law(D(Z)) As result, to sample approximately from n, we first sample approximately from ग using any diffusion posterior sampling algorithm with pre-trained DDM for Egg, then decode the obtained samples using D.
A.3 MIDPOINT DECOMPOSITION
Before we proceed with the midpoint decomposition, we first recall that under the joint distribution obtained by initializing the posterior with the forward process (see (3.1)), it holds that for all i
म. (xx)))((( (x,] = " (x,) ) (xxlx)),
(A.7)
where TH u (x,(xx)) == mi (x 1y X / m (xy) and integrates to one.
Proof of Lemma 3.1. Let k) E [0, n1] be such that << k. Applying repeatedly the definition of the bridge kernel (A.5) and the identity (A.7),
Taje (xjr(Xx xx))x=+ (X
9k|6,++ (X5 Xe, xx+1)
9k+1je (xc)
Ty xx(xx)gn(((Xx) xe Dh+ (X+++ xc)
(x( Mh++1 (x++ (xe)
"Tej (xx ( 1 (x jsl X k+1 (X8+1)
IT 1 (xe) x Th+ (x++1)
T Teje (xx/) Xx Ao xx(x)+11
IT T2p4+11 (2ce xc++1)
then follows that
f t Tejk (xe xxx )T Rp (xx Xx++ 1) dxe
kg (xx/XK-1)
= f e, 1 (xx be, x Tej-tt (xel X dxe.
B THE GAUSSIAN CASE
B. DERIVATION
In this section we derive the recursions verified by the first and second moments of the marginal distribution af the surrogate model (3.6) in the simplified setting of Example 3.2. We recall
that in this specific example assume that q N(m, Σ) where (m, Σ) E I2 x S and p(yl-)= + Ny;; Ax, og
Denoiser and DDPM transitions. Since we are dealing with a Gaussian prior, the denoiser majk can be computed in closed form for any l [1, Using Bishop (2006, Eqn. 2. 2.), we have that (xo (x+) x (x+(x0)
= =N((xo: Eojk Eook((/anll (Van/vk) >x + E-' -m), Eoje),
16

---

Published as a conference paper at ICLR 2025
where Zoje ((ax/u4)1 + Hence,
mx)o((x)) Σoju ((j/((/u//v /x/// E 1m)
and we assume in the remainder of this section that m = Majk- From the expression of goja (-12%) we can immediately derive the more general backward transitions (=jx(-(x+) for E [1, 1] by noting that
4etk (xp =
4ejo, (x₂) Xo, X2 goja (xp xx. dxo-
From this, (A.3), (A.4), and the law of total expectation and covariance, follows that getu (exx)(((xs) N(mqu (x+), Eg (xx)). where
ar ax//ae22 (1 ana)²
mxj((x)) mejo, (xx),,)) Egit
Dule veju,ela.
On the other hand, the DDPM transitions are pe xx)) aejo,ee (xx), xx) N(mej,, (mo)) xx), velo, which shows that in this case, the true transitions and approximate ones differ only by their covariance.
(B.1)
Moments recursion. For [[0, n]. we let o denote the x₃ marginal of the surrogate model (3.6). We remind the reader that # = NOa,, Ia). The marginals satisfy the recursion
(xxx) - f #/ # (xxl xx+ ) (x++ dx E [0, 1J.
Since (y(x*)) = Ny; A (xx), La and m is linear in it is easily seen that #0 Tttlk+1 (X8+1) the density of a Gaussian distribution. Consequently, by definition (3.5) and the definition (A.5) of the bridge kernel, this is also the case for of Tk (-Ix+11.. Now assume that
* = -x(x+11; 1
(B.2)
Mk|-+1 (x((xx+1)= -N(x;; M X +
CxR+1 E 11p4+))
(B.3)
where M kjk+1 Idd E k1k42 E Stt and ck4t2 E Rdd Using the definition (A.5) of the bridge kernel we find that of = N(jg, Ex). where
3 Ma CjR+II
E M4 Ex MT R//041 + S
2Mj++1
Iterating these updates until reaching à: O, starting from the initialization j = 02 and Σ = Ia. yields the desired moments of the surrogate posterior πh- It now remains to show that the backward transition # ( (-1x++1) writes in the form (B.3) and identify M kjk41 and of 4//41
First, we write the approximate likelihood in the form
p)(((xx) = Ny Aexi + DR, La,))
where
A = (Vax/un) A b Ao
We also denote by m (xx) the mean of the Gaussian distribution with density given by the DDPM transition p in (B.1). We have that
m & (xx+1) = Hé Xx+1 + heglk+1,
where
ae2 ax41/aae)) Eojk + V /at41 - ore,)
Id
(B.4)
Vc. Uk+1
Uk+1
Vare (1 - ax41 / Eojo '-'m.
heg/411
(B.5)
Uk
17

---

Published as a conference paper at ICLR 2025
Then, applying (Bishop, 2006, Eqn. 2.116), get
# fk|
(xL, xx-1) = (x M ++ X6+1 + Fellttt))
where
T 14k++1 = (1610,,411 I oyy A Act)
M 4/k++ = vaj0,k+1 Tictt-Tiii Aulh+1
E2aktt1 = DeujR+1|e loy AT, (y be.) + Usa 0, hes [et1
Finally, following (3.5) we find that
M kjk+1 Var/ar, (1 ax+1/ak)
/ax+1/au ///l///111111 - ax/ace) Id
M 8el-11
1 /
1 akt1/0ee
G/pt11 Vak/aa, ae, (1 014+1//s))
01E1//0EN
C2tlk+1
B.2 EXPERIMENTAL SETUP
In the experiment described in Example 3.2, we set d 100 and generate 500 instances of inverse problems (A, m. Σ). For each instance, we compute the Wasserstein-2 distance between the resulting posterior distribution π and A
Prior. The mean of the prior is sampled from a standard Gaussian distribution. To generate the covariance Σ St+,, we first draw matrix G E Rdxd with i.i.d. entries sampled from N(0, 1). Then, for better conditioning Σ, we normalize the columns of G and set E X + GG where X2 is the mean of the squared singular values of G.
Likelihood. To sample an ill-posed problem, we generate rank-deficient matrix A Rd xd with dy I d. We sample uniformly dy from the interval [d/10, d]] and draw the entries of A i.i.d. from ((0, 1). Regarding Jy, we sample uniformly from the interval 0.5).
Finally, the resulting posterior is also Gaussian (Bishop, 2006, Eqn. 2.116)
(xx & Ply()()((xx x mx:
my. Ey),
where Ey = - (1/a²)A -1 and my Ey (11/3)) ) T " mm). The Wasserstein-2 distance between the true posterior and Ao is thus (Olkin Pukelsheim, 1982)
W2(=,,0f3 = ||my مbll + tr (E, + E - 2 2
C MORE DETAILS ON MGPS
C.1 DETAILS ON THE LOSS
First, by the Data Processing incquality (Van Erven & Harremos, 2014, Example 2),
KL KL 2//-+
(-1x++1) #f
# 1 (x++))
$ f log 9M/ (x Xe, Xk+1 be (Xtx.
Akles, (xx (Xe+ Xx+ 1) (Xex) dx
Oklee,841 (Xx Nle Xx++ A
1o+1 (Xes (x+41
1/0 1 (Xtx)
log # [k++1 (xt, (x(, (x(+1) OM162 (X
X 10
E (Xex. dx
KL(A) ( - 12,11+11 (xx+1))
The gradient of so + KL 1.|++1 # (-1x)+1)) for given X1+11 writes
V LuLu(yp Xet1)
= =EE log pl, (yl + diag(e )Z)] + VorK(A
ll (1 (C.1)
18

---

Published as a conference paper at ICLR 2025
and the second term can be expressed in closed form since both distributions are Gaussians, i.e.,
Ea/k+1 VicK((A
1l 11 1-1x(((--+++1)))
lase - ( (X6+1): xx+1)|12 + Ej=1 e Peg
V
Pau +
12,00,+11
208,10,847
C.2 WARM START
In this section we describe the warm-start approach discussed the main paper. The complete algorithm with the warm-start procedure is given in Algorithm 3. The original version presented in Algorithm relies on first sampling approximately Xčx, given Xk+1, from the surrogate transition # + (+1x and then sampling X5 from (k| (+I .x+1). X In our warm-start approach, which we apply only during the first iterations of the algorithm (see the hyperparameter settings in Table 6), draw inspiration from the decomposition
f 9:|1,A+1 1 Xx- 1ie,
T (xxxx++1))
1l6, (x1 (e,,
k+1 (Xe, (x++1 dxy dxx,, (C.2)
It suggests introducing a second intermediary step that involves sampling approximately from the transition X1 ~ 1ie, (-) X5 is then sampled from the bridge ( Xx+1).
In order to draw approximate samples from 51j6, (+)Xa,) we leverage again a Gaussian variational approximation Mip, : N( diag(e"P:)); which we fit by minimizing proxy of the KL divergence between A and 11/4 (+)Xe.) of which the gradient w.r.t v == (p1, P1) writes
d
1|xee /11) 112 + (01e/a1) Σラ=1 eRi.
V TyLl 114 (V: Xex) == -V
Σ
PI.j
2(1 ass/a1)
1000
V (ūn + diag(eP: )Z) (V, logp(yl.) (i + (iag(e )Z (80) (C.3) E
This expression corresponds to the gradient w.r.t. & of KL || 1ije (-1Xa,)) )) combined with
the score approximation sf of Va log d and the mild likelihood approximation V. (ogpi(y))) of V. log p(yl-), .. Indeed, we have that
KL 114 || (- -EAG [Iog))(y(Xx)) log 91 (Xi)] *
+ f
log
qea11 (Xe [x1] Miee. (xI) dx
ilfe
(x1)
The second term can be computed in a closed form and its gradient corresponds to the first term in (C.3). As for the first term, under standard differentiability assumptions and applying the raparame- terization trick, we find that
V B (ogg((yl/x,) + log (X1)] woo
V (yg-)(y)))) log q (iz + diag(e )2)] 1000
E
V Ve diag(e TZ), Lo (vll)) Va log (i,, + (iag(eP1)) ) (w)
Plugging the previous approximations yields the second term in (C.3). The warm-start procedure is summarized in Algorithm 2. We also use single sample Monte Carlo estimate to perform the optimization. Finally, at step ls and given Xes the initial parameter th, of the variational approximation is chosen so that
i1e (x1) 91j0.6, (xi Im (Xee) XEe.)
19

---

Published as a conference paper at ICLR 2025
Algorithm 2 WarmStart
1: Input: step k; samples (Xex, xxt1) gradient steps M
2: fi < m0.6. o (Xta). Xc,), PI + 2 log 2110.4
3: for j to M do
4: -~ N(O,, Ia)
2an 5: (i1z P1) OptimizerStep( V Ci (s Xox): P1:P2)
6: end for
7:
X1 11 + diag(eP:) where 71 ~ N(Oa, I2)
8: X5 f- mx Xk+1) 1|11,++1 2x
1/2
9: Output: Xx, 81
Algorithm MGPS with warm start strategy
1: Input: (Cx)i=n with en n, l gradient steps (Mx (M warm start threshold w.
2: Xn NOa, La), Xn Xn
3: Xy f m (X,)
4: for k 1 to do
5: fr. t mr m,+ XR+I); Pi. + 2 log ve, 10,4+1
6: for 1 MR do
7:
7~ N(Os, La)
8:
(Aee Pe.) + OptimizerStep(( V (. Xu+1)i Pec Pt.)
9:
end for
Zen, 7k Lid N(Oa, I2)
10:
11:
Nex + jr, + diag(e ZZ,
12:
A 2 w then
(Xx, X1) WarmStart(k: Xtx, Xk+1, M&)
14:
Xo xx
X1
15:
else
Xx + - (Ree, Xx+1) + 1/2
16:
kkl Zz
17:
Xo < mbee (Xee)
18:
end if
19:
end for
20:
Output: Xo
After having sampled X1
Aijk use it to first sample X₃ (- Xx+1) and then
initialize the next variational approximation A/ 1/k" The initial parameter 4px-1 is set so that
Pk-1 le (X&.-) = Te-_ 10. (X 18,,
line 5 in Algorithm 3.
C.3 DIFFERENCE WITH DPS
In this section, we detail the differences between MGPS and the DPS algorithm proposed in Chung al. (2023). While DPS cannot be seen instantition of our methodology, highlight the main differences by deriving specific case of MGPS that closely resembles DPS.
We assume that la
for all k [L,n - I] and that p(y|x) Ay A(x), o3I;, where A Td - Rdy We consider the same optimization procedure as in Algorithm but we: (i) optimize only the mean parameter b4 and fix the covariance of the variational approximation Ukjk-1 Is, (ii) perform a single stochastic optimization step.
Following that, the initialization strategy (3.9) boils down to setting
SP. Aik (xx
xx++))
P P (xx (x++1)
(C.4)
20

---

Published as a conference paper at ICLR 2025
which means that the inital mean parameter is j4g majo, + 1 x++11) With this ini- tialization Pp: r>
Mjk+1 1 (-1x++1) ) P (-1x++1)) has its global optimum attained at Pp: = i Thus, the gradient (C.1) writes yx, Cx (4gg X -E V log P j + //Kjj++12 1 129
Next, updating the parameters of the variational approximation using single stochastic gradient descent step with the gradient estimate - Vv log pg (y j + /Ukik+1 2gg where Za ~ N(O,, Ia) and step-size
ςoz lly A (pl + 2
yields the variational approximation
F Mjk+1 (x+) = N(xx; ae <V v, lyy ( + V 1 I+1
In order to simply the expression we have used the fact that
Ve. log (yg + Voijk-12
lyy - A (P& + (UIj++1Z))) oz V lly AAm--a////// + VuNj++1 14)122 1
Therefore, given Xx+1, draw from the variational approximation in MGPS is obtained following the update
X Xx <Vx, |ly A(mb) ( (Xx, Xx) iid. e PkIK+1 (. (((x++))- On the other hand, the DPS transition is given by
DPS 1 (xx|X*++)
x++1) (xxs; P
< Xx lly A(m (xx+1))) 11)k+110d)
and, hence a sample X is drawn following
x = Xx CV x Ily
A(m (xx+1)) xx
Xx ~ P
The difference between MGPS and DPS is in: (i) the diffusion step used for the denoiser (k: for MGPS, + 1 for DPS), (ii) the sample where the gradient evaluated.
D EXPERIMENTS
In this section we provide the implementation details on our algorithm as well as the algorithms we benchmark against. We use the the hyperparameters recommended by the authors and tune them on each dataset if they are not provided.
D. IMPLEMENTATION DETAILS AND IYPERPARAMETERS FOR MGPS
We implement Algorithm I with 4 | and use the Adam optimizer (Kingma & Ba, 2015) with a learning rate of 0.03 for optimization. The number of gradient steps is adjusted based on the complexity of the task: posterior sampling with the ImageNet DDM prior or FF#Q LDM prior more challenging and therefore requires additional gradient steps. Detailed hyperparameters are provided in Table 6.
The warm-start strategy outlined in Appendix C.2 improved reconstruction plausibility and eliminated potential artifacts. similar effect was observed when performing multiple gradient steps (M 20) during the initial stages. For latent-space models, switching the intermediate step to & = k for the second half of the diffusion process has been crucial and significantly enhanced reconstruction quality by mitigating the smoothing effect, which often removes important details. A similar strategy has been useful for the Gaussian deblurring and motion deblurring tasks on the Tma TmageNet. dataset.
D.2 IMPLEMENTATION DETAILS OF COMPETITORS
DPS. We implemented Chung et al. (2023, Algorithm 1) and refer Chung et al. (2023, App. D) for the values of its hyperparameters. After tuning, we adopt y = 0.2 for JPEG 2% and y = 0.07 for High Dynamic Range tasks.
21

---

Published as a conference paper at ICLR 2025
Table 6: The hyperparameters used in MGPS for the considered datasets.
steps
Learning
steps
20
if 2 n mod otherwise
[3n/d]
100,300)
[k/2]
if 2
A/2]
R [n/22 otherwise
(LDM)
[3n/4]
100, 2003
0.03
if mod otherwise
2
Imagelett
[3n/4]
[50, 300)
[4/2]
0.03
M2
10 if mod 20 2
otherwise
if 2
Gousdaan Mixture
X
300
[4/2]
0.1
M2
if mod 10 otherwise
if 2 ini: mod otherwise
PTB-XL. (ECG)
[3n/4] Ni (30,300)
[4/2] 0.08
Mr
DiffPIR. We implemented Zhu et al. (2023, Algorithm 1) to make it compatible with our existing code base. We used the hyperparameters recommended in the official, released version Unfortu- nately, did not manage make the algorithm converge for nonlinear problems. While the authors give some guidelines to handle such problems (Zhu et al., 2023, Eqn. (13)), examples are missing in the paper and the released code. Similarly, we do not run on the motion deblur task as the FFT-based solution provided in (Zhu et al., 2023) is only valid for circular convolution, yet we opted for the experimental setup of Chung et al. (2023) which uses convolution with reflect padding.
We adapted the implementation in the released code? to our code base. Since DDNM utilizes the pseudo-inverse inverse of the degradation operator, noticed that it is unstable for operators whose SVD are prone to numerical errors, such as Gaussian Blur with wide convolution kernel.
DDNM.
RedDiff. We used the implementation of REDDIFF available in the released code". On nonlin- ear problems, for which the pseudo-inverse of the observation is not available, we initialized the variational optimization with sample from the standard Gaussian distribution.
We relied on the implementation provided in REDDIFF's code some of its authors are co-authors of PGDM. The implementation features a subtle difference with Song et al. (2023a, Algorithm l): in the last line of the algorithm, the guidance term g is multiplied by Va, but in the implementation it is multiplied by /at-10t. This modification stabilizes the algorithm on most tasks. For JPEG 2%, we found that it worsens the performance. In this case we simply multiply by va,, as in the original algorithm.
PGDM.
PSLD We implemented the PSLD algorithm provided in Rout et al. (2024, Algorithm (2)) and used the hyperparameters provided by the authors in the publicly available implementation".
ReSample We used the original code provided by the a and modified it to expose several hyperparameters that were directly accessible in the released version. Specifically, we exposed the tolerance e and the maximum number of iterations N for solving the optimization problems related to hard data consistency, as well as the scaling factor for the variance of the stochastic resampling distribution 7. Our experiments revealed that the algorithm is highly sensitive to E. We found that setting equal to the noise level of the inverse problem gave the best reconstruction across tasks and noise levels. We set the maximum number of gradient iterations to N 200 to make the algorithm less computationally prohibitive. Finally, we tuned y for each task but found it has less impact on the quality of the reconstruction compared
22

---

Published as a conference paper at ICLR 2025
D.2.1 RUNTIME AND MEMORY
To get the runtime and GPU memory consumption of an algorithm dataset, we average these two metrics over both samples of the dataset and the considered tasks in Section 4.
Runtine minntes
H
MGPS
PSLD RESAMPLE
Runtine to inbusss
Menwary usago
MGPSs MGPS1co MGPSaa REDDIFF
PGDM
DPS
DIFFPIR
DDNM
in minttes
Menoay asage in: GB
2-
MGPSp MGPS MGPSpM REDDIFF PGDM
DPS
DIFPPIR
DDNM
Figure 7: Runtime and memory requirement of the considered algorithms on datasets: FFHQ latent space (1" row), FFHO pixel space (2nd row), and ImageNet (3td row). The left axis displays the runtime in minutes, whereas the right axis the GPU memory requirement in Gigabytes (GB).
D.3 GAUSSIAN MIXTURES
In this section, elaborate more on the Gaussian mixture experiment in Section 4, where consider a linear inverse problem with Gaussian mixture as prior. Therefore, the likelihood is p(yl-) + Ny; Ax, Ax, I. Recall that a Gaussian mixture with C E N components, whose weights, means, and covariances are w; > 0, m; E IRd, and La respectively, has the density
(x))
ofla).
The denoiser is obtained via Tweedie's formula (Robbins, 1956), Moja (xx) = (xx + (1 a&) log qu (xx)) / /axx, where the densities of the marginals (qx) are straightforward,
Denoiser.
qu (x(x)) / / q(x0)qm0(xx) xo) dxi Σ wi N ofa)) (xo) dxo
wxN(xai (a (oro? + vx)
Posterior: Furthermore, following Bishop (2006, Eqn. 2.116), the posterior can be shown to be Gaussian mixture
x(x) x (xx)((x)
x T. (x; Σ,)
23

---

Published as a conference paper at ICLR 2025
with new weights, covariances, and means given by
W; = wyN(y; Am; Lf + +of Σ. = (11/f]
m, = (.(00)) (L/of)m:), where the weights are un-normalized.
Experimental setup. We consider two setups where d E {00, 200 and generate Gaussian mixtures of C' 25 components with means (m, ((8, 8j.... .,, 8i, 8j) E Rd (i,j) E 1- 2]²), unit covariances, and weights wN drawn from a uniform distribution on [o. 1] then normalized to sum to 1. The linear inverse problem (y, A) is generated by first drawing the matrix A E IR xx with entries i.i.d. from N(0, 1), then sampling x" from the prior and finally computing y = Ax* + gye with & ~ NO, 1) to get the observation. The standard deviation of the inverse problem fixed to dy 0.05 across all problem instances.
To assess the performance of each algorithm, we draw 2000 samples and compare against 2000 samples from the true posterior distribution using the Sliced Wasserstein (SW) distance by averaging over 10$ slices. In Table 1, we report the average SW and the 95% confidence interval over 100 replicates.
D.4 EXTENDED IMAGE EXPERIMENTS
To strengthen our conclusions and calculate the FID, we reran MGPS and the two closest competitors on 1000 images for the following five tasks: half mask, JPEG, motion deblur, nonlinear deblur, and high dynamic range, using the three priors (FFHQ, ImageNet, FFHC-LDM). The Table shows that MGPS significantly outperforms the other competitors, including for the FID, with no significant change in other metrics compared to what we calculated on smaller dataset.
We extend, in Tables 8 and 9, the results in Table 2 by including MGPS with 50 and 100. In the setting n = 100, the runtime of MGPS is twice lower than that of DPS, PGDM and comparable that of DIFI-PIR, DDNM and REDDIFF, see Figure It also outperforming them on most of the tasks, especially the nonlinear ones, as is seen in the table below. In the setting n = 50, MGPS has the lowest runtime among all the methods while maintaining competitive performance.
Finally, in Appendix D.6, we present sample reconstructions on the various tasks. For each algorithm, we generate five samples and select the four most visually appealing ones. Completely black or white images correspond to failure cases of DPS.
D.5
ECG EXPERIMENTS
D.5.1 IMPLEMENTATION DETAILS
PTB-XL dataset We use 12-lead ECGs at a sampling frequency of 100 Hz from the PTB-XL dataset (Wagner et al., 2020). For both training and generation, we do not use the augmented limb leads aVL, avR and aVF as they can be obtained with the following relations: a aVL=(I-III)2, aVF=(I+III)2 and a (I+I22. This leads to inputs of shape T x 9 instead of T x 12 (where T is the length of the signal). For evaluation metrics, we reconstruct the augmented limb leads. For the MB task, following Alcaraz & Strodthoff (2022), we use 2.56-second ECG random crops, hence the inputs are of shape 256 x 9. For the ML task, we use the full 10-second ECGs and pad them into a tensor of shape 1024 9. Table 11 summarizes the input shapes for models and algorithms for each task. Table 12 summarizes the distribution of train/val/test splits and the number of cardiac conditions (RBBB, LBBB, AF, SB) per split. Note that for posterior sampling, we only use the test set, and for both training and sampling we never use the cardiac condition.
Diffusson model Following Alcaraz Strodthoff (2023; 2022), we used Structured State Space Diffusion (SSSD) models (Gu et al., 2022; Goel et al., 2022). SSSD generates sequence u(t) with the following differential equation denpending on the input seuqnece x(t) and hidden state h(t):
h' (t(t)) = Ah(t) + Bx(t)
(D.1)
u(t)= Ch(t) + Da(t),
(D.2)
24

---

Published as a conference paper at ICLR 2025
Table 7: Mean and confidence intervals of LPIPS/PSNR/SSIM values and FID value for various linear and nonlinear imaging tasks on 1k images for the 3 priors. Best bold.
ImageNet DDNM
FFHQ-LDM
Metric
MGPS
DDNM
DIFPPRR
MGPS
DIVPIR
MGPS
RESAMPLE
Half mask
HID
27.0 0.19 1: 0.00
38.6
45,2
40.0
50.0
57.0
49,5
66.6
LPIPS
0.23
0.00
0.25
0.00
0.30 0.00
0.38 0.01
0.40 1. 0.01
0.26
0.00
0.30 1. 0.00
PNSR
15.9
16.3 0.2
16.1 0.3
15.0 0.1
1.1
1.1
15.6 0.1
15.7 = 0.1
SSIM
0.70 1. 0.00
0.74
0.00
0.72 0.01
0.63 0.00
0.68 0.01
0.67 0.01
0.69 0.00
0.67 1. 0.00
MGPS
DPS
DIFFPIR
MGPS
DPS
DIPFIR
MGPS
RESAMPLE
Motion deblur
FID
29.7
36.7
77.0
35,3
55.0
87.3
44,6
51.8
LPIPS
0.12 = 0.00
0.17 0.00
0.22 0.00)
1.20::000
01:0 0:00
0,39 0.01
0.19 0.00
0.00
0:20 0.00
PNSR
2617=
21.
27 0.1
211+
2.1
212
26.
26.7 + 0.1 0.72 = 0.00
SSIM
0.77 = 0.00
0.70 = 0.01
0.71 = 0.00
0.67 : 0.01
0155 0.01
0.61 = 0.00)
0.76 0.00
JPEG (QF=
FID
31.6
87.6
109
61.4
128.8
92.8
45,0
65.3
LPIPS
0.01
0.00
0.00
0.21
-:
PNSR
0.1
19.0 0.2
24. 0.1
2.1
16.7 0.1
22.2
21.6 1216=011
21.8 0.1
SSIM
=:
0.53 0.02
0.70
0.00)
0.0010000
0.02
0.01
0.00
Nonlinear dehlur
FID
50.8
164
88.4
113
272
112
69.2
71.5
LPIPS
0.23 1: 0.01
0.51 0.02
0.68 0.01
0.43 0.01
0.83 0.00
0.66 2 0.01
0,26 0.01
0.32 1: 0.01
PNSR
21.3 0.2
16.2 0.5
21.9 0.1
22.2 0.2
9.9 0.4
20.7:
23.
0.1
24.2 0.1
SSIM
0.70 1 0.01
0.45
0.02
0.42 0.01
0.58 0.01
0.41 0.01
0.241 0.01
0.69
0.01
0.67 1: 0.01
High dynamic range
35.7
38.7
LPIPS
0.08 1. 0.01
0.40 0.04
0.20 1 0.01
0.11 L 0.01
0.83 1 0.02
0.20 1. 0.01
0.14 0.00
0.12 0.00
PNSR
27.0++0.1
18.7 0.2
26.3
0.2
25.5
26.0
SSIM
0.83 1 0.01
0.55 0.04
0.72 1 0.01
0.83 1 0.01
0.23 1. 0.02
0.71 L 0.01
0.80 0.01
0.83 1.331.000 0.01
where A, B, C, D are transition matrices. We use the SASDSA (Alcaraz & Strodthoff, 2022) (also denoted as Sashimi (Goel et al., 2022)) architecture publicly available 78 We parametrize the matrix A = A pp" where p is vector and A is diagonal matrix. This parametrization with facilitates the control of the spectrum of A to enforce stability (negative real part of eigenvalues). We use multi-scale architecture, with stack of residual S4 blocks. The top tier processes the raw audio signal at its original sampling rate, while lower tiers process downsampled versions of the input signal. The outputs of the lower tiers are upsampled with up-pooling and down-pooling operations and combined with the input to the upper tier to provide a stronger conditioning signal. are deseribed in table 13. The selected model is the model of the last epoch. Training time 30 hours for 2.5 seconds ECGs, 42 hours for 10 seconds ECGs.
Algorithm parameters We the same parameters described in table 6 for MGPS and the implementation described in appendix D.2 for competitors. Since we had to run the experiments over 2k samples, set the number of diffusion steps such that the runtime approximates 30 seconds for generating 10 samples. This leads to the parameters shown in table 14. For the ML experiment, we also tested MGPS with 300 diffusion steps, which resulted in a runtime of minutes and 30 seconds per batch of 10 samples. The results improved, but we already outperform competitors with just 50 diffusion steps and 30-second runtime.
the MB task, for each observation, we generate 10 samples instead of 100 in in (Alcaraz & Strodthoff, 2022). We then compute the Mean Absolute Error (MAE) and Root Mcan Squared Error (RMSE) between each generated sample and the ground-true The final score is the average of these errors over the 10 generated samples. We report the confidence intervals over the test set table and table 16.
Evaluation metries
For the ML task, we classifier trained detect four cardiac conditions: Right Bundle Branch Block (RBBB), Left Bundle Branch Block (LBBB), Atrial Fibrillation (AF), and Sinus Bradycardia
'https: : http://gttttb.coo 'https:/ / thub. com Teeert fgu/di ffwave- sashimi htt////eeeeeeeeeeeeeeeeeeee
4Hea I twtolo
25

---

Published as a conference paper at ICLR 2025
Table 8: Mcan LPIPS/PSNR/SSIM values for various linear and nonlinear imaging tasks on the FFHQ 256 x 256 dataset. Best is in bold and second best is underlined.
Task
MGPS
MGPS
MGPS
DPS
PGDM
DDNM
DIPPPIR
REDDIFF
LPIPS
SR (x4)
0.13
0.10
0.09
0.09
0.33
0.14
0.13
0.36
(x16)
0.27
0.26
0.26
0.24
0.44
0.30
0.28
Box inpuinting
0.16
0.12
0.10
0.19
0.17
0.12
0.18
0.19
Half mask
0.24
0.22
0.20
0.24
0.26
0.22
0.23
0.28
Guussian Deblur
0.16
0.87
0.19
0.12
Motion Deblur
0.19
0.15
0.13
0.16
0.21
JPEG (QF =
0.20
0.17
0.39
1.10
0.32
Phase retrieval
0.20
0.14
0.11
0.46
0.25
Nonlineur deblur
0.23
0.23
0.23
0.52
0.66
High dynamic range
PSNR
SR
27.83
27.79
27.79
28.24
23.34
29.52
27.17
27.25
SR (x16)
20.45
20.34
20.22
20.67
17.65
22.43
20.75
21.91
Box inpainting
21.55
18.39
21.13
22.35
Half mask
15.10
15.32
15.54
14.82
16.03
16.16
15.17
16.21
Guussian Deblur
25.09
25.19
25.89
24.20
13.36
26.69
25.89
26.72
Motion Deblur
26.07
26.64
26.48
24.24
27.58
JPEG (QF 2)
25.00
25.23
24.94
18.50
12.76
24.42
Phase retrieval
24.20
27.25
14.87
Nonlineur deblur
24.16
24.21
24.37
15.89
21.97
High dynamic range
24.77
26.07
27.74
16.83
21.25
SSIM
(x-))
0.81
0.79
0.81
SR (x16)
0.58
0.57
0.55
0.59
0.38
0.67
0.60
0.62
Box inpuinting
0.80
0.81
0.82
0.76
0.70
0.83
0.80
0.70
Half mask
Gaussian Deblur
0.71
0.73
0.75
0.69
0.14
0.77
0.73
0.76
Motion Dehlur
0.77
0.78
0.77
0.71
0.71
JPEG (QF
0.75
0.74
0.73
0.51
12.76
0.71
Phase retrieval
0.74
0.78
0.79
0.43
0.61
Nonlineur deblur
0.70
0.70
0.69
0.46
0.42
High dynamic range
0.79
0.81
0,86
0.48
0.71
(SB) on the FTB-XL dataset. We follow the XRESNETIDSO described in (Strodthoff et al., 2020)
with hyper-yaramnne papee--rrameters reported in table 15. We apply the classifier to the moound-truthe ECG of the test set and to the samples generated from lead I. As in the MB task, for each observation, 10 samples are generated. The output of the classifier averaged over these 10 samples. For each cardiac condition, we compute balanced accuracy to account for class imbalance (see table 12). The classification threshold is selected using PTB-XL validation-set.
D.5.2 ADDITIONAL. RESULTS
Discussion In table demonstrated that most posterior sampling algorithms outperform the trained diffusion model for the missing block reconstruction task. This result is particularly interesting as it suggests that training a diffusion model (which takes several days) is not necessary for this task. However, when Alcaraz & Strodthoff (2023) trained the model the reconstruction of random missing blocks (RMB), where 50% of each lead is independently removed, the model outperformed all posterior sampling algorithms two algorithms, as well as the model trained on the MB task and the RMB task. The significant improvement between RMB and MB can be seen enhancement due data augmentation.
the MB task. We report in table 16 the results of the top
Generated samples
D.6 SAMPLE IMAGES
While may appear that some of the methods underperform on some tasks/images compared to the original publications, for instance Figure 14 and Figure 16, they still produce competitive
26

---

Published as a conference paper at ICLR 2025
Table 9: Mcan LPIPS/PSNR/SSIM values for various linear and nonlinear imaging tasks on the IMAGENET 256 x 256 dataset. Best is in bold and second best is underlined.
Task
MGPS
MGPS
MGPS
DPS
PGDM
DDNM
DIFPPIR
REDDIFF
LPIPS
SR (x4)
0.36
0.33
0.30
0.41
0.78
0.34
0.36
0.56
(x16)
0.58
0.53
0.50
0.60
0.70
Box inpuinting
0.31
0.26
0.22
0.34
0.29
0.28
0.28
0.36
Half mask
0.39
0.34
0.29
0.44
0.38
0.38
0.35
0.44
Guussian Deblur
0.35
0.35
Motion Deblur
0.35
0.25
0.22
0.39
0.40
JPEG (QF 2)
0.50
0.46
0.42
0.63
1.31
0.51
Phase retrieval
0.54
0.62
Nonlineur deblur
0.49
0.47
0.44
0.88
0.67
High dynamic range
0.22
0.15
0.10
0.85
0.21
PSNR 1
(x-1)
24.68
24.70
24.77
23.52
15.67
25.55
24.26
24.24
SR (x16)
18.56
18.42
18.04
18.22
15.80
20.43
19.37
19.95
Box inpainting
17.52
17.95
18.25
14.34
17.35
20.08
19.77
18.90
Half mask
14.98
14.65
17.06
Gaussian Deblur
22.56
21.96
20.10
21.20
9.93
23.29
22.10
23.27
Motion Deblur
23.91
25.03
24.50
21.59
24.43
JPEG (QF 2)
21.96
22.17
22.44
16.11
5.29
22.15
Phase retrieval
16.36
16.94
18.10
14,40
15.78
Nonlinear deblur
21.89
22.23
22.36
20.76
High dynamic range
23.93
25.64
27.04
9.32
22.88
SSIM
SR (x4)
0.65
0.66
0.66
0.60
0.23
0.70
0.63
0.61
(x16)
0,41
0.40
0.46
0.47
Box inpainting
0.71
0.74
0.76
0.70
0.62
0.78
0.74
0.67
Half musk
0.61
0.63
0.65
0.55
0.53
0.69
0.63
0.62
Gaussian
Motion Deblur
0.64
0.69
0.65
0.55
0.60
JPEG (QF
0.59
0.60
0.39
0.01
Phase retrieval
0.37
0.40
0.43
0.29
0.26
Nonlineur deblur
0.57
0.58
0.58
0.24
0.41
High dynamic range
0.75
0.81
0.84
0.25
Table 10: Mean LPIPS/PSNR/SSIM values for various linear and nonlinear imaging tasks FFHQ 256 x 256 dataset with LDM prior. Best in bold and second best is underlined.
MGPS
RESAMPLE
PSLD
MGPS
RESAMPER
PSLD
MGPS
RESAMPLE
PSLD
LPIPS
PSNR
SSIM 1
SR (x:4)
0.11
0.20
0.22
28.46
26.08
25.53
0.83
0.69
0.70
(x16)
0.36
Box inpainting
0.22
0.26
22.94
18.80
20.39
0.79
0.75
0.66
Half mask
0.25
0.30
0.31
15.11
14.59
14.75
0.69
0.67
0.61
Gaussion Deblar
19:95
Motion Deblur
0.19
0.41
26.49
26.85
18.14
0.77
0.72
0.39
JPEG (QF
0.26
24.75
24.33
0.72
Phase retrioval
0.41
22.21
19.05
0.62
Nonlineur deblur
0.26
0.50
23.79
24.44
0.70
0.68
High dynamic range
0.15
25.17
25.42
0.81
VO
II
V1
V4
Figure 8: Missing block imputation with MGPS 2.56s 12-lead ECG. Ground-truth in blue, 10%-90% quantile range in green, random sample in orange.
27

---

Published as a conference paper at ICLR 2025
Table 11; ECG size and shape-shape per task.
Task
ECG size (seconds)
Total leads
Used leads
Input shape
MB
2,56
II III V1-6 II III V1-6
256
ML
1024
Table 12: PTB-XL XL dataset description.
Split
All
RBBB
LBBB
AF
SB
Truin
17.403
432
428
1211
503
Val
2,183
55
54
Test
2,203
54
MGPSu (NSR)
DPS (NSR)
SVR
V0
V3
aVR
VO
3VL
V2
VS
aMt
V2
V5
PGDM (USR)
DONM (NSR)
DVR
V0
V3
aVR
VO
V2
V5
aM
V2
difffir (NSR)
reddiff (NSR)
3VR
V0
V3
avR
VD
V3
3V.
V2
V3
aVL
V2
VB
Figure 9: Missing lead reconstruction from lead on 10s 12-lcad Normal Sinus Rythm (NSR) ECGs. Ground- truth in blue, 10%-90% quantile range green, random sample in orange.
reconstructions on others; see for example Figure 15, Figure 18, and Figure 23. Similar patterns are also displayed in Zhang et al. (2023, Figure 10) and Liu al. (2023, Figure 9). With that being said, we highlight that these discrepancies appear more frequently on the ImageNet dataset than the FFHQ one. This can be explained by the fact that Ima geNe is notoriously challenging due its diversity, encompassing 1000 classes. This also seems to happen one of the most difficult tasks, namely the half mask one.
28

---

Published as a conference paper at ICLR 2025
Table 13: ECG diffusion generative model hyper parameters.
Hyper-parann prppe-pprrmeter
Value
Residual layers
Pooling factor
[1. 2. 2J
Feature expansion
Diffusion embedding dim. 1
128
Diffusion emhedding dim. 2
512
Diffusion embedding dim. 3
512
Diffusion steps
1000
Optimizer
Adam
Number of iterations
150k
Loss function
MSF
Learning rate
0.002
Batch size
128
Number of parameters
16 millions
Table 14: Number of diffusion steps used posterior sampling algorithms ECG tasks.
MGPS MGPS300
PGDM
DDNM
DIFFPIR
REDDIFF
MGP5 (R8BB)
DPS (RB86)
3VF
aVF
aVL
PGDW (RDDB)
DDMM (ROBO)
3VF
V1
aVF
MV
V2
V5
WM
diffpir (RB86)
reddiff (RBG6)
SVF
V1
aVF
VI
Figure 10: Missing lead reconstruction from lead on 12-lead Right Bundle Branch Block (RBBB) ECGs. Ground-trus blue, 10% 90% quantile range in green, random sample in orange.
Table 15: XRESNETI D50 downstream classifier
Slpper-arameter
Value
Blocks
Layers per block
[3, 4,
Expansion
4
Stride
1
Optimizer
Adam
Learning rate
0.001
Batch size
0.001
Epochs
100
29

---

Published as a conference paper at ICLR 2025
Table 16: MAE and RMSE missing block task PTB-XL dataset.
Metric
MGPS
DDNM
TRAINEDDUT-ME
TRAINEDDUFT-RN
2e-8
0.0879 0.217 6e-3
RMSE
0.225 + 4
0.224 = 4e-3
0.266 + 3e-3
MGPSso (LDD0)
DPS (LDG0)
DVR
Q0
3VR
20
V3
av
aV
PGDM (LB68)
DONM (.666)
20R
20
2WR
VV..
V5
diffpir (LDGO)
reddiff (LD68)
DVR
R0
2VR
80
V33
avi.
V5
AVE
VS
Figure 11: Missing lead reconstruction from lead on 10s 12-lead Left Bundle Branch Block (LBBB) ECGs. Ground-truth blue, quantile range green, random sample orange.
MGPSsp (AF)
DPS (AF)
VB
AVR
VO
3VL
V2
V5
3VL
V2
V5
PGDD (AF)
DDNM (AF)
20R
V0
39R
V0
4v
V2
V5
V2
V5
diffpir (AF)
rediiiff (AF)
SVR
VO
V3
aVR
VO
V3
3M.
V2
V5
aVL
V2
V5
Figure 12: Missing lead reconstruction from lead I 10s 12-lead Atrial Fibrillation (AF) ECGs, Ground -ruuh blue, 10% 90% quantile range green, random sample orange.
30

---

Published as a conference paper at ICLR 2025
MGPSa 150)
DPS (50)
V0
3VL
V2
V5
VVL
V2
V5
PGDM (SB)
DDNM (SB)
RVR
VB
AWR
VB
VVL
V2
V3
VLL
V2
V5
diffpir (5.0)
reddiff (50)
DVR
V0
V3
avR
V0
V3
3VL
V2
aVL
V2
V5
Figure 13: Missing lead reconstruction from lead 10s 12-lead Sinus Bradycardia (SB) ECGs. Ground-truth blue, 10%-90% quantile range green, random sample orange.
31

---

Published as a conference paper at ICLR 2025
Olwervdiio Ground troth
Observation Grourd trath
MGPS
DONA
DDNM
DPS
DPS
HGDD
HGDM
DIFFPIR
DEFFIR
REDDire
REDDIFF
Figure 14: Sample reconstructions with box mask on ImageNet dataset.
32

---

Published as a conference paper at ICLR 2025
MGPS
MGPS
DIXN
DDKM
DPA
DPS
Oloervalimm
Greand teath
Ilberraaiio Ground keutl
MGPS
MGPS
DANA
HICION
Figure 15: Sample reconstructions with half mask FFHQ dataset.
33

---

Published as a conference paper at ICLR 2025
MCPS
DDX
Brober
Oloercoliim
Groand teath
Gronnd keath
MGPS
DONA
DDXM
DTS
HCTOO
MCDM
Doos
TzoDay
Figure 16: Sample reconstructions with half mask ImageNet dataset.
34

---

Published as a conference paper at ICLR 2025
Observation
Ground truth
Observation
Ground truth
MGPS
MGP3
DPS
DFS
RESDDrF
Observalion Ground truih
Observalio
Grounl truil
MGPS
MGGS
DPS
DPS
REDDIPS
REDDer
Figure 17: Sample reconstructions on motion deblurring task.
35

---

Published as a conference paper at ICLR 2025
MGPS
MGPS
DONN
DPS
MODM
Oloervaliam Greand teath
libervation Ground keath
MGPS
DDNXI
Figure 18: Sample reconstructions Gaussian deblurring task.
36

---

Published as a conference paper at ICLR 2025
Observation
Ground truth
Observation
Ground truth
MGPS
MGPS
DPS
DPS
RESDIFF
READaY
Observalion Ground truih
Obsrrvalio
Ground truil
MGPS
MGPS
DPS
DPS
REDDiPS
REDDur
Figure 19: Sample reconstructions on phase retrieval task.
37

---

Published as a conference paper at ICLR 2025
Obsrrvtion
Cround truth
Obecrviioa
Cround truth
MGPS
DPS
TIDD
ReoDir
Hanbarr
Obscrvation Ground truth
Obeervtion
Cround truth
MGPS
MCPS
D75
1325
R&DM
RunDir
Figure 20: Sample reconstructions on JPEG task.
38

---

Published as a conference paper at ICLR 2025
Obsrration Ground truth
Obverntion Grond truth
MGPS
MCPS
DPS
RuDDee
KiDDur
Obscrvation Cround trath
Oboervtion
Ground teuth
MGPS
MCPS
DPS
RoODwe
Figure 21: Sample reconstructions high dynamic range task.
39

---

Published as a conference paper at ICLR 2025
MGPS
DINN
DDNN
191
mP5
DosePPo
UNU
Oloervalim Greund trith
Gronn trath
MGPS
MGPS
DDX
DrS
MGDM
Dhos
Figure 22: Sample reconstructions on SR (16x) task.
40

---

Published as a conference paper at ICLR 2025
MCPS
MCTS
DDNV
DPS
Oluerroaim Groand inilh
MGPS
DDXM
DPS
N&DD
Dies
TroDrr
Figure 23: More sample reconstructions SR (16x) task.
41

---

Published as a conference paper at ICLR 2025
Observtion Ground truth
Obiervation
Ground trunh
MGPS
MGPP
ReSAOPLE
JPEG 2
Observatio Ground truth
Obscrvation
Ground trunh
MGPS
MGGPS
TEESAAPLE
HOsS KXPPL
PSLD
PSLD
(b) Outpainting with half mask
Observation Ground
MOPS
MGPS
RUSAMPLE
HrSSAMPL
PSLD
PSLD
(c) SR 4x
Figure 24: Sample reconstructions with latent diffusion models on FFHQ dataset.
42

---

Published as a conference paper at ICLR 2025
Otservation
Ground truth
Figure 25: More sample reconstructions with MGPS on the expand task.
43

---

Published as a conference paper at ICLR 2025
Obscration
Ground truth
Figure 26: More sample reconstructions with MGPS and half mask.
44

---

Published as a conference paper at ICLR 2025
Observation
Ground truth
Figure 27: More sample reconstructions with MGPS and half mask.
45


Reviews:
[{"review_id": "OAK1izDebt", "reviewer": "Reviewer_U6Fn", "summary": "This paper introduces a novel diffusion-based posterior sampling method called Midpoint Guidance Posterior Sampling (MGPS) to address Bayesian inverse problems. In cases where denoising diffusion models (DDMs) are used as priors, MGPS aims to approximate the posterior while balancing the complexity between guidance and prior transition terms. The method leverages an intermediate midpoint state to improve posterior approximation and incorporates a Gaussian variational approximation for additional flexibility. MGPS is validated on linear and nonlinear inverse problems across various domains, including image and ECG signal reconstruction, and outperforms several state-of-the-art methods.", "questions": "1. Would there be an optimal mid-point schedule for $l_k$, which allows fewer diffusion model evaluations and comparable performance? Or better technique can be used to find the optimal schedule? Or can we suggest a criteria for evaluating the trade-off between performance and computational cost?\n\n2. What is the key reason that makes mid-point guidance perform better? When the same prior is applied, what difference does the midpoint sequence make to sample algorithm as difference stages of the posterior distributions?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The approach introduces a midpoint guidance mechanism that provides a novel trade-off for guidance and complexity of the learned transition term in diffusion models. This make it different from other diffusion-based posterior sampling methods.\n\n2. The method is well-justified with mathematical rigor. The decomposition of the backward transition and the use of Gaussian variational approximations.\n\n3. MGPS is extensively evaluated across both synthetic and real-world tasks, such as Gaussian mixture sampling, image super-resolution, and ECG imputation. Experimental results show significant improvements in reconstruction quality over baseline methods.\n\n4. The paper is generally well-organized, providing clear problem definitions, methodological details, and experimental results. Detailed explanations and algorithmic steps (Algorithm 1) support reproducibility.", "weaknesses": "1. More exploration of $\\eta$ and the midpoint sequence's impact on different task types and data complexities would clarify MGPS's adaptability across scenarios. The authors showed the effect of $\\eta$ in the Gaussian toy example. It would be interesting to see $\\eta$'s influence in other tasks.\n\n2. The trade-off introduced by the midpoint state could be theoretically explored further. The authors mention the need for tuning this midpoint sequence but provide limited theoretical insights on why this works well. For instance, the bounds on the approximation errors may be further derived, or the balance between the prior and guidance terms could be analyzed with the midpoint sequence.\n\n3. Concerns about the correct implementation of other compared methods remain for me, as those methods perform in Figures 13 and 15 worse than those in the original publications related to some image domain tasks. Or are those methods undertuned?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oTWsX6kJpl", "reviewer": "Reviewer_bHyn", "summary": "The paper proposes a novel midpoint guidance strategy for posterior sampling of diffusion models. The main idea is to first move to a mid-point state l_k that is a function of k for guidance, and then noise back to obtain X_k unconditionally. This seems to solve issues associated with other SOTA approaches that perform the guidance based on variations of Tweedie's formula. Extensive experiments are provided, along with comparison to SOTA methods. Improvements are shown in almost all inverse problems, and notably for nonlinear inverse problems. \n\nI think the paper has good merit, but its exposition is overly complicated and notation does not match the rest of the literature. These dampened my enthusiasm, but I'm happy to reevaluate after the authors respond.", "questions": "Following up on the weaknesses:\n- Why was a different notation used compared to existing works on posterior sampling for inverse problems?\n- Why was only 50 randomly selected evaluation points used for FFHQ and ImageNet? How was this random selection done?\n- Why was only LPIPS used for evaluation, and other standard metrics like PSNR/SSIM were not reported?\n- Why was DPS run for 300 steps instead of the more standard 1000?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The idea of midpoint guidance is novel, and intuitively appealing to solve the issues associated with guidance issues especially at the early stages of the diffusion process.\n- Very thorough evaluation.\n- Results are good, showing improvement over SOTA methods, including both DDMs and LDMs (that are commonly used in these applications).\n- Multiple nonlinear inverse problems are studied, an area where other posterior sampling methods have issues. A good level of improvement is shown in these applications.\n- Performance also evaluated across NFEs.\n- Good sample variety is shown.", "weaknesses": "- Unfortunately, the exposition is overly complicated. The idea can be explained much more clearly, but also partly due to non-standard notation, the gist does not come across easily. Section 3 would benefit from a substantial rewrite that changes the notation (please see below), and highlights the main ideas, and potentially even including a figure to show the midpoint guidance idea.\n- Similarly, the notation does not match rest of the literature on posterior diffusion sampling for inverse problems. For instance, the posterior is denoted by \\pi(x), which is written as a marginal, though this should depend on the observations y. Similarly g(.) in (2.1) should be conditional on y, but this is not done either. This propagates throughout the paper, and makes it hard to appreciate the contributions. Please use standard notation to match other existing works.\n- The 50 randomly selected evaluation points for FFHQ and ImageNet is a bit unconvincing. There are standard validation sets that are publicly available for both databases, and these are commonly used in other papers (e.g. DPS, PGDM). Please report your results over a larger set (& also explain how random selection was done).\n- Unclear why only LPIPS is provided. PSNR/SSIM must be provided as well. I understand improvement may not be uniform for those metrics, but this should be up to the reader to figure out.\n- DPS is typically run with NFE = 1000, but it was used for N = 300 in this paper for comparison. This may further close the LPIPS gap.\n\nThe following points are not really weaknesses, but I wanted to note them:\n- The ECG application comes out of nowhere. Without enough motivation and knowing the difficulty of the task, the contribution here is a bit hard to appreciate.\n- There are also two additional references that may be of interest:\na) arXiv:2402.02149, which is similar to Boys et al in spirit\nb) arXiv:2407.11288 (ECCV 2024), which learns w_k in (2.5) and also uses a diagonal approximation to circumvent vector-Jacobian calculations", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SmZTpHbbCp", "reviewer": "Reviewer_eMe4", "summary": "The authors propose 'midpoint guidance posterior sampling,' where the reverse diffusion process is decomposed into two steps:\n\n1. Denoising the current diffusion measurements into a 'midpoint' state.\n2. Renoising the midpoint measurements to obtain the next diffusion iterate.\n\nThe denoising approach consists of approximating a Gaussian variational approximation in conjunction with the DPS guidance proposed by Chung et al to sample an estimated image. The variational approximation is learned at each diffusion timestep.\n\nThe renoising stage appears to be the typical DDIM/DDPM computation of the next diffusion iterate.\n\nThis approach of midpoint guidance achieves very strong empirical performance on a wide variety of problems/datasets and has strong theoretical foundations.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "S1) The approach is well-motivated and has strong theoretical backing, with a novel theoretical result included in Appendix A.3.\n\nS2) The authors evaluate many different problems on multiple datasets, achieving relatively strong performance across the board.\n\nS3) The paper is well-written and reasonably easy to follow (although I have some gripes with notation, outlined in the 'Weaknesses' section below.\n\nS4) Many great visuals, especially in the appendices.\n\nS5) The authors provide very detailed and explicit implementation details for their competitors. I think that this is of vital importance and appreciate the efforts made by the authors to share these details.", "weaknesses": "At a high level, I quite like this work. However, as described below, I feel that the experimental results are incomplete.\n\nW1) Even though the paper is well-written, and all mathematical elements check out (at least to me), the notation makes all of the math in Section 3. In particular, the differentiation between scalar and vector quantities is not sufficient. I would suggest that the authors make vector quantities boldface (e.g., $\\boldsymbol{x}$). This would make things much easier to read.\n\nW2) While I appreciate the robust slate of experiments, using test sets of size 50 (at least for FFHQ, ImageNet) is not sufficient for two reasons:\n1. I would argue that the standard test set size for any diffusion inverse solver is 1k images. This seems to be the accepted number, and large enough to truly understand model performance. A test of size 50 is just not sufficient for acceptance to a conference like ICLR. I feel that the results would be more convincing if there were fewer experiments with a larger test set. Note that I am explicitly speaking on the image datasets, where samples are plentiful. There are plenty of problems where there is not enough data to have a test set comprised of 1k samples and that is fine. To summarize: I think that fewer experiments are not a bad thing if it means more robust testing.\n2. Only using 50 test samples does not enable reliable computation of FID (a metric that is noticeably absent from the paper). FID is a standard metric when evaluating diffusion inverse solvers. With it absent, the experimental results are incomplete.\n\nW3) I also think that the experimental results are incomplete due to the lack of a pixel-space quality metric. The authors argue against PSNR/SSIM in Section 4, but I disagree. I think that LPIPS is a great choice, but that PSNR is still a necessity when testing because similarity in the pixel-space matters too. If the samples don't respect the measurements, that is a problem and LPIPS may not catch it since it is a feature-based metric.\n\nTo summarize W2 and W3, I think that the authors need to reconsider their evaluation of problems which use the image datasets. I would suggest:\n1. 1k test samples.\n2. Computing PSNR and FID in addition to LPIPS.\n\nThese changes are critical to fully understanding model performance. Without this sufficient experimental evaluation, I cannot recommend this paper for acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tb2VinwGR1", "reviewer": "Reviewer_HCev", "summary": "The paper proposes MGPS, a variational inference approach to diffusion model-based inverse problem solving (DIS) in the standard guiding reverse diffusion setup. Two clever ideas are used:\n\n1. Instead of using the DPS approx after denoising $k + 1 \\rightarrow k$, denoise it up to $0 < \\ell_k < k$, and use the posterior mean of $x_{\\ell_k}$ instead of the posterior mean of $x_k$ to compute the DPS gradient. This way, you can control the trade-off arising in the approximation error for the denoising kernel and the likelihood computation.\n\n2. For every reverse diffusion timestep $k$, use $j$ iterations of stochastic optimization for fitting a variational distribution for the reverse posterior kernel. The authors propose to use a Gaussian kernel but also optimize over the diagonal elements of covariance. This is different from the previous works where typically, an isotropic variance is used, probably with the exception of [1].\n\nOverall, the paper is well-written, has a clear theory, and has great results. The reviewer especially appreciates the efforts that the authors made to make the experiments as fair as possible, carefully addressing the details that are often ignored. I do have some questions and concerns on the practical implementation of the method, and in some places, how to derive it. Nevertheless, I think the paper should be a clear accept.\n\n\n**References**\n\n[1] Peng, Xinyu, et al. \"Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance.\" ICML 2024.", "questions": "1. In Tab. 2, only the LPIPS values are reported. I understand that this is probably to save space, but I would recommend also including other standard metrics such as PSNR, SSIM, and FID.\n\n2. MGPS is based on DDPM sampling. Would there be any way to incorporate faster solvers into this?\n\nI am willing to further raise the score after clarification.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper is well-written and relatively straightforward to follow.\n\n2. When an approximation is used, the authors do a good job of explaining the rationale, by either showing it theoretically or experimentally.\n\n3. MGPS is a good balance between a theoretically grounded solution and a practical solution, not requiring too much computation.\n\n4. The results of image restoration tasks are clearly SOTA.\n\n5. Experiments are complete. Numerical experiments on toy data, image restoration experiments, and ECG completion all indicate the superiority of the method.", "weaknesses": "1. Some parts of the derivation are unclear. The authors propose a variational distribution $\\lambda^\\varphi$ for $\\hat{\\pi}^\\theta$ in (3.6), which, to my understanding, is already tractable. Since $\\hat{g}^\\theta$ is Gaussian from DPS, and $p_{\\ell_k}^\\theta$ is also Gaussian, then isn't $\\hat{\\pi}^\\theta$ already a Gaussian? Why would one need an additional variational distribution to approximate this?\n\n2. Adding on 1, I don't quite understand why including the diagonal terms of the covariance for additional optimization would induce better fitting, when $\\hat{\\pi}^\\theta$ would have istropic covariance. Both these points maybe from my misunderstanding, but it should be clarified.\n\n3. Balancing the denoising and likelihood approximation errors is interesting. Is it safe to say that the mainstream previous works that use something similar to (2.5) be considered as simply taking $\\ell_k = k$? Or is it not directly comparable? I believe the authors could spend some more effort on clarifying the difference between the existing methods. There is a related works section, but the connections seem a bit vague. An additional appendix section could be nice.\n\n4. The upper bound of the variational inference problem is defined and used, but it is not explained how this is derived.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)