Sample-efficient evidence estimation of score based priors for model selection
Authors: Frederic Wang
Katherine Bouman
Venue: ICLR 2026 Poster (2026)
Keywords: Computational imaging
inverse problems
model selection
posterior sampling
diffusion models
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: The choice of prior is central to solving ill-posed imaging inverse problems, making it essential to select one consistent with the measurements $y$ to avoid severe bias. In Bayesian inverse problems, this could be achieved by evaluating the model evidence $p(y \mid M)$ under different models $M$ that specify the prior and then selecting the one with the highest value. Diffusion models are the state-of-the-art approach to solving inverse problems with a data-driven prior; however, directly computing the model evidence with respect to a diffusion prior is intractable. Furthermore, most existing model evidence estimators require either many pointwise evaluations of the unnormalized prior density or an accurate clean prior score. We propose DiME, an estimator of the model evidence under a diffusion prior by integrating over the time-marginals of posterior sampling methods. Our method leverages the large amount of intermediate samples that are naturally obtained during the reverse diffusion sampling process to obtain an accurate estimation of the model evidence using only a handful of posterior samples (e.g., 20). We demonstrate how to implement our estimator in tandem with recent diffusion posterior sampling methods. Empirically, our estimator matches the model evidence when it can be computed analytically, and it is able to both select the correct diffusion model prior and diagnose prior misfit under different highly ill-conditioned, non-linear inverse problems, including a real-world black hole imaging problem.

Published as a conference paper at ICLR 2026
SAMPLE-EFRICIENT EVIDENCE ESTIMATION OF SCORE BASED PRIORS FOR MODEL SELECTION
Frederic Wang & Katherine L. Bouman
Department of Computing and Mathematical Sciences
Caltech
Pasadena, CA 91125, USA
{fzwang, klbouman] 0 caltech.edu
ABSTRACT
The choice of prior central to solving ill-posed imaging inverse problems, mak- ing it essential to select consistent with the measurements y avoid severe bias. In Bayesian inverse problems, this could be achieved by evaluating the model evidence p(y MM) under different models M that specify the prior and then se- lecting the one with the highest value. Diffusion models are the state-of-the-ar approach to solving inverse problems with a data-driven prior; however, directly computing the model evidence with respect to diffusion prior is intractable. Fur- thermore, most existing model evidence estimators require either many pointwise evaluations of the unnormalized prior density or an accurate clean prior score. We propose DiME, an estimator of the model evidence of diffusion prior by integrat- ing over the time-marginals of posterior sampling methods. Our method leverages the large amount of intermediate samples naturally obtained during the reverse diffusion sampling process obtain accurate estimation of the model evidence using only a handful of posterior samples (e.g.. 20). We also demonstrate how implement our estimator in tandem with recent diffusion posterior sampling methods. Empirically, our estimator matches the model evidence when can be computed analytically, and is able to both select the correct diffusion model prior and diagnose prior misfit under different highly ill-conditioned, non-linear inverse problems, including real-world black hole imaging problem.
INTRODUCTION
In Bayesian inverse imaging problems across science and engineering, the chosen prior distribution p(x) plays pivotal role in shaping the posterior p(æ y) from which candidate solutions are drawn (Feng et al. 2024 Stuart 2010). In ill-posed settings, the prior acts as regularizer, steering reconstructions toward solutions consistent with expected image properties. the ground truth image lies outside the support of the chosen prior, the resulting reconstruction can be severely biased.
central challenge in Bayesian inference is how to select prior to use when the true distribution is unknown. Consequently, the priors used in practice are often chosen somewhat arbitrarily. If the likelihood of prior could be easily evaluated, would provide a principled basis for prior selection. Specifically, given set of potential models {M, that specify the prior p(æ M.). would like to find the most likely candidate model by computing p(M, y) x p(y M)))(M.. It is commonly assumed that p(M) is uniform, it suffices compute the (log) model evidence, log p(y M,). Model evidence is not only useful for selecting the best prior for reconstruction, but also for quantifying epistemic uncertainty and assessing the assumptions underlying model or physical theory (Trotta] 2007).
Although the model evidence provides principled way select the best prior for reconstruction, computing it generally requires evaluating an intractable integral over the full prior: log p(y M,) = log p y x, M,)p(x M,)dx. This motivates the development of estimation methods, such
* Use footmote for providing further information about author (webpage, altemaatve address) for knowledging funding agencies. Funding leddemeegeeentss go the end of the paper.

---

Published as a conference paper at ICLR 2026
as sequential Monte Carlo (Del Moral et al. 2006 Chen al. 2024). nested sampling (Skilling 2006 Cai et al. 2022), annealed importance sampling Neal (2001): Doucet et al. (2022). and har-
monic mean estimators (Newton & Raftery, [194]] McEwen et al. 2021 Polanska et al.] 2023
Spurio Mancini et al. 2023). However, these methods rely on sampling using the clean prior score x log p(z) the unnormalized density log p(z), which may be inaccurate for out-of-distribution
2, expensive to evaluate, and can lead to slow mixing due to the ill-conditioned data geometry.
Bayesian inverse imaging solvers achieve state-of-the-art performance with diffusion- based priors [Sng et al. 2020; Ho et al., 2020; Song et al. 202 Jalal et al. 2021 Mardani et al. 2023) yet
an accurate model evidence remains intractable using existing methods. Diffusion models generate unconditional samples by denoising sequence of intermediate noised images, each corresponding to time-marginal distribution between pure noise and the clean image. Diffusion-basel posterior sampling methods adhere to posterior time-marginals instead (Chung et al. 2022 Sonn et al. 2023
Coeurdoux et al., 2024 Li et al., 2024 Zhu et al.) 2023 Wu al. [024]] Zhang et al. 2025). While
efficient diffusion density estimators have been proposed (Skreta et al. (024), density- density-based estima- tion methods are still typically high-variane for high dimensional problems and require thousands of posterior samples, which is computationally challenging with diffusion models. Furthermore, as diffusion models learn the score of intermediate noised priors, their estimates of the clean prior score usually less accurate and/or highly ill-conditioned, making it difficult for traditional methods to explore the full distribution, while using their score from a higher noise levels can add bias. As a result, existing estimators are not suitable for or give biased estimates for diffusion- based priors.
We propose the Diffusion Model Evidence (DiME) estimator, a method for es- timating the model evidence under diffusion model prior that does not require the prior score or density. We derive the estimator for posterior sampling methods that anneal along the standard (pos- terior) marginals p(æ, y). as well as a generalized estimator for any arbitrary path of marginals that anneal to the true posterior. By integrating along these posterior time- ttre-magginals, DiME accu- rately estimates the model evidence using intermediate samples already generated during posterior sampling. D ME thus only requires small number of posterior samples (e.g.. 20) for accurate esti- mation. We also describe how to practically implement D DiME alongside a state-of-the- posterior sampling method Decoupled Annealing Posterior Sampling (DAPS).
Our contribution:
We perform several experiments validating our method. We first test in a mixture of Gaussian setting where an analytic evidence can be derived, and find that DiME provides nearly unbiased estimates of the evidence and performs comparably to basclines sequential Monte Carlo, annealed importance sampling, and thermodynamic integration despite not using the prior score. We also test Di ME. on two non-convex inverse problems, Gaussian and Fourier phase retrieval. D DiME consistently selects the correct prior from set of 10 diffusion models trained on MNIST (Deng) (012) digits given single noisy measurement of one MNIST digit, while the baselines fail to do so.
Finally, we use our method perform both model selection and validation on real M87* black hole observations from the Event Horizon Telescope (EHTC) 2019a b). DiME indicates that a prior derived from synthetic black hole images generated using Seneral-Relativisti namics (GRMHD) (Mizuno) (022) has higher likelihood compared to the Radiatively Inefficient Accretion Flow (RIAF) (Broderick et al. 201T) black hole prior, prior trained on general space images (Alam et al., 2024). a prior trained on faces (Liu et al. 2015). and a prior trained on MNIST digit Os. Furthermore, we diagnose the validity of the GRMHD prior by prior predictive checking on the model evidence. Our results suggest that the M87* observations are statistically in-distribution with respect to the GRMHD prior, while still leaving room in the model for refinement.
Magnetohydrody.
BACKGROUND
2.1 DIFFUSION MODELS
Given a clean image Fg ~ Po (zo)) the diffusion forward process at time t [0, T) is given by:
x; = dgEo + dižt, zt ~
~N I)
(1)
where a, is the signal scaling, de is noise scaling, and af, of are the time derivatives. We can sample from the clean image distribution by discretizing the reverse SDE:
d x; - g¹º Va, lo: dt + ge dw,
dx;
(2)
at
2

---

Published as a conference paper at ICLR 2026
where gf = (0l0; df wy is Brownian motion and
a,
Ve, log ((())) ) learned. We can obtain the posterior mean via Tweedie's formula (Efron 2011
1 ax
2 + of Vax log pog)(
2.2 DIFFUSION-BASED POSTERIOR SAMPLING METHODS
Most diffusion posterior sampling methods fall broadly under two categories: (1) explicitly approx- imating the intractable likelihood score V æ, log (3 (t) as guidance during reverse diffusion, and (2) methods that anneal along a path of specified intermediate marginals, using techniques such as sequential Monte Carlo (Cardoso et al., 2023 Wu et al., 2023), gradient-based optimization (Mo- ufad et al. 2024). or Langevin dynamics (Zhang al. 2025] Wu al. 2024 Zhu al. (023). The approximation errors of (1) compound over the sampling process, resulting in intermediate time- marginals with no tractable form and a biased posterior, while (2) corrects these errors by adhering anneal to the true posterior. As
to specified marginal path, allowing for their result, in this work we focus on two different sampling methods of (2) that have demonstrated strong performance for solving physical inverse problems (Zheng et al. 2025)
Concretely, given measurements from an inverse problem y = A(x) + e we introduce two likelihood terms. The first, p(y at)) f p(y zo)p(aro 22 ) is the marginal likelihood of conditioned on the current diffusion state ot. The second, q(3) xx) == p(y 2o xx) is
time-independent data likelihood term that assumes æ; is the clean image.
The Decoupled Annealing Posterior Sampling (DAPS) method (Zhang et al. 2025) alternates be- tween noising and sampling from the posterior. Given initial noise samples FT .O I) and
N Ntstep annealing schedule [t,,...,ty], .ty. we sample from the clean image posterior using Langevin dynamics before re-adding noise according to the diffusion process:
To p(x0 Fiay 4) x p(y) zo)p(<z xtt))
(4)
-t--1 p( Tb)
(5)
Zhang et al. (2025) proposes two approaches: Gaussian approximation DAPS, where p(æo xe)
approximated via Gaussian (V az). 2 and the covariance Σz, le, heuristically chosen as of. and exact DAPS, where the prior score is computed using the diffusion model tiny noise level V log p(z.) = V log p(æ, min)) The unconditional distribution of intermediate samples closely match the standard marginals: Dt ~ p(æ; y) X p(y) (x)))()))))))))))))))))))))))(
for all times
The Plug-and and-Play Diffusion Models (PnP-DM) sampling method e et al. 2024) takes inspiration from the Split Gibbs Sampler (Vono al. 2019) and related diffusion-basee samplers (Coeurdoux) et al., 2024) by splitting each annealing iteration into likelihood step and a prior step. In partic- ular, given -sstep annealing schedule [d.,..., ......,N! tn] and clean image initialization æo. we alternate between running Langevin dynamics to obtain sample from the next mire-nar posterior and
using the diffusion model to sample a clean image. We rewrite the updates for one timestep tx in terms of standard diffusion notation:
Tte ~ g(x y, 2o) xx g(vl Tzg ()x))(( (0))
(6)
æo p( xt.))
where æty is sampled from joint distribution of the noised do and the time-independent likelihood q. We write g(y are) to emphasize this proxy likelihood applies the measurement model directly to noisy rather than clean image.
We refer to the resulting unconditional distributions of intermediate samples as PnP-DM marginals, denoted by H ~ q(x, y) g(y (x))((;)) for all t.
2.3 EXISTING MODEL EVIDENCE ESTIMATORS
The simplest estimator approximates the evidence by averaging p(y x) over samples x ~ p(x), though this requires exponentially many samples in high dimensions. The standard harmonic mean
3

---

Published as a conference paper at ICLR 2026
estimator (Newton & Raftery| [1944) mitigates this issue but has infinite variance. Learned harmonic mean estimators (MeEwen et 2021 Polanska et al. 2023 Spurio Mancini et al. 2023) stabilize the variance by importance sampling with surrogate neural density, under the assumption that the prior density is known. However, surrogate training requires thousands of posterior samples and importance sampling requires similar amount of density evaluations even in mediun-dimensional settings, making these approaches infeasible for diffusion-ben based priors.
Nested sampling (Skilling 2006 Cai et al. 2022) re-parametrizes the model evidence into a one- dimensional integral over the prior volume, but still requires the score or density and is limited to logvconvex likelihoods. Thermodynamic integration (TI) (Lartillot & Philippel 2006), annealed im- portance sampling (AIS) (Neal) 2001) and sequential Monte Carlo (SMC) (Del Moral et al.) (0066) compute the evidence by integrating over path of distributions ps, with 0 3 1. For diffusion priors, accurately sampling from arbitrary (æ) is computationally difficult as we lack clean prior score. Reverse diffusion Monte Carlo (RDMC) (Huang et al.) 2023) methods simulate the diffusion process to sample from unnormalized densities, using a Monte Carlo approach to approximate the noised scores. Guo et al. (2025) independently derived an evidence estimator for the reverse dif- fusion process under very different assumptions: they rely closed-form posterior score, obtain exact posterior marginals directly from an analytic diffusion that is not learned, and provide only theoretical convergence results without experimental validation.
ESTIMATING THE MODEL EVIDENCE ALONG THE STANDARD MARGINALS
We introduce DiME to estimate model evidence from the standard marginals p(æ, y), aligning closely with the annealing path of DAPS. All proofs in this section are shown in Appendix A A general model evidence estimator for arbitrary marginals annealing the true posterior is derived in Appendix B and an unnormalized model evidence estimator that instead uses the PnP-DM marginals (DiME PM) is derived and implemented in Appendix B.T
Proposition 1 (DiME: Model evidence estimation along the standard marginals). Given diffusion process D = au + 2t 2, ~ (0, I), posterior marginals p(æ; y) x p(xx) J p(y | zo)p( zn z, ),), and timesteps 0 = to < tN T, the model evidence can be estimated via:
log Eporol Iwvl log ply
x0)
DrL (p(zo y))l((z))))
(8)
The log- likelihhodd term can be estimated with posterior samples and the KL divergence with:
N
DKL (p(zo vv)ll((z))) =
Σ Ca. At,,E
~plas ~plas, (w) Vay, logp(y xx,))
where Cz2 0f,00u oz
d/z as,
is dependent on the diffission schedule and At; = t; f;-1.
The sum across each sample-path Jim,J....... can be viewed as the distance of the resulting pos- terior sample Do to the prior. In the remainder of the section, we showcase how to practically implement DiME in tandem with the DAPS method but with two modifications: in Section 3.1 design an improved covariance approximation for p( xx) if the Gaussian approximation DAPS is used, and in Section 3.2 we propose sample two ätn p( 24,, y) for each 2; estimate ||V,,, an log log 2²2 needed for Eq. 9 The full algorithm can be seen in Algorithm
3.1 AN IMPROVED APPROXIMATION FOR THE POSTERIOR COVARIANCE
While the exact DAPS sampler can be used for accurate sampling of To p(#n I x.,8) x p(y Zo to)o(( a.)) (Eq. 4h, Gaussian approximation DAPS provides significantly cheaper alternative by approximating p(ag a..) (EEaa, (x).o²)) However, this covariance heuristic only ac- counts for p(æ; æo) and ignores p(zo). While this heuristic is accurate at low noise levels,
For likelihonds with Gaussian noise, if the posterior and prior have enough overlap such that posterior samples have a data x² 1, this term essentially becomes constant under the same likelihood function.
Whhile approximating p( ax) is necessary for the standard marginals annealing path, it is not necessary for many annealing paths under the generalized framework introduced in Appendix
4

---

Published as a conference paper at ICLR 2026
In Distribution
Out of Distribution
B|Ko I Xe] Go Gugitta
Large
(Fo)
diloat Limesley
Small
R6 (B0
E-0
Legend:
p(x)
pllzal
By 3
estimators F2 log((rlss)
Figure 1: Visualization of the unbiased estimators rhig, tow of the likelihood score Va, log p(y 2x) described in Lemma2for both in-distribution (left) and cut-of-distribution (right) measurements y. At large diffusion time steps, the high noise estimator (top) uses the distance between Øo and E[æo æz). At small diffusion time steps, the low noise estimator (bottom) uses the likelihood score øø. The estimator larger (longer arrows) when y is disttiiiiiiiitinnnnn as there greater KL divergence from the posterior to the prior; this results in DiME computing lower evidence. Note the scaling factors of and Deala of the estimator are not shown.
overestimates the variance in the high noise regimes: at the terminal noise level f = T, we expect p(ao ( = p( (D)) with covariance that should match the prior Cov(zv), but the heuristic gives Cov(an Dog FT o > Cov( This mismatch at high noise was also observed empirically in Zhang al. (2025). Since DiME uses all time-marginals it is important accurately estimate p(aco 20 2c, for all I. Therefore, we include knowledge of p(æ:o) by approximating it as Gaussian N N(p, Eo). using empirical covariance computed from the training data similar to the covari- ance approximation introduced in Linhart et al. (2024). Introducing an approximate prior provides more accurate representation of p( 22), which remains a Gaussian approximation as it is now the product of two Gaussians. The resulting covariance is given by:
Lemma 1. Suppose we have diffusion process æt 0,30 + Oi 2t ~ (O, I). Let æs p(æ+) and suppose p(æo) se N(Ho, Eo). Then we can express the posterior covariance Dayjas ln as
Ezgla la:
E
I
(10)
For large images where computing full covariance matrix is intractable, diagonal approximation in a transform basis has also been demonstrated to work well (Peng et al., 2024). and for linear inverse problems, analytic local covariance is computable (Boys et al. 2023). Our approximation aligns the annealing path more closely with p(æ, y) for all t. is worth noting that DAPS re-samples every annealing step, ensuring that these Gaussian approximation errors never compounded. As shown in Sections/44TJand44.... using the proposed improved approximation leads nearly unbiased evidence estimates for both multimodal Gaussian toy example with varying local curvature as well as real-world scientific priors.
3.2 ESTIMATING THE SQUARED LIKELIHOOD SCORE
While directly computing Va, log p x;) is intractable, DiME only requires an unbiased estimate as seen in Eq. 9 Estimators using unconditional samples of æp æ, are high-variance: these samples usually have low likelihood p(y an) resulting volatile scores. Thankfully, DAPS provides samples of æo p(zo It, y). which
use to create estimators with lower variance:
Lemma 2. Suppose we have diffusion process I, = 0, + D, 2, ~ (0, and sample io ~ p( 3t, y)- Under the assumption that p(æ: ax.) R A/ pr æ.). Exole.) both Ohigh (wo)

---

Published as a conference paper at ICLR 2026
and (Zo) are unbiased estimators
V W: log xx))
with
4t (To Elwo æts)
thigh (To)
(11)
1
Ot
Low (To)
Emole V logp(y Tp))
(12)
oi
Under low diffusion noise (at + 1, + 0), Var(Olow) + bus Var( = 0 (² Under high noise (u, - 0, + 1), Var(Onigh) + but Var(@ = 0 Var(Onigh)
A proof of these bounds is displayed in Appendix A.1 The lack of the measurement in the high noise estimator bounds the worst possible case, where the data is weak; when the measurement con- tains a lot of information, the variance would be much lower. As such, r is used more towards the beginning of the sampling process while ①ow is used towards the end. As both estimators are cheap to compute, we can simply evaluate both and choose the lower variance estimator at each timestep. Finally, as we want an unbiased estimate of the squared likelihood score, squaring the estimator is biased: ||| = ||O|| Tr(Cov(O)). Therefore, for each intermediate sample Tt, we propose to sample two i.id 2 2 ~ p(zu It, y), giving us the following unbiased esti- mator: (a)) (*) (eeee")TT"O To(=))))))))))))))))))))))) The full method displayed in Algorithm [ and visualization for both in-distribution and out-of-distributie measurements is visualized in Figure]] !
Algorithm One posterior sample-path of DiME implemented with DAPS
Require: Score-based model se, measurement y, noise schedule de (t.)ie(1,....N))
1: Sample Bty (/0 of I)-
2: Initialize the integral sum, f =
3: for N, N 1,, .,... do
4: Compute & = E æe.] using Tweedie's formula and sa. p(2o ot, 3) using Langevin with p( xx)) -át E m. 3.1
5: Sample 2b 2
(2)
6: Sample Tt,-1 Vat.. 1 1)
7: fi arg min (6 Aright Var(O((#)) ( Te(=! computed using all sample-paths.
3.2
8: Update the integral sum I + + (1i 11-1) (0/0--07 oz?
of ) St.
9: end for
10: return To. ff + logp(y xo)
4 EXPERIMENTS
We compare DiME and DiME-PnPDD with five baselines: naive Monte Carlo (Naive MC), thermo- dynamic integration (TD), annealed importance sampling (AIS), Sequential Monte Carlo (SMC). and the original DAPS covariance heuristic (Original DAPS heuristic). Baseline methods are described in Appendix C and all tuning and implementation details are provided Appendix
4.1 BENCHMARKING ON GAUSSIAN MIXTURE PRIOR WITH GROUND TRUTH EVIDENCE.
To verify DiME, we first test on multimodal 1000-D Gaussian mixture prior and linear forward model Ax + E, ~ (0, o² where the model evidence has an analytic form to compare to. We use two Gaussian components with means at (-0.75. 0.75) and identical covariances of 0.25I. The entries of A E 20 1000 are i.i.d. (O, 200 ), and = 0.1. We run three experiments where the ground truth æ² has varying model evidence to test the robustness our method: in-distribution 2", out-of-distribution ø", and x" 0 located at saddle point. The analytic V. log p(æ:) is used to sample the intermediate distributions for TI, AIS, and SMC. In contrast, for DiME the analytic Vz, log p(æ;) is used to compute E[x,, æx] for methods are tested using 100 discretization steps (equivalently, annealing time steps for D DiME) and 20 sample paths for 50 trials. The mean and standard deviation is reported.
0.05 and the analytic score is never used. All

---

Published as a conference paper at ICLR 2026
In-distribution at'
Ont-of-distriaation it
Saddlo point
Estimate
Rel.
Fatimate
Estimate
Rel. Enor
Ground truth log
128.7
1680.2 +
251.3 + 0.0
0.0
Naiee MC (1000 sampless)
3283 + 130
2461%
-41289 + 677
2357%
6022 + 1.70
2299%
Naive (10000 samples)
-3041 + 113
2263%
-40123 + 487
2288%
-5666
144
2133%
Original DAPS heuristie
316.2 + 79.5
146%
-1735.4
4.7
3.3%
-20..6 3.3
7.3%
II
132.8 +
3.2%
1773.8 + 11.8
5.6%
-254.2 3.2
1.2%
AIS
1135.9
2.5
5.8%
-1754.8
7.1
4.4%
-259.1
2.9
3.1%
SMC
132.1
2.8%
-1701.1
D.8
1.2%
-233.0
+ 3.4
0.7%
DINE
126.8 2.8
1.6%
1674.8
2 4.7
0.3%
253.3
=2 3.1
0.8%
DANE-PAPOM
2.4%
-1912.1
6.5
13.8%
3.6
2.1%
Table 1: Evidence estimates on Mixture of Gaussians study with analytic ply) and ground truth x:" in-distribution (top), out-of-distribution (middle), and saddle point (hottom). Mean and standard deviation of 50 trials displayed each with 20 sample paths. When the analytic prior score is available, DiME performs comparably to the baselines for all cases, while AAME--PPDD performs well when the ground truth is roughly in in-distribution. *Important: as DiME-PnPDe outputs an unnormalized estimate, we normalize via the analytic constant factor for ease of comparison.
Results are displayed in Table DiME gives nearly unbiased estimates of the evidence in all cases, with comparable performance to the strongest baseline (SMC), despite never using the true prior score. To probe the effect of the covariance choice discussed in Sec. we first consider the case where x= lies within a prior mode (in-distribution). Here the original DAPS heuristic consistently overestimates the variance of the marginals, pushing posterior samples into the wrong mode and inducing a large bias in the evidence estimate. Even when x" is outside any single mode, the original heuristic continues incorporated in DiME, always generates samples from the correct mode and eliminates this bias, when the true x² lies in regions of local curvature far from the unimodal Gaussian assumption.
overestimate the evidence. In contrast, our improved covariance heuristic,
Unlike DiME, DiME-PMPDM only performs well for roughly in-distribution tasks. For out-of- distribution measurements y. while the PnP-DM marginals asymptotically anneal to the correct posterior, we find that in the finite-step case this annealing path results in greater bias than DAPS. Because out-of-distribution measurements frequently arise in model selection, we conclude that PnP-DM is less well suited for this task.
4.2 MODEL SELECTION USING LEARNED SCORE ON NON-CONVEX INVERSE PROBLEMS
We test both DiME and the SMC baseline on two non-convex inverse problems, Gaussian phase retrieval and Fourier phase retrieval. In Gaussian phase retrieval, the forward model is y |Ax|+e where A E Cmxn consists of i.idd Gaussian (0, 1) and & ~ N o² o2)). In Fourier phase retrieval, the forward model is y = |Fa| + E, No0,, o² o² where F the discrete Fourier transform. In our experiments, many more measurements, the structured forward operator means that both translation and flips of invariances under the forward model. We test with diffusion model priors, one on each MNIST (Deng 2012) digit, and compute the evidence for each prior given a measurement of each digit. For DiME, we use 50 annealing iterations and 20 generated sample-paths for each entry. For SMC, we use the learned V log p(æ;) as surrogate prior score at different low noise levels.
m 0.1n and
0.3. While Fourier phase retrieval has
DiME results are shown in Figure 2 and SMC results displayed in Appendix E For both inverse problems, DiME always selects the correct model given a single noisy measurement y, while SMC often fails to do so, demonstrating that methods relying on a clean prior score are not suitable for iestusion-based model selection. For Gaussian phase retrieval, DiME predicts higher likelihoods of digits that look similar, such as 4 and (Figure bottom left, third row). For Fourier phase retrieval, DiME predicts high evidence for the MNIST 9 prior when the ground truth image is (and vice versa), but as both vertical and horizontal flips are invariances under Fourier phase retrieval, performing these operations on a 6 results in a 9 (Figure 2 bottom right, second row). The normalized evidence can also provide useful insights: as the MNIST digit prior has higher evidence for in in-diistribution y and lower evidence for -fstrrribbitiioon y. it must be narrower prior. On the other hand, the MNIST 3, 5 and 8 priors have higher out-of of -istiriibtttoon model evidence, suggesting that they wider priors, likely due to the variation in which humans write these digits.

---

Published as a conference paper at ICLR 2026
Model Evidence: Gaussian Phase Retrieval
Model Evidence: Fourfer Retrieval
2007
489-10
402
-229 4953
4009 41B 499 458
0
4933
464
444
399
1270
-215 -285
400
430
245
2300
-267
-299
-335
2 <3000
-001
600
415
420
4000
437
-000
200
090
1213
244
150 182
408
KEB0
202
438
<39
muth
:351
-40
223
130
4508
120
300
800
Ground
-200
-291
-2:00
100
-100
-200
-11
-400
-636
-838
4100
+021
-137
110
1000
4913
-008
-185
-88
199
459
1086
326
209
-600
4696
118
-22
+237
118
203
-200
290
+30
192
+290
7
+308
+53
221
+999
-376
+008
4088
-44
1027 +160
4105
01.0
160
4103 403
400
135 -953
488
4,03
1200
755
48000
920
480
0
Mode
Model
samples:
6T0
O
GE
O
O
GTS
6
7
5
G
6
9
6
9
9
9
9
9
ModelS
Model 9
9
9
9
A
4
4
X
+
X
X
9
Model4
Figure 2: Model evidence confusion matrix for Gaussian phase retrieval (left) and Fourier phase retrieval (right) for each (ground truth measurement, model) pair of MNIST digits. Our method selects the correct model for all cases. Posterior samples shown for dotted matrix entries below. For Gaussian phase retrieval, DiME estimates higher model likelihood for visually similar digits, such as 4 and 9. For Fourier phase retrieval, both translations and reflections are invariances as seen in the posterior samples, so DiME estimates a high likelihood of model 9 given a measurement of a
4.3 EVALUATING PRIOR MODEL EVIDENCE FOR REAL M87 BLACK HOLE DATA
In this section, we perform model selection and validation on real black hole M87* observations from April 6, 2017 EETT 2019abb) b). The need for such analysis was first emphasized during the release of the initial M87* images by the Event Horizon Telescope Collaboration (EHTC (019d), where the goal was not only to reconstruct images but also to test which simple parametric models best matched the data particularly whether a ring provided the most suitable description (EHTC, 2019d, BBroderick et al.] 2020). In addition, the team directly compared the M87* data to discrete set of synthetic black hole simulations (GRMHD) to assess which physical model parameters were most consistent with the observations (Fromm al. 2019 EHTC c019d]c). However, noted in EHTC (2019d), these conclusions are only valid to the extent that the selected simulations provide an adequate description of M87 question that can be answered by our proposed method.
Measurements, known as visibilities, are obtained by combining signals collected by telescopes observing simultancously at different locations around the world. The forward model for complex visibilities between two telescopes i, j is Vajj g.ggee((0 (o + Ei.j where vf,, is the true visibilities measurements, 9i is a station-dependent gain error, Φ; is a station-dependent phase error arising from atmospheric noise and E.,j is Gaussian thermal noise. To become invariant to the phase error, we use the closure phase Yep = 2i Py,.DKk, for a minimal set of 3 telescopes (i. k) and for similar invariance to the gain error, we use the closure amplitude yea 2L. for minimal set e.
of 4 telescopes (i.j. j.j. k. () (Thompson et al. 2017 Blackburn et al. 2020 We use the InverseBench (Zheng et al., 2025) library to compute the likelihood. We generate 20 posterior samples using 50 annealing steps and 1000 Langevin dynamics steps per iteration. Gaussian approximation DAPS takes around minutes to compute, while exact DAPS takes around 30 minutes to compute for the same parameters, speedup of around 7x.
8

---

Published as a conference paper at ICLR 2026
Exact DAPS
Gaussian Approximation DAPS
250
230
200
230
Esicbreaa bagpit
1:0
Evidercc log
1:0
150
120
plyl Distribution ply) estimates
Distribution log estimates individuall pestericesenn
335
indrrnduall postier
Figure 3: Model evidence estimates on real M87* observations across 5 different priors using exact DAPS (left) and Gaussian approximation DAPS (right). Our method concludes that, these prior models, GRMHD is the most likely model. The violin plots show the distribution of evidences from 20 different posterior-sample paths from the DAPS sampling process, and the mean the overall evidence estimate. Gaussian approximation DAPS gives highly accurate evidence estimates with 7x less compute than exact DAPS, but with slightly higher variance.
Exact DAPS
Gaussian Approximation DAPS
Prior Samples
Mean
Fatt inia/msanimani
Fath eviciences
GRMHD
Spocohett
Prior
76
-255
-95
-21
Figure 4: M87 reconstructions using the priors using exact DAPS (left) and Gaussian approxi- mation DAPS (middle). While all posterior samples have data fit of around reduced x" ~ 1, they have varying path evidence estimates (shown in the top left of each image), which are negatively proportional to the KL divergence between that posterior sample and the prior. Right: Example unconditional samples from each prior.
4.3.1 MODEL SELECTION
We trained five candidate diffusion priors: (1) on 48000 Generra-Relativistie Magnetohyyrodyynmii
(GRMHD) simulations (Mizuno, 2022), (2) on 9070 Radiatively Inefficient Accretion Flow (RIAF)
simulations (Broderick al. 2011). I). (3) on 1 11000 general space images (SpaceNet) (Alam et al. 2024), (4) on 5923 MNIST (Deng) 2012) digit 0 images, and (5) on 48000 radially tapered CelebA Liu et al. (2015) images (Faces). Additionally, we augment our data during training to improve generality and robustness of our priors. The GRMHD images are randomly flipped horizontally and/or vertically and zoomed in and out in the range [0.75, 1.25]. The RIAF images were randomly flipped, zoomed in and out in the range (0.75, 1.25), and rotated uniformly in the range [0. 2m). The MNIST digit 0 images were zoomed in and out in the range [0.5, 1].
Model selection results for the real M87* data can be seen in Figure 3 The reconstruction and posterior samples with the minimum, mean and maximum evidence (negatively proportional to KL divergence prior) shown in Figure 4 Synthetic tests validating the method on this problem

---

Published as a conference paper at ICLR 2026
Ground truth
Moan image
Puth evidencee janvimmasimme
observations morphed
Gauss t0
25077L e223.06)
Applo Morghed
denuty for in-dlertb.oor)
eople NUAF
0.02
pigeon
0:00 image
RULF
Pigaon
0.00
50
100
150 200
250
300
ply!
Figure 5: Left: GRMHD model validation results on M87 observations by comparing to evidence of in-distribution measurements y. Our method shows that the evidence of M87* observations have z-Score of about -0.81 compared to the evidence distribution of GRMHD measurements, indicating that M87* is statistically in-distribution of GRMHD. The evidence of simulated measure- ments of out-of-distribution images are also shown, demonstrating that measurements from out-of- distribution (OOD) images have OOD evidence. Right: Mean reconstruction and posterior samples of the OOD images with the highest and lowest distance the GRMHD prior. The blurred ground truth image, or the maximum resolution that can be obtained from measurements, is also displayed.
are shown in Appendix DiME determines that, for static imaging, GRMHD is the most likely prior for the M87* observations, followed by SpaceNet, Faces, MNIST, and finally RIAF. While SpaceNet contains few black hole images in its training data, method penalizes it for being too general, as demonstrated with its wide range of unconditional samples (Figure 33 right, second row). The RIAF model is very narrow prior that enforces strong assumption holes, so any out-of-distribution images (such as M87* as determined by our method) will have low likelihood. We also find that Gaussian approximation DAPS gives nearly identical estimates exact DAPS with 7x less compute.
the structure of black
4.3.2 MODEL VALIDATION
We also diagnose the validity of the GRMHD model for static imaging of the M87* observations by computing the evidence p(y). We simulated measurements from 100 GRMHD test images using the same forward model as the EHT observations of M87*, and then computed the evidence for these measurements. Under a central limit theorem argument, we can approximate this distribution as Gaussian. We then compared the evidence of the true M87* observation to this Gaussian as well the evidence of GRMHD prior is valid for M87*.
measurements under the same forward model to assess if the
Results are shown in Figure The fitted Gaussian has μ 250.71 and 23.86. M87* has an evidence of 231.46, giving a z-score -0.81 and p-value of 0.209 with respect to the previous Gaussian fit. Therefore, M87* likely lies within the support of the GRMHD image prior, offering credence to our current physical model of black holes through the images it produces, while still leaving open the possibility of improved model. On the other hand, the morphed GRMHD image has evidence 210.37, giving it a z-score of -1.69 and p-value of 0.046, suggesting that it is not in the true GRMHD distribution. The apple, RIAF, and pigeon images have even lower evidences of 176.52, 150.41, and 74.68 respectively and are all considered out-of--istributie
CONCLUSION
We introduce D DiME, method that estimates the Bayesian model evidence when solving inverse problems using diffusion model priors. DiME computes the KL divergence between the posterior and prior by integrating along posterior time-marginals, a task made easy by using the intermediate samples of recent diffusion posterior sampling methods. Empirically, DiME outperforms all baseline methods and we demonstrate how DiME can be used on a real-world black hole imaging problem. Altogether, DiME makes it possible to hamness diffusion priors not only for reconstruction but also for principled model selection and validation, laying the foundation for more reliable inference in scientific imaging.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our method statistical estimator and can occasionally select the incorrect model, potentially leading to incorrect conclusions. Furthermore, while it could in principle be applied to sensitive characteristics (e.g., facial recognition), we do not support its use in this context.
REPRODUCIRILITY STATEMENT
All code and trained models will be made available. All mathematical derivations are detailed in the appendix. The MNIST dataset is available through torchvision (Marcel & Rodriguez, http://wwaaaa...........ee 2010), the CelebA dataset is available https:/mmmla.iie.. the SpaceNet dataset is available at https//www..aggge.cooo//ttttttts opiimall-distribueed-sstoooon for public use.
and
The RIAF and GRMHD datasets are not currently available
ACKNOWLEDGMENTS
This work was supported by NSF Award 2048237, an Amazon Al4Science Discovery Award, Ope- nAL, and Sloan Research Fellowship. FW is supported by Kortschak Fellowship. The authors would like to thank Angela Gao for helpful discussions.
REFERENCES
Mohammed Talha Alam, Raza Imam, Mohsen Guizani, and Fakhri Karray. Flare up your data: Diffusion-biai basedd augmentation method in astronomical imaging, 2024.
Lindy Blackburn, Dominic W Pesce, Michael D Johnson, Maciek Wielgus, Andrew A Chael, Pierre Christian, and Sheperd S Doeleman. Closure statistics in interferometric data. The Astrophysical Journal, 894(1):31, 2020.
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and O Deniz Akyildiz. Tweedie moment projected diffusions for inverse problems. arXiv preprint arXiv:2310.067211 2023.
Avery E Broderick, Vincent L Fish, Sheperd S Doeleman, and Abraham Loeb. Evidence for low black hole spin and physically motivated accretion models from millimeter-ilbi observations of sagittarius The Astrophysical Journal, 735(2):110, 2011.
Avery E Broderick. Roman Gold, Mansour Karami, Jorge A Preciado-Lopez, Paul Tiede, Hung- Yi Pu, et al. Themis: A parameter estimation framework for the event horizon telescope. The Astrophysical Journal, 197()): 2020.
Xiaohao Cai, Jason D McEwen, and Marcelo Pereyra. Proximal nested sampling for high-
dimensional bayesian model selection. Statistics and Computing, 32(5):87, 2022.
Gabriel Cardoso, Yazid Janati Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided diffusion for bayesian linear inverse problems. arXiv preprint (tt.Xi::23880799300 2023.
Junhua Chen, Lorenz Richter, Julius Berner, Denis Blessing, Gerhard Neumann, and Anima Anand- kumar. Sequential controlled langevin diffusions. arXiv preprint arXiv:24/2.0708/ 2024.
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Mare L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209. 14687, 2022.
Florentin Coeurdoux, Nicolas Dobigeon, and Pierre Chainais. Plug-and-pls split gibbs sampler: embedding deep generative priors in bayesian inference. IEEE Transactions on Image Processing, 33:3496-3507,- 2024.
Pierre Del Moral, Amaud Doucet, and Ajay Jasra. Sequential monte carlo samplers. Journal the Royal Statistical Society Series B: Statistical Methodology, 68(3):41 1-36,, 2006.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web). IEEE signal processing magazine, 29(6): 141-)1441142. 2012.
11

---

Published as a conference paper at ICLR 2026
Arnaud Doucet, Will Grathwohl, Alexander G Matthews, and Heiko Strathmann. Score-based diffu- sion meets annealed importance sampling. Advances in Neural Information Processing Systems, 35:21482- 244944 2022.
Bradley Efron. Tweedie's 's formula and selection bias. Journal of the American Statistical Associa- tion, 106(496):1602-1611 2011.
EНTC. First m87 event horizon telescope results. i. the shadow of the supermassive black hole. arXiv preprint arXivv1906. 1238, 2019a.
EHTC. First m87 event horizon telescope results. iv. imaging the central supermassive black hole. arXiv preprint arXiv: 19066v/900.11 /241, 2019b.
EHTC. First m87 event horizon telescope results. v. physical origin of the asymmetric ring. arXiv preprint 1rxXiv:906.112422 2019c.
EHTC. First m87 event horizon telescope results. vi. the shadow and mass of the central black hole. The Astrophysical Journal Letters, 875(1):L6, 2019d.
Berthy Feng, Katherine L Bouman, and William Freeman. Event-horiizn-scall imaging of m under different assumptions via deep generative image priors. The Astrophysical Journal, 975(2):201, 2024.
CM Fromm, Z. Younsi, A Baczko, Y Mizuno, O Porth, M Perucho, H Olivares, A Nathanail, E An- gelakis, Ros, et al. Using evolutionary algorithms to model relativistic jets-application to ngo 1052. Astronomy & Astrophysics, 629:A4, 2019.
Wei Guo, Molei Tao, and Yongxin Chen. Complexity analysis of normalizing constant estima- tion: from jarzynski equality to annealed importance sampling and beyond. arXiv preprint Drxiv:2502.04575. 2025.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Xunpeng Huang, Hanze Dong, Yifan Hao, Yi-An Ma, and Tong Zhang. Reverse diffusion monte carlo. arXiv preprint arXiv:2307.020377 2023.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G Dimakis, and Jon Tamir. Robust compressed sensing mri with deep generative priors. Advances in neural information processing systems, 10:14938--4954. 2021.
Nicolas Lartillot and Hervé Philippe. Computing bayes factors using thermodynamic integration. Systematic biology, 55(2): (952))195-207) 2006.
Xiang Li, Soo Min Kwon, Shijun Liang. Ismail R Alkhouri, Saiprasad Ravishankar, and Qing Qu. Decoupled data consistency with diffusion purification for image restoration. arXiv preprint arXiv:2403.06054 2024.
Julia Linhart, Gabriel Victorino Cardoso, Alexandre Gramfort, Sylvain Le Corff, and Pedro LC Rodrigues. Diffusion posterior sampling for simulation- based inference in tall data settings. arXiv preprint arXiv:2404.07599. 2024.
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Sébastien Marcel and Yann Rodriguez. Torchvision the machine-vision package of torch. In Pro- ceedings of the 18th ACM international conference on Multimedia, pp. 1485-1488, 2010.
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. variational perspective on solving inverse problems with diffusion models. arXiv preprint 0431v:23055.0431,, 2023.
Jason D McEwen, Christopher GR Wallis, Matthew A Price, and Alessio Spurio Mancini. Machine learning assisted bayesian model comparison: learnt harmonic mean estimator. arXiv preprint arXiv:2111.122200 2021.
12

---

Published as a conference paper at ICLR 2026
Yosuke Mizuno. Grmhd simulations and modeling for jet formation and acceleration region in agns. Universe, 8(2)855, 2022.
Badr Moufad, Yazid Janati, Lisa Bedin, Alain Durmus, Randal Douc, Eric Moulines, and Jimmy Olsson. Variational diffusion posterior sampling with midpoint guidance. arXiv preprint arXiv:2410.09945 2024.
Radford M Neal. Annealed importance sampling. Statistics and computing, 11(2):125-13 139, 2001.
Michael A Newton and Adrian E Raftery. Approximate bayesian inference with the weighted likeli- hood bootstrap. Journal of the Royal Statistical Society Series B: Statistical Methodology, 56(1):
3-26, 1994.
Xinyu Peng, Ziyang Zheng, Wenrui Dai, Nuoqian Xiao, Chenglin Li, Junni Zou, and Hongkai Xiong. Improving diffusion models for inverse problems using optimal posterior covariance. arXiv preprint 4rX11:2402.02149, 2024.
Alicja Polanska, Matthew Price, Alessio Spurio Mancini, and Jason D McEwen. Learned har- monic mean estimation of the marginal likelihood with normalizing flows. In Physical sciences forum, volume 9, pp. 10. MDPI, 2023.
John Skilling. Nested sampling for general bayesian computation. 2006.
Marta Skreta, Lazar Atanackovic, Avishek Joey Bose, Alexander Tong, and Kirill Neklyudov. The superposition of diffusion models using the it' o density estimator. arXiv preprint arXiv:2412.17762, 2024.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guiee diffusion models for inverse problems. International Conference Learning Representations, 2023.
Yang Song. Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint WwXiv:2011.134555 2020.
Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances neural information processing systems, 34:1415- 1428, 2021a.
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. arXiv preprint arXiv:2111.08005 2021b.
A Spurio Mancini, MM Docherty. MA Price, and JD McEwen. Bayesian model comparison for simulation-based inference. RAS Techniques and Instruments, 2(1):710-722, 2023.
Andrew M Stuart. Inverse problems: bayesian perspective. Acta numerica, 19:451-559, 2010.
Richard Thompson, James M Moran, and George W Swenson. Interferometry and synthesis in radio astronomy. Springer Nature. 2017.
Roberto Trotta. Applications of bayesian model selection cosmological parameters. Monthly Notices of the Royal Astronomical Society, 378(1):72-88) 2007.
Maxime Vono, Nicolas Dobigeon, and Pierre Chainais.
Split-and-augmennted gibbs sam- appliccatiiionn to large-scale inference problems. IEEE Transactions on Signal Processing, 67(6):1648-16611 2019.
Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36:31372-31403 2023.
Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, and Katherine Bouman. Princi- pled probabilistic imaging using diffusion models as plug-and-play priors. Advances in Neural Information Processing Systems, 37:1 17:118889. 11889-1188427 2024.
13

---

Published as a conference paper at ICLR 2026
Bingliang Zhang, Wenda Chu, Julius Berner, Chenlin Meng, Anima Anandkumar, and Yang Song. Improving diffusion inverse problem solving with decoupled noise annealing. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
2025.
Hongkai Zheng, Wenda Chu, Bingliang Zhang, Zihui Wu, Austin Wang, Berthy T Feng, Caifeng Zou, Yu Sun, Nikola Kovachki, Zachary Ross, et al. Inversebench: Benchmarking plug-and- play diffusion priors for inverse problems in physical sciences. arXiv preprint arXiv:2503.1 11043, 2025.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1219-1229 2023.
14

---

Published as a conference paper at ICLR 2026
A
PROOFS
Although the main the diffusion SDE formulation, introduce the Probability Flow ODE (PF-ODE) for the proofs and for Appendix B This ODE has identical marginals as the SDE under ideal conditions. Given diffusion process It 0,20 + oi #t NO, (, I). the PF-ODE is:
[aj az dx.
d oi at
(oia)
Va, pog di (p((se) dt
(13)
at
where (x)) is the unconditional flow. The conditional flow up( y) can be derived via Bayes:
0/ 0000
ai 0,
(pp y)
V logp(æz y)
(14)
a1
at
0 a
[xt)))) - 0,00,
m. logp((> xx).
(15)
For the following proof, we impose the regularity and smoothness assumptions listed Appendix A of Song et al. 2021a for both the prior marginals p(æ,) and the posterior marginals <xx y).
Lemma 3. Let p(x,) be the unconditional diffusion marginal and ((, y) x p(a;)=(y xx) be a posterior time-mo marrginal obtained at diffusion timestep t. Denote (2(1)) and un (xt y) as their corresponding velocity vector fields. We have the following identity:
(T ()(()()) Ems-(( [w= ()) up(xx))" Va, ( x1) (16)
dt
Proof. Simplifying the derivative gives:
d
dt (x ())lp((z))) dt ((x; y) [log y) log p(xx)) ) dx:
= / d
(xxx v) [log (xx, y)) logp((z)) xx:)
dx;
dt
+ / (xx,
8)
[log (xx y) ((g,)( dx (Product rule)
Now we expand the first term. Let (xx)) Un (xt y) be the flows at p(x:), ((x; y) respectively:
/
d
dt (xx 8) [log ग (xx
y) log p(xx)] dx;
/ V (IT( w)x x)) log ग (wx
yy) log p(x:)) dx;
(Continuity equation)
/ n(m, u),,(m v) V log N (x; y) log p(x,)) dx;
(Integration by parts)
/ π(x v))=(=)) y) V... log क xx)da
(Bayes)
15

---

Published as a conference paper at ICLR 2026
For the second term:
/ (xx;
y)
[log क (x,
y) 1 logg dæ,
df
d log क (xx (n) 1t (( 3) d dt
E
log (ogp(a,)dz
dt
y) d dt
T (x
log p(a: )dx;
क (x) y) up( + (p) Vva logp(zt)) dæa
(Log-continuity equation)
कt (xx v) V (x)) + ( VV an log (x; 1 y) dx:
/ ( (x,) VV log (y x dx,
(Bayes)
f (कم (m,) Va. Va, log ( x. dx,
(Stein's identity)
Adding the two terms back together gives us:
dt DaL (T(x, ())l)()))) / (TZ,, y) (U v) (x,))) Va, Ty x dd
(( (w= y) up(x)))) Vay log ( m:)
Proposition 1 (DiME: Model evidence estimation along the standard marginals). Given diffusion
process 0; = 0,20 + 2 2, ~ NN0 I), posterior marginals p(æ; y) x p(xx) p(y | z )ao, and timesteps 0 to < IN T, the model evidence can be estimated via:
p(y
wo)p((n)
logrly) Ex wil log logp(y mo)]
DRI. (p(zo y))lp(())).
(8)
The log- likellhelhood term can be estimated with posterior samples and the KL. divergence with:
N
(KL(p(x0 v)llp(zo)) ~
Σ Ct
~pla, (w) Mn Va,, logp(y xx,))l
(9)
ai where C2 = (d..0--0 - 0f, aia
dependent on the diffusion schedule and Al; = bi 4--11
Proof. Expanding out the KL divergence gives us an expression for the model evidence:
(pl(p( v))l(())))
=
Egg pp w)
log y)) log (ogr(an) [x))
En p( w))
log To) log (gg)())
Emg ( m)
log p( y
xx)) log
logoop( p Ema-p((all)
log p(y
xo) DKL(p(æ0 v))l(())))
Forr likelihoods with Gaussian noise, if the posterior and prior have enough overlap such that posterior samples have data fit of x² this term essentially becomes constant under the same likelihond function.
16

---

Published as a conference paper at ICLR 2026
At the final diffusion step, the forward process has destroyed all information about the measure- ments, so we have p(ær y) = p(ær), or (Kl(p(æT) y v)(pp(z)) = Therefore, we have:
T d L/'
(Kl(p I ())lp(())))
dt (pl(p y))lp((:)))tt
(
Emj
(up(2n y) up(xx)) V Vas logp(y m;)) dt
20 (d.01
Emp Ve, logply x,)]| dt.
oz a,
N
ri li
Ali d',00:
E
V mi logp(y a;,)
ac;
24 ap (v)
where (i) uses Lemma] but with (xx; ()) = p(0e
y) and (ii) uses the PF-ODE of the conditional
flow (Eq. 15
17

---

Published as a conference paper at ICLR 2026
A.1 PROOFS AND DERIVATIONS FOR THE DAPS-BASER IMPLEMENTATION
Lemma 1. Suppose have diffusion process 0, + Ty Zt NO, 1). Let p(xr) and suppose p(ar)) se Nup: 2o). Then we can express the posterior covariance Dzol as
2 I of
Σ
(10)
Proof. Our diffusion corruption process gives us:
Cov( a Do + I Cov( xx) Cov a,30 + o22,) a. (.C(((z)) a, Eo.
Plugging these into the Gaussian posterior covariance formula gives:
Cov ax)) Cov(2n) - Cov(an: Tz Cov(() -Cov xx,)
Eg 0: Ed
[ En
+ of
04 GgEo
0 +
-1
Eo Eg
Eg
Eg
Eo
(Woodbury identity).
Corollary 1. Suppose the training data has >max (Eo) Then ED wx
o²
o/t
Proof. Plugging in the covariance from Lemma gives:
2 En
o 0f + dg
Enn (mz
Amax
I
Amin
Σn1
a3
1
Lemma 2. Suppose we have diffusion process 24, 0,30 0, 2. N(O, I) and sample io
p(z0 xt, y)- Under the assumption that p(ao xx,) R. (E xz). Exole.). both Qhigh (ão)
and (āo) are unbiased estimators of Va, log p(y xx,) with
Ohigh (wo)
(Ep E[zy xx()
(11)
tow (Zo)
Emolar Va, logp(y kn))
(12)
of
Under low diffusion noise (at - 1, + 0), Var(O + 0 but Var = 0
Under high noise (at - 0, + 1), Var(Onign) + but Var(O = 0 Var
Proof. Using Tweedie's rule, we can show that Onigh is unbiased for the likelihood score: Va, log p xx) Vx, log p(xe y) Væ, log p(x:)
a+ E|0
2t
Ør, y]
E
E|z0
x a,
o?
az
o
(E acn
y)
IF 20
xx.))
of
18

---

Published as a conference paper at ICLR 2026
Our Gaussian approximation of p(z. 2=]) Using the fact that the expectation of the score is zero, we have:
xx) gives Vwo log p(zo x
E
ze me (zu
E xcu
El
Sr. y
Ea + Ee Van mg log p(ao
We. y) 2t, y
E[z, + Da m. (Van p(zg xx) V mo log ax)) 2 y]
E(zy (xu E[xy æ1)) mal V m. p(y æu) z y)
Elæe axs] + E Eaole (
[ mo logp(y æo)].
(**)
Substituting (**) into (+) shows that Olow is unbiased. To compute variance bound for thigh
Var
Cov(aro æt; y)
Cov xx))
(Gaussian conditional covariance)
0,
a;
(Corollary D
o [o? + af)
The term of + as is of constant order over the diffusion process, resulting in a variance of aî/o To compute a bound for Olows we linearize the likelihood f at a fixed æ with du and get
at
Var(Oiow)
E
m Cov( V wn log xx)
xt, y) Σ 2 le,
o,
ai
Eaul Cov( 1
Σ
A (y Aaro) Ft: - #a/=:
o
0
1
Cov(zo mm.))) AT AET 20/=
Balle ATA
A
||| 4
of
Var(Onigh).
(Corollary
o
(o? + a2)²
||
+ 0fag
o (o + aF)3"*
Unlike thigh, the variance of this estimator goes to 0 a. - 0, but for the rest of diffusion process the constant factor makes it a higher variance estimator than 1high in practice.
19

---

Published as a conference paper at ICLR 2026
B GENERALIZING D DIME TO ARBITRARY POSTERIOR MARGINALS
While we only present DiME along the standard marginals in the main text, DiME can be extended to any arbitrary diffusion posterior marginals that anneal to the true posterior follows:
Corollary 2 (Generalized DiME). Let p(axs) be the unconditional diffusion marginal and x: <( v) ( x:)) be path of marginals that anneals to the true posterior, x a y)
p( y). Denote up(2e) and Ux (It y) as their corresponding velocity fields. Given reverse
diffusion timesteps 0 to < < IN T, we have:
logp(y) Eearp(a wil log ply xo)] - (p yy)lp(())))
where
N
(pl(p v))l((()))
Σ
At, (.
(u+ (1) up(2xe) Va, log T mi)
+ (xT((T(TT y)(((())))
Proof. Follows directly from the proof of Prop.
While Up can be directly computed using the PF-ODE (Eq. 13 computing V, can be difficult for arbitrary time-mare margenalg By crafting sequence of diffusion marginals with known velocity, we can avoid any approximation like the one used for p(æo xx) in the DAPS implementation. The marginals (ær y) should also ideally have high overlap with the unconditional marginal p(æ,) to ensure accurate computation of (2,)) when the score is learned. If ( y) # p(ær), ) then we must use other estimation methods to compute (TL(T(22) y)(l(<zT)) or settle for an unnormalized model evidence estimate. The following section discusses two estimators that does not require any approximation, but are not normalized.
B. ESTIMATING THE MODEL EVIDENCE FROM ALTERNATIVE SAMPLING METHODS
Algorithm One posterior sample-pate path of DiME-PaPDM
Require: based model sø, measurement y, noise schedule de, (I...E
1: Sample 2 ~ pl (zo)
2: Initialize the integral sum, f == O.
3: for = N, N ..... 0 do
4: Sample Zt, ~ q( y, 20) x q(yl xt, p((xe, (
5: Sample 2yg p( xt.)) using 80-
6: Update the integral sum f + (4 t 11-1), )² Vx, log g(y) x..))
7: end for
8: return (
The PnP-DM marginals follow g(x;; y) x q( ax)p(a;) where q(y xx)) is a time-independent likelihood term. As a result, the initial distribution q(æT y) + p(æT) and the KL divergence DKI(g | y) [[p(ZT)) is non-zero, so we can only compute the unnormalized model evidence. However, the unknown constant is dependent on y and the forward model, so this unnormalized quantity can still be used for model selection. We derive the estimator in Proposition 2 and imple- ment it with PnP-DM in Algorithm 2
Proposition 2 (DiME PP:M-PP:PDM: model evidence estimation along the PnP-DM marginals). Given diffusion process 2; = 0,20 + Zt 2e ~ NN(O, 1), PnP-DM marginals g( y) xx p(æx)q(y w.), and timesteps 0- to
IN T, the model evidence can be estimated via:
N AI, ALE
logply) R C(q. v)
Σ 1=1
2,, log q xe.)
(17)
where C(g. y) log (o, (a(y x)) is constant for fixed measurement y and forward model, and up
d' cr at
(ofaz = a V log p(x).) is the PF-ODE velocity at e; (Eq. 13
20

---

Published as a conference paper at ICLR 2026
Proof. While apply Corollary 2 take shortcut to cancel some extra terms. Under
the PnP-DM marginals, log ( t) is time-dependent while the likelihood q 2x) is time- independent, can directly compute the derivative of the evidence:
log ( t) = Bmng(a)l))) dt
log p(
(Fisher's identity)
dt
V (p())) )) + (z(1)) ? Va, logp(x;))
(Log-cont. equation)
-Ep-g(
V op(2)) + 0p())) "Va; log q y)]
+ Ee w))
(1p()x)) Vn. log ( æx]
(Bayes)
lw) (1x)) V log (y xx)]
(Stein's identity)
Integrating over all gives us the estimator:
log p(y) log q(v =0) 0)
loga(y) | t= =T) - f/ d
logg t)dt
dt
N
= =log 13 11 1= =1=17 TT)
A M (v) [up(are,) "Va, me, log ((y) xtt;)
where log q t= T) = log E Bee [4(3 x)] is the model evidence under a Gaussian prior,
and (x(1)) = o x; - (ofor of a v p(x;) is the PF-ODE from Equation 13
We can also do a similar derivation for the marginals used for the Twisted Diffusion Sampler (Wul et al. 2023), which follow Tæx y) x p(z))p(( E[ æı]); note that this evidence gradient can be computed before resampling using SMC weights after resampling without weights.
Proposition 3 TEiME-TDSS model evidence estimation along the TDS marginals). Given diffu-
sion process 2 0g + 0i 2t NO. I). TDS marginals π (x)1y) x p(z))p(y) and timesteps 0- = to < ... < IN = T, the model evidence can be estimated via:
N
logp(y) 22 C(g. y)
Σ
AG, M. upp(ar.)) "Ma ply ((xx))) + log (( ))
di
(18)
where Cly. y) log Em (ply [((e))]] is constant for fixed measurement y and forward model, and up(xx)) = aix; (0f0, of Va log p(z)) is the PF-ODE velocity at æ, (Eg. 13).
Proof.
d
d / p(ax)p(y p(x:))dxx
log (3 11)= ( | 1) dil
dl
d < (t) /
di ( ply dlx;))ddx ( 1) / p(are) d
d p m(ce)) dae
d log logp(æx)) di logp(y u((x)))
u,, T logply u(xx)) di log m(æ:))
Eg, Ea.~n(a.lv))
where the last line is derived in a similarly to Proposition Integrating over all t gives the estimator:
/' d
log ()) log (3 T)-
log ग
t)dt
dt
Σ At, F. (2, lv) op( TVa, logp(y (lxx,)) + d log u(ax)))
log 1 T)
dt
21

---

Published as a conference paper at ICLR 2026
where log aTy IT) log ((0.0)))))) poy ((xr))].
22

---

Published as a conference paper at ICLR 2026
C BASELINE METHODS
In Section
we compared to five baseline methods, described here. Define ps as an intermediate distribution the power-posteriae path, used in TI, AIS, and SMC: pa(x:) oc p(z)p(y x) , B E Note that this path is equivalent the geometric path under different parametrization.
4.1
[0, 1].
Naive Monte Carlo (Naive MC): We generate many unconditional prior samples and compute the expectation of the likelihood follows: (og)()) = log (=z~p( pyy
Original DAPS covariance heuristic (Original heuristic): We perform the same steps as described in Algorithm but using the originally proposed covariance heuristic Ezole. - ooZ-
Thiiiodynamic integration (TT) (Lartillot & Philippe 2006); Originally from statistical mechan- ics, TI can also compute expectations by integrating the evidence over the power-posterior path:
d
p dB PaP p0( /
1 d
log Pa dB
p(x) Zz dx
1
/ p(a))p(y x) logp(y æ)dx = Exppy Dogp(y x)]-
p2(y)
Using the fact that log p = log = O, using K integration steps we have the estimator:
log logps=1(3)
A Exmpe, [log((y) x=]]
Samples from each pa generated using Sequential Monte Carlo (details below).
Annealed importance sampling (AIS) (Neal 2001): We can compute the ratio of evidence:
Pau (y)
I p(x)pp(y x Fx *pyy
x)) AAr dx
Ex-pe, IP x) A0k]
D3 (3))
P8. (y)
-Bpano
lexp( A Bg pg x))]]
Using the fact that log po= = log 0, telescoping over all By gives:
K
P3 (V)
II k=1
E Byo [exp(A,8 log a))]
Exan 1mp3pok-1
exp
Σ
3 logp(y) (x-1))
Note that 00:k P represents the sample-path of particle, where each transition kernel from 1 to le was performed using Langevin dynamics.
Sequential Monte Carlo (SMC) (Del Moral pling step from Sequential Monte Carlo methods to replace particles with degenerate weights (i.e. particles stuck in the wrong mode) while sampling the next intermediate distribution, potentially reducing both bias and variance. These resampling steps are performed either using fixed schedule or adaptively when the effective sample size falls below threshold.
2006): An extension of AIS that the resam-
23

---

Published as a conference paper at ICLR 2026
D IMPLEMENTATION DETAILS
D.1 TIME
As DiME implemented alongside DAPS, we have a similar set of hyperparameters.
Number of diffusion steps for sampling p( xz): While the original DAPS method uses many diffusion steps to estimate the denoised image each annealing iteration, we use 1 diffusion step to ensure an exact estimate of E/zo 2t] by Tweedie's ' formula.
Langevin dynamics: The learning rate of Langevin dynamics was tuned that posterior samples To p(2o æt; y) have mean reduced x² data likelihood fit that converges as low possible (ideally around 1, which indicative of good posterior sample for likelihoods with Gaussian noise). We also use linear learning rate decay is done in Zhang al. (2025). The table below lists the learning rates and maximum step counts used for each experiment. practice, sample convergence usually happens much sooner than the maximum step count.
Task
1r
Max steps
Mixture of Gaussians (in- (i--disrribution
5 X 10 4
2,000
Mixture of Gaussians
2 10
10,000
Mixture of Gaussians (saddle point)
2 10-5
10,000
Fourier phase retrieval
1 x 10-3
1,000
Gaussian phase retrieval
5 10
1,000
M87 black hole imaging
1 10
1,000
Selection of Jmin: This parameter can be viewed as the minimum noise level that the learned score can still be trusted. For the analytic Mixture of Gaussians experiment, we use minimum annealing
noise level min
0.05, and for the diffusion model experiments, we use d minn = 0.1 as is done in
Zhang al. (2025). The remaining integral from t 0 to tmin in Eq. is approximated by trapezoid, using the fact that the integrand equals 0 at = O.
Computing the precision matrix Σn : for some datasets, the covariance of specific pixel locations nearly 0 (such as the corner regions of MNIST), so we added jitter of 1e-2 for stability.
D.2 BASELINES
We use TI, AIS, and SMC as baseline methods for the mixture of Gaussians experiment (Section 4.1 and SMC for the MNIST experiment (Section 4.2, Appendix We used Langevin dynamics with 2000 steps and learning rate 0.0002 for our transitions between distributions. For SMC, we resample when the effective sample size (ESS) is less than 0.6.
We test with fixed linear, exponential, sigmoidal as well as adaptive 3 schedules that decrease the Conditional ESS at constant rate of [0.5, 0.6, 0.7]. For the mixture of Gaussians study, an exponential schedule with = 6 was found to be optimal. For the MNIST experiment, the fixed schedules use 50 annealing steps, while the adaptive schedule had minimum 3 step size of 0.002 which practice results in 150 steps at most.
24

---

Published as a conference paper at ICLR 2026
E BASELINE RESULTS ON MNIST MODEL SELECTION
Gaussian Phase Retrieval (SMC Baseline)
Fourier Phase Retrieval (SMC Baseline)
317
426
20h
-306
(590)
250
4000
-100
-397
-552
-471
-997
-408
63053
40000
4300
-190
-458
2219
--900
6000
4000
4000
-000
4000
281
4000
Groune truth dight
-196
7705
2100
1300
4200
4000
4300
420
7000
2150
4 4202
+056
333
+88
068
+088
+0800
346
+300
-888
399
5308
860
6886
425
488
Ground
5333
4200
3925
2286
0000
+201
322
-308
+320
+000
-337
430
0318
000
455
+038
429
6 #20%
4000
-319
6727
4008
6007
6200
-250
4193
4000
4000
400
4000
27.3
-00
+06
-289
4220
.10m
(320
<200
-200
+000
4000
2310
1220
4100
-000
4000
400
4100
4.20
-221
400
6
7
1
2
7
6
Model
Model
Figure 6: Model evidence confusion matrix using sequential Monte Carlo (SMC) for Gaussian phase retrieval (left) and Fourier phase retrieval (right) for each (ground truth measurement, model) pair of MNIST digits. When SMC uses the learned diffusion score Va loz)( at = 0.1 as the prior score, it often fails select the correct model.
Results of the MNIST experiment (Section 4.2 0.1 as a prior score are shown in Figure SMC is often unable to determine correct MNIST model. Each SMC annealing step also takes about 10x longer than equivalent annealing step for DiME, due to having to query the diffusion model many times for the prior score. Interestingly, narrower priors (i.e. MNIST digit 1) leads to higher model evidence estimates here; since the training data is more clumped together, the learned score becomes more accurate at low noise levels.
using SMC and the learned V log p(x:) with
For completeness, the model evidence estimates of a few ules as well as for a second noise level, a = 0.05, are reported in Tables and Mean and standard deviation over runs shown here. As be seen, lowering the noise level only further increases overfitting to training data, causing the estimated evidence to drop.
all temperature annealing sched-
Omti = U.1
Emin 0.05
schedule
6
Linear
182 4:
221
210 4:
-27 26
3352
422 4:
Expomential
- 183 +
-224 5
-33 2
2992 1: 20
-363 1:
-66 1:
Sigmoidal
-17 1:
211 1 13
-191 2.2
-273 1: 15
-374 1:
180 1: 17
Aduptive 0.5
1995 #:
202 23
-261
-29 4: 52
346 16
489 ::
Adaptive 0.6
1
-18 7
-25
279 4: 45
1
Adaptive
Table 2: SMC model evidence estimates on Gaussian phase retrieval.
Omin 0.1 GT Model
Tain 0.05
schedulee
6odel
GT Model
Model
GT Model
GT Model
Linear
416
-431
-409
-485 25
-577 +
-563
Expanential
422 + 20
422 2
407 +
-227 + 10
582 12
338 3:50:27
Sigmoidal
415 4
-61 3
-41
-20 133
550 +:
-576 2:
Aduptive 0.5
400
460
522 33
:
Adaptive
20
4
Adiptive
442 +
437
-584
370
Table 3: SMC model evidence estimates on Fourier phase retrieval.
25

---

Published as a conference paper at ICLR 2026
F SYNTHETIC MODEL SELECTION
this section, we perform model selection on the same set five priors as in Section 4.3.1 but simulated closure quantities measurements of test images from each prior verification. The correct model chosen for all measurements. The posterior samples with the lowest, mean, and highest path evidences are also shown in each case.
Mcan image
Path evidences (min/meon/maa/
Mean estimated ply)
GRMID
196
Distribution loy plyl estimetes
250
individual sosterionsample paths
147
178
212
Ficcee
200
23
150
log
100
102 24
Prior MN INT
0-
Spacssee
162
-20
RUAF
-500
750
1.
GRMHD
faces
MNIST
Spacellet
RIAF
Figure 7: Ground truth: GRMHD.
Moan Image
Path evidenoes /minn/mmaa/maaa
200
Spocc ehoo
Mean std estimated log ply)
Distribution ply) estimates
150
individual posterior-sample
100
-354
250
112
Fultanca tog ply)
886
614
Prior Facds
-1264
-1102
MNNIST
-500
-1000
2.797
.2211
-1649
1105
RMAF
2000
SpaceNet
GAMHD
Faces
MNIS
RI
Figure Ground truth: SpaceNet.
26

---

Published as a conference paper at ICLR 2026
Moan evidences Jmin/mean/mak
f
Mean ato estimated log ply) Distribution of estimates over indivioual saaarlo paths
198
230
276
Feccs
250
200
170
10
129
168
224
169 28
160
227
100
Prior
50
170
-100
RIAF
-228
-335
-72
-200
-300
GRMHD
Figure 9: Ground truth: CelebA
Moan image
Path evidences (min//man/maxi)
209
240
MNIST
230
2 1:
1100
Focos
150
154 4
100
Phior SpacsNet
Evidence
140
GRMHD
Meen std of estimeted log ply)
-1192
-500
Distribution ply) estimates
RIAE
-1000
posterion-serpple paths
-1500
NNIST
Faces
SpaceNet
GRMHD
RUC 113
Figure 10: Ground truth: MNIST.
Image
(min//man//max
192
241
270
log pty)
Distribution log ply) est over individual posterta-sample paths
RIAE
250
194
Eacaot
200
17
135
176
Pijor
GIMIDD
100
2
SmacehNet
156
50
55
104
143
RIAE
RIAF
Faces
GRMHD
Spaceñett
MNIST
Figure 11: Ground truth: RIAF,
27

---

Published as a conference paper at ICLR 2026
G LARGE LANGUAGE MODEL USAGE
We used ChatGPT for coding assistance, polishing text and LaTeX, and checking mathematical derivations.
28


Reviews:
[{"review_id": "xZyNakWqPQ", "reviewer": "Reviewer_w9CR", "summary": "- The paper addresses the problem of selecting an appropriate prior model in inverse problems by proposing an estimator of the model evidence\n- The estimator leverages samples generated during the diffusion reverse process and decomposes the evidence computation into two components: the expected log-likelihood and KL divergence whose considered approximation involves the norm of the gradient of the intermediate log-likelihood\n- The intermediate likelihoods are approximated using two estimators depending on low/high noise setting.\n- As an example, the authors apply the approach to the DAPS and PnP-DM samplers \n- for DAPS, they also introduce a new approximation of the covariance $X_0 | X_t$, estimated empirically from the training dataset through the covariance of the prior $p_0$", "questions": "- Since Equation 9 intervenes computation of quantities under different the expectation of different marginals: how many sample paths are used to evaluate them in Algorithm 1 ?\n- Can the authors provide the runtime of the algorithms (For DAPS and PnP-DM)?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The introduced procedure to estimate the model evidence is practical and may enable informed model selection in inverse problems.\n- the proposed approximation of the conditional covariance $X_0 | X_t$ within the DAPS framework is novel", "weaknesses": "**Methodological issues**\n- The authors criticize alternative methods for model evidence approximation for being inaccurate and biased (Line 52-53 and Line 64-65). this claim is misleading as the proposed approach relies heavily on Gaussian approximations; and more precisely, the authors don't quantify the impact of such assumption on the approximation\n- In Lemma 2, the variance computation of both estimators is incorrect. In particular, equation (12) incorrectly states that $Var(\\Theta_{\\text{high}}) = O(1/\\sigma_t^2)$; however, this ignores the dependence of $\\Sigma_{0|t}$ on $\\sigma_t^2$, which would yield, if considered, a variance $O(1)$. Consequently, the practical criterion for choosing between low- and high-noise estimators is based on a flawed derivation. In addition, in practice, the authors choose the variant of the estimator with the lowest variance, but this irrelevant as the variance of the estimator are computed with different asymptotic assumptions\n- It is concerning that the high-noise estimator in equation (11) does not depend on the observation $y$, but may still be used during the algorithm\n- The proposed approximation of $Cov(X_0 \\mid X_t)$ is impractical. the authors propose to estimate the empirical covariance $\\Sigma_0$ of $p(x_0)$ from training data but this computationally expensive (quadratic memory cost); in addition, this covariance $\\Sigma_0$ is involved in matrix-matrix operations and must also be inverted\n\n**Typos and technical inaccuracies**\n- Section 2.2: The categorization of posterior sampling methods is irrelevant and doesn't reflect state-of-the-art. Methods such as TDS [2] and MCGDiff [3] are based on SMC and evolve particle ensembles to solve inverse problems; MGPS [4] uses midpoint guidance for estimating transitions; and RedDiff [5] is a fully variational approach.\n- Line 269: The equation is mathematically incorrect: the square of $\\Theta$ is computed but $\\Theta$ is a vector.\n- Lines 60–63: The claim that computing the log-density of a prior sample requires integrating the Jacobian of the score overlooks existing work (see Eq. (13) in [1]), where this can be done without vector jacobian product over the score.\n- The statement in Lines 207–209 is ambiguous, at low noise, $p(x_t \\mid x_0)$ concentrates near $x_0$, so saying it \"dominates over the Gaussian approximation of $p(x_0)$\" lacks clear meaning.\n\n**Limitations**\nThe method is applicable in settings where the posterior sampler follows a noise–then-denoise procedure.\nThis restriction should be explicitly stated. Furthermore, the paper does not discuss how $\\Sigma_{0|t}$ is handled in practice. Since the space complexity of thus matrix scales quadratically with the dimension if the problem and appears in the estimator’s computation, it poses computational/memory challenges\n\n\n---\n\n.. [1] Skreta, Marta, et al. \"The Superposition of Diffusion Models Using the It\\^ o Density Estimator.\" arXiv preprint arXiv:2412.17762 (2024).\n\n.. [2] Wu, Luhuan, et al. \"Practical and asymptotically exact conditional sampling in diffusion models.\" Advances in Neural Information Processing Systems 36 (2023): 31372-31403.\n\n.. [3] Cardoso, Gabriel, et al. \"Monte Carlo guided diffusion for Bayesian linear inverse problems.\" arXiv preprint arXiv:2308.07983 (2023).\n\n.. [4] Moufad, Badr, et al. \"Variational diffusion posterior sampling with midpoint guidance.\" arXiv preprint arXiv:2410.09945 (2024).\n\n.. [5] Mardani, Morteza, et al. \"A variational perspective on solving inverse problems with diffusion models.\" arXiv preprint arXiv:2305.04391 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0J0UnW2BkH", "reviewer": "Reviewer_mdpM", "summary": "This paper provides an efficient method for estimating normalising constants (model evidence) which can be utilized for model selection.", "questions": "- The literature on alternative posterior covariance approximations were not surveyed properly, there are some relevant methods authors should discuss whether they can utilize it.\n\n> Boys, B., Girolami, M., Pidstrigach, J., Reich, S., Mosca, A., & Akyildiz, O. D. Tweedie Moment Projected Diffusions for Inverse Problems. Transactions on Machine Learning Research, 2024.\n\nPlease discuss the approximation provided above in the light of Lemma 1. Would the use of approximations given above (or others in the literature) improve your method?\n\n- Proposition 1 is written as a result of this paper, however, the formula for $\\log p(y)$ is quite standard (just an application of Bayes rule) and the approximation of the KL divergence also follows from the definition of the diffusion model. I think this is misleading - authors should clarify that this is a standard and well-known consequence of Bayes rule.\n\n- Can you explain the robustness properties of your method? If the data has outliers, how badly does it affect the results?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper presents the methodology cleanly and displays interesting empirical results, especially presenting an example on a GMM case as well as a practically interesting case of choosing priors for scientific datasets.", "weaknesses": "The paper has multiple weaknesses - ranging from the formulation and empirical results. Mainly\n\n- Some of the theoretical results (see my comments) are well known and not novel\n- Empirical comparisons miss some relevant work that also consider full covariance-based approximations of $p(x_0 | x_t)$\n\nIn general, it feels like the methodological contribution is limited, although I think experimental section is nicely done (see strengths)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1PQKh2jM5x", "reviewer": "Reviewer_U7kJ", "summary": "This paper introduces DiME, an estimator for the  marginal likelihood $\\log p(y)$ when the prior is a pre-trained diffusion model. The key contribution is the estimation of a KL path integral representation of the evidence, leading to a practical estimator that leverages samples from the diffusion posterior sampling algorithm DAPS.", "questions": "I am not sure to understand what's the difference between analytic score and clean score; cf line 309: the clean score is never used", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Through compelling applications with extensive experiments, the proposed method seems to exhibit strong performance. The authors have tested the method on:\n- Controlled Gaussian prior benchmark:  here the method matches the performance of the SMC baseline. \n- Phase retrieval on MNIST: DiME correctly identifies the generating prior for most digits; confusions appear only for symmetric cases and the method clearly outperforms SMC. \n- Black hole imaging: Here the method provides results that align with physical evidence.\n\nThese results show that the method is in itself useful and could be interesting for the ICLR community.", "weaknesses": "I only see one weakness of this work: \n\n- One of the contributions of the paper is the improved approximation for the posterior covariance but this approximation is already leveraged in a previous work that the paper doesn't cite [1]; see last paragraph of section 3 right before the algorithms. This is the same approximation as the one proposed in this work. \n\n[1] Linhart, J., Cardoso, G.V., Gramfort, A., Corff, S.L. and Rodrigues, P.L., 2024. Diffusion posterior sampling for simulation-based inference in tall data settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qIKFCePQTW", "reviewer": "Reviewer_56j4", "summary": "The choice of the prior distribution plays a central role in Bayesian inverse problems and a common way to select the prior is to use model evidence estimation. In recent years, diffusion models have emerged as powerful tools for constructing expressive priors across a range of challenging tasks. However, the corresponding model evidence is intractable in this framework as all marginal probability density functions are not available explicitly.\n\nTo address this limitation, the authors introduce DIME, a novel method for estimating the log-evidence of diffusion-based priors using only a limited number of samples. The proposed approach relies on samples drawn from the marginal posterior distributions of the diffusion process using a procedure inspired by Decoupled Annealing Posterior Sampling (DAPS). Furthermore, the authors enhance the Gaussian approximation of the denoiser by introducing a new covariance estimator, derived from a Gaussian approximation of the true underlying distribution.\n\nThe proposed estimator is evaluated using five baselines including sequential Monte Carlo methods across several scenarios: a high-dimensional Gaussian target with known ground truth, two non-convex inverse problems, and a model selection and validation task based on real observations of the M87* black hole. Empirical results demonstrate that DIME consistently outperforms all baseline approaches and effectively applies to real-world black hole imaging problems.", "questions": "The covariance estimate proposed by Lemma 1 used a Gaussian assumption. Would you have insights on how to propose an estimate in a more general setting ? Or to provide guarantees that the proposed estimates is good even in non Gaussian settings.\n\nThe literature providing theoretical guarantees for score-based diffusion models is very rich for unconditional and conditional sampling. Could this theory be used to provide guarantees on the proposed logevidence estimator ? \n\nDid you try score-based alternatives to DAPS to obtain the samples to support that this choice outperforms other posterior sampling algorithms based on pretrained diffusions ?\n\nCould you include computational time to your experiments to understand the trade-off between number of samples from the diffusion and computational complexity ?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed method is derived using only a few posterior samples and the squared likelihood score can be estimated with state-of-the -art methods such as Decoupled Annealing Posterior Sampling.\n- The algorithm is compared to various baselines in several challenging tasks and consistently outperforms the alternatives.\n-  The model selection results for the real M87* data and the posterior samples with the lowest and highest evidence highlights that the method can be used in real data settings, using numerous potential priors.", "weaknesses": "- The proposed algorithm is well motivated and easy to derive from pretrained denoisers. However there are no theoretical guarantees or no intuitions on the data distribution for which the logevidence estimator is good. \n- The algorithm uses DAPS whose annealing process reduces the dependency between samples at consecutive time steps,and allows to obtain marginal posterior samples for any noise level starting from samples from any other noise level. However, the reasearch on posterior sampling using pretrained score-based models has been very active for the past few years and alternatives could be considered. As there is no theoretical guarantee on the proposed estimate, exploring other samplers would support the choice made by the authors.\n- The proposed estimate of the posterior covariance is based on a Gaussian approximation of the data distribution using training data. Although it provides a very easy to compute estimate, the assumption that the data distribution is close to a Gaussian is strong in settings where this distribution is likely to be multimodal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)