Title: Provable Posterior Sampling with Denoising Oracles via Tilted Transport
Venue: NeurIPS 2024
Accepted: yes
Keywords: diffusion based model; posterior sampling; inverse problem; provable sampling
Abstract: Score-based diffusion models have significantly advanced high-dimensional data generation across various domains, by learning a denoising oracle (or score) from datasets. From a Bayesian perspective, they offer a realistic modeling of data priors and facilitate solving inverse problems through posterior sampling. Although many heuristic methods have been developed recently for this purpose, they lack the quantitative guarantees needed in many scientific applications. This work addresses the topic from two perspectives. We first present a hardness result indicating that a generic method leveraging the prior denoising oracle for posterior sampling becomes infeasible as soon as the measurement operator is mildly ill-conditioned. We next develop the *tilted transport* technique, which leverages the quadratic structure of the log-likelihood in linear inverse problems in combination with the prior denoising oracle to exactly transform the original posterior sampling problem into a new one that is provably easier to sample from. We quantify the conditions under which the boosted posterior is strongly log-concave, highlighting how task difficulty depends on the condition number of the measurement matrix and the signal-to-noise ratio. The resulting general scheme is shown to match the best-known sampling methods for Ising models, and is further validated on high-dimensional Gaussian mixture models.

Provable Posterior Sampling with Denoising Oracles via Tilted Transport

Abstract

Score-based diffusion models have significantly advanced high-dimensional data generation across various domains, by learning a denoising oracle (or score) from datasets. From a Bayesian perspective, they offer a realistic modeling of data priors and facilitate solving inverse problems through posterior sampling. Although many heuristic methods have been developed recently for this purpose, they lack the quantitative guarantees needed in many scientific applications. This work addresses the topic from two perspectives. We first present a hardness result indicating that a generic method leveraging the prior denoising oracle for posterior sampling becomes infeasible as soon as the measurement operator is mildly ill-conditioned. We next develop the tilted transport technique, which leverages the quadratic structure of the log-likelihood in linear inverse problems in combination with the prior denoising oracle to exactly transform the original posterior sampling problem into a new one that is provably easier to sample from. We quantify the conditions under which the boosted posterior is strongly log-concave, highlighting how task difficulty depends on the condition number of the measurement matrix and the signal-to-noise ratio. The resulting general scheme is shown to match the best-known sampling methods for Ising models, and is further validated on high-dimensional Gaussian mixture models.

1 Introduction

Inverse problems consist in reconstructing a signal of interest from noisy measurements. As such, they are a central object of study across many scientific domains, including signal processing, imaging, astrophysics or computational biology. In the common settings where the measurement information is limited, a reliable solution for these problems usually depends on prior knowledge of the data. One popular approach is to choose a regularizer that utilizes data properties such as smoothness or sparseness, and then solve a regularized optimization problem to obtain a point estimate of the original data. However, this approach often struggles with selecting an appropriate regularizer and might be unstable in the presence of large measurement noise. A more robust approach takes a statistical formulation and seeks to sample the posterior distribution of data based on Bayes's theorem, which allows for uncertainty quantification in the reconstructed data by leveraging a model for the prior data distribution.

While accurate models for high-dimensional distributions are notoriously complex to estimate, the resurgence of deep neural networks has provided unprecedented capabilities for modeling complex data distributions in certain high-dimensional regimes. Specifically, score-based diffusion models [55, 33, 58] have achieved remarkable empirical success in generating high-dimensional data across various domains, including images, video, text, and audio. These models implicitly parameterize data distributions through an iterative denoising process that builds up data from noise. Furthermore, there is a growing literature developing theoretical foundations of score-based diffusion models

Jiequn Han

[17, 7, 45, 16, 19], giving a comprehensive error analysis including score estimation, initialization error and time-discretization error. By generating high-fidelity data, these models can also serve as data prior for posterior sampling in inverse problems in high dimensions. Following this idea, many studies (see, e.g., [40, 22]) have leveraged diffusion models for posterior sampling. However, as discussed below, various categories of approaches for posterior sampling introduce different uncontrollable errors, such as those arising from the approximation of the conditional score or the use of a limited variational family. This abundance of heuristics contrasts with the principled sampling used in prior data generation, and is often at odds with the statistical guarantees needed in many scientific applications.

In this work, we aim to bridge the gap between principled diffusion-based algorithms for both prior and posterior distributions. Focusing on the canonical setting of linear inverse problems, where measurements are of the form 𝑦 = 𝐴𝑥 + 𝑤 , with 𝑥 ∼ 𝜋 the signal to be estimated and 𝑤 an independent noise, we first illustrate a negative result, revealing that no method can efficiently sample the posterior in general cases, even with the prior denoising oracle. Subsequently, we develop the tilted transport technique, which utilizes the quadratic structure of the log-likelihood in linear inverse problems in combination with the prior denoising oracle to exactly transform the original posterior sampling problem into a new one that is easier to sample. Figure 1 illustrates a schematic plot of the method using two-dimensional Gaussian mixture examples, showing that while the original target posterior problem remains multimodal, the boosted posterior resembles a unimodal distribution.

same reverse SDE

posterior data generation:

from a chosen

and boosted posterior

from a large

prior data generation:

target posterior

boosted posterior

(provably easier to sample)

MW

NurTHeriCalaTplen(Ou

snpetpovtenia

MPZ

We establish a precise condition where the density of the transformed posterior problem becomes strongly log-concave, making it suitable for efficient sampling via Langevin dynamics. This condition showcases the interplay between a geometric property of the prior (what we call tilted spread ; see Section 5) and the conditioning and noise level of the measurements. Interestingly, the condition can be satisfied when the signal-to-noise ratio (SNR) is either moderately low or moderately high, in contrast with traditional sampling methods, which typically excel only within a specific regime.

As a first application, we show that tilted transport can sample from Ising models of the form 𝜈 ( 𝑥 ) ∝ 𝑒 -1 2 𝑥 ⊤ 𝑄𝑥 , where 𝑥 ∈ {± 1 } 𝑑 is supported in the hypercube, up to the critical threshold determined by the gap 𝜆 max ( 𝑄 ) -𝜆 min ( 𝑄 ) = 1, thus matching the performance of Glauber dynamics [24, 1] as well as the computational threshold predicted by the low-degree method [42]. More generally, even when the boosted posterior is not strongly log-concave, it remains easier to sample than the original one. Thus, tilted transport can be combined with any existing black-box posterior sampling methods to enhance their performance. This technique operates without any additional computational cost and functions in a plug-and-play fashion, allowing for straightforward integration into various frameworks. When working with high-dimensional Gaussian mixtures, where an analytical solution to the posterior is available, we numerically validate our theory and demonstrate enhanced posterior sampling performance.

1.1 Related Work

Numerous studies in recent years have explored score-based priors for posterior sampling. We note that several recent works [60, 20, 29, 54, 21] introduce hyperparameters to balance the influence of the prior and measurements, resulting in sampling strategies that guide output to regions where the given

observation is more likely. These strategies typically deviate from the principles of Bayesian posterior sampling and often lack a precise definition of the resulting distribution. In contrast, other approaches adhere more closely to Bayesian principles. One such approach is variational inference, which involves designing variational objectives and optimization methods based on the structure of score-based diffusion [41, 47, 26, 37]. However, even with an accurate prior score, the accuracy of posterior sampling heavily depends on the choice of variational family and optimization procedures, not to mention the additional optimization cost. Another popular strategy focuses on approximating the score conditional on the measurement using various heuristics [58, 40, 36, 22, 48, 56, 57]. In this approach, approximation errors typically remain largely uncontrollable due to the challenges associated with tracking the conditional distribution for intermediate states. Recently, some studies have adopted sequential Monte-Carlo methods to systematically approximate the conditional score [62, 13, 23], providing consistency as the number of particles used to approximate the conditional distribution of the intermediate states increases. However, this particle-based method still struggles with highdimensional problems due to the curse of dimensionality [9]. Alternatively, [61, 63] propose plug-and-play methods with denoising oracles for posterior sampling, offering asymptotic guarantees, though the required steps may grow prohibitively in high dimensions.

We note that [13] also intuitively explores the possibility of reducing the original posterior to an equivalent one under restrictive conditions in the discrete-time setting. In contrast, our tilted transport technique operates in a fairly generic setting and is supported by a clear theoretical foundation. Concurrently, [49] proposes a conceptually similar two-stage approach for posterior sampling in sparse linear regression, based on a different structural prior rather than denoising oracles.

2 Preliminaries

Notations:. P( ℝ 𝑑 ) denotes the space of probability measures over ℝ 𝑑 . 𝛾 𝑑 denotes the 𝑑 -dimensional standard Gaussian measure, and by slight abuse of notation, 𝛾 𝛿 or 𝛾 Σ denote the centered Gaussian measure with covariance 𝛿𝐼 𝑑 or Σ when the context is clear. For 𝑄 ⪰ 0 in ℝ 𝑑 × 𝑑 and 𝑏 ∈ ℝ 𝑑 in the span of 𝑄 , the quadratic tilt of 𝜋 is the measure T 𝑄,𝑏 𝜋 ≪ 𝜋 with density proportional to d T 𝑄,𝑏 𝜋 d 𝜋 ( 𝑥 ) ∝ exp GLYPH<8> -1 2 𝑥 ⊤ 𝑄𝑥 + 𝑥 ⊤ 𝑏 GLYPH<9> . We also use the notation T 𝑄 when 𝑏 = 0. ∥ 𝑄 ∥ denotes its operator norm. 𝜋 ★ 𝛾 denotes the convolution of two measures 𝜋 and 𝛾 . For 𝛼 ≥ 0 and 𝜋 ∈ P( ℝ 𝑑 ) , we define D 𝛼 𝜋 ( 𝑥 ) : = 𝛼 𝑑 𝜋 ( 𝛼𝑥 ) as the dilation of 𝜋 . For 𝛽 ≥ 0 and 𝜋 ∈ P( ℝ 𝑑 ) , we define C 𝛽 𝜋 ( 𝑥 ) : = 𝜋 ★ ( D 𝛽 -1 / 2 𝛾 𝑑 ) as the Gaussian convolution of 𝜋 .

Problem Setup. Consider a high-dimensional object of interest 𝑥 ∈ ℝ 𝑑 , drawn from a certain probability distribution 𝜋 ∈ P( ℝ 𝑑 ) . We suppose that one has managed to learn a generative model for 𝜋 via the DDPM objective [33]; in other words, for any 𝑦 ∈ ℝ 𝑑 and 𝜎 ≥ 0, we have access to the denoising oracle DO 𝜋 ( 𝑦, 𝜎 ) : = 𝔼 [ 𝑥 | 𝑦 ] , where 𝑦 = 𝑥 + 𝜎𝑤 , with 𝑥 ∼ 𝜋 and 𝑤 ∼ 𝛾 𝑑 independent. It is now well-established that, such denoising oracle enables efficient sampling of 𝜋 , well beyond the classic isoperimetric assumptions for fast relaxation of Langevin dynamics [17].

Suppose that we now measure 𝑦 = 𝐴𝑥 + 𝜎𝑤 , where again 𝑥 ∼ 𝜋 and 𝑤 ∼ 𝛾 𝑑 ′ are independent, but now 𝐴 ∈ ℝ 𝑑 ′ × 𝑑 is a known linear operator different from the identity. Given these linear measurements, we are now interested in the posterior sampling of 𝑥 given 𝑦 . This corresponds to the basic setup of linear inverse problems, encompassing many applications such as image inpainting, super-resolution, tomography, or source separation, to name a few. We are interested in the following natural question: can the power of denoising oracles be provably transferred to posterior sampling?

By Bayes' rule, the posterior distribution 𝜈 𝑦,𝐴 (denoted simply by 𝜈 when the context is clear) has density proportional to 𝜋 ( 𝑥 ) 𝑝 ( 𝑦 | 𝑥 ) ∝ exp n -1 2 𝜎 2 ∥ 𝐴𝑥 -𝑦 ∥ 2 o 𝜋 ( 𝑥 ) , and thus we can write it as a quadratic tilt of 𝜋 :

We readily identify certain regimes where sampling from 𝜈 might be easy:

If 𝜆 min ( 𝑄 ) is sufficiently large, 𝜆 min ( 𝑄 ) ≫ 1, then one expects 𝜈 to be strongly log-concave, enabling fast relaxation of Langevin dynamics.

If 𝜆 max ( 𝑄 ) is sufficiently small, 𝜆 max ( 𝑄 ) ≪ 1, then one expects 𝜈 ≈ 𝜋 in the appropriate sense, and therefore that samples from 𝜋 (which can be produced efficiently thanks to DO 𝜋 ) may be perturbed into samples from 𝜈 .

If 𝐴 ∈ O 𝑑 is a unitary transformation, then 𝑄 = 𝜎 -2 Id and the inverse problem reduces to isotropic Gaussian denoising, and is thus at first glance 'compatible' with the structure of the denoising oracle (such observation will be formalized later).

At this stage, we can already identify two key parameters of the problem that are likely to drive the difficulty of posterior sampling: on one hand, a proxy for the signal-to-noise ratio, measured e.g., by SNR : = 𝜆 min ( 𝑄 ) = 𝜆 min ( 𝐴 ) 2 𝜎 2 . On the other hand, the conditioning of the measurement operator 𝐴 , 𝜅 ( 𝐴 ) : = 𝜆 max ( 𝐴 ) 𝜆 min ( 𝐴 ) . As we shall see, these two characteristics of the linear measurement system will characterize necessary and sufficient conditions for probable posterior sampling. In the following, we assume the log of prior density 𝜋 is smooth and its Hessian exists ∀ 𝑥 ∈ ℝ 𝑑 .

Denoising Oracles and Score-Based Diffusion. Let us first review the natural connection between denoising and score-based generative modeling. Score-based diffusion models consist of two processes: a forward process that gradually adds noise to input data and a reverse process that learns to generate data by iteratively removing this noise. For example, one widely used family for the forward process is the Ornstein-Uhlenbeck (OU) process 1 :

where 𝑊 𝑡 is the standard Wiener process. We use 𝜋 𝑡 to denote the density of 𝑋 𝑡 , given by the action of the OU semigroup 𝜋 𝑡 = O ∗ 𝑡 𝜋 , defined by O 𝑡 𝑓 ( 𝑥 ) = 𝔼 [ 𝑓 ( 𝑋 𝑡 )| 𝑋 0 = 𝑥 ] , and explicitly given by dilated Gaussian convolutions, O ∗ 𝑡 : = C 𝛽 𝑡 D 𝛼 𝑡 , with 𝛽 𝑡 = 1 -𝑒 -2 𝑡 and 𝛼 𝑡 = 𝑒 𝑡 . With a sufficiently large 𝑇 , we know that 𝜋 𝑇 is close to the density of standard Gaussian 𝛾 𝑑 , owing to the exponential contraction of the OU semigroup: KL ( 𝜋 𝑇 | | 𝛾 𝑑 ) ≤ 𝑒 -𝑇 KL ( 𝜋 | | 𝛾 𝑑 ) .

Finally, the measure 𝜋 𝑡 solves the Fokker-Plank equation

By writing (2) as a transport equation 𝜕 𝑡 𝜋 𝑡 = ∇ · (( 𝑥 + ∇ log 𝜋 𝑡 ) 𝜋 𝑡 ) , we can formally reverse the transport starting at a large time 𝑇 and solving

Since ˜ 𝜋 𝑡 = 𝜋 𝑇 -𝑡 for 0 ≤ 𝑡 ≤ 𝑇 , introducing again the dissipative term leads to 𝜕 𝑡 ˜ 𝜋 𝑡 = ∇ · (-( 𝑥 + 2 ∇ log ˜ 𝜋 𝑡 ) ˜ 𝜋 𝑡 ) + Δ ˜ 𝜋 𝑡 , ˜ 𝜋 0 = 𝜋 𝑇 , which admits the SDE representation

In practice, one runs this reverse diffusion starting from ˜ 𝑋 0 ∼ 𝛾 𝑑 rather than ˜ 𝑋 0 ∼ 𝜋 𝑇 . However, by the data-processing inequality, we have that KL ( 𝜋 | | ˜ 𝜋 𝑇 ) ≤ KL ( 𝜋 𝑇 | | 𝛾 𝑑 ) = 𝑂 ( 𝑒 -𝑇 ) , thus incurring in insignificant error. To facilitate later exposition, we write the above process reverse in time [2, 31]

and interpret the data generation process as running the reverse SDE from 𝑇 back to 0.

By the well-known Tweedie's formula, and up to time reparametrisation, the denoising oracle is equivalent to the time-dependent score ∇ log 𝜋 𝑡 :

Fact 1 (Tweedie's formula, [32]) . We have ∇ log 𝜋 𝑡 ( 𝑥 ) = -( 1 -𝑒 -2 𝑡 ) -1 ( 𝑥 -𝑒 -2 𝑡 DO 𝜋 ( 𝑥, 1 -𝑒 2 𝑡 )) .

Log-Sobolev Inequality and Fast Relaxation of Langevin Dynamics. Given a Gibbs distribution 𝜋 ∈ P( ℝ 𝑑 ) of the form 𝜋 ∝ 𝑒 -𝑓 , a powerful and versatile method to sample from 𝜋 is to consider the Langevin dynamics

where 𝜇 0 is an arbitrary initial distribution. It is easy to verify that these dynamics define a Markov process that admits 𝜋 as its unique invariant measure. Perhaps less obvious is the fact that the Fokker-Plank equation associated with eq. (6), given by 𝜕 𝑡 𝜇 = ∇· (∇ 𝑓 𝜇 ) + Δ 𝜇 (and where 𝜇 𝑡 is the law of 𝑋 𝑡 ) is in fact a Wasserstein gradient flow for the relative entropy functional KL ( 𝜇 | | 𝜋 ) [38]. Under this interpretation, one can quantify the convergence of Langevin dynamics to their invariant measure, i.e., its time to relaxation, by establishing a sharpness or Polyak-Lowacjevitz (PL)-type inequality. Indeed, by noticing that 𝑑 𝑑𝑡 KL ( 𝜇 | | 𝜋 ) = -I ( 𝜇 | | 𝜋 ) , where I ( 𝜇 | | 𝜋 ) = 𝔼 𝜇 [∥∇ log 𝜇 - ∇ log 𝜋 ∥ 2 ] is the Fisher divergence, the PL-type inequality in this setting is given by the Logarithmic Sobolev Inequality (LSI): we say that a measure 𝜋 satisfies LSI ( 𝜌 ) if for any 𝜇 ∈ P( ℝ 𝑑 ) it holds KL ( 𝜇 | | 𝜋 ) ≤ 1 2 𝜌 I ( 𝜇 | | 𝜋 ) .

This functional inequality directly implies KL ( 𝜇 𝑡 | | 𝜋 ) ≤ 𝑒 -2 𝜌𝑡 KL ( 𝜇 0 | | 𝜋 ) . While for general 𝜋 it is typically hard to establish the LSI, there are two important sources of structure that lead to quantitative (i.e., 𝜌 = Ω 𝑑 ( 1 ) ) bounds: when 𝜋 is a product measure 𝜋 = ˜ 𝜋 ⊗ 𝑑 (in which case 𝜋 satisfies LSI with the same constant as ˜ 𝜋 ), and when 𝜋 is strongly log-concave 2 , i.e., -∇ 2 log 𝜋 ( 𝑥 ) ⪰ 𝛼𝐼 for all 𝑥 , in which case the celebrated Bakry-Emery criterion [3] states that 𝜌 ≥ 𝛼 .

3 Evidence of Computational Hardness in the Generic Case

Westart our analysis of posterior sampling by discussing negative results for the general case. Recently, [30] established computational lower bounds for this task using cryptographic hardness assumptions. In this section, we complement these results by illustrating a correspondence with sampling problems on Ising models, leading to an arguably simpler conclusion.

For this purpose, consider ¯ 𝜋 = Unif ({± 1 } 𝑑 ) the uniform measure of the hypercube. Quadratic tilts of ¯ 𝜋 define generic Ising models, a rich and intricate class of high-dimensional distributions. Since ¯ 𝜋 is a product measure, its associated denoising oracle becomes a separable function that can be computed in closed-form:

Given a symmetric matrix 𝑄 ∈ ℝ 𝑑 × 𝑑 , an Ising model is given by the tilt T 𝑄 ¯ 𝜋 ∈ P({± 1 } 𝑑 ) . In our setting, we can thus view such models as the posterior distribution of a linear inverse problem associated with the uniform prior ¯ 𝜋 . Efficiently sampling from Ising models is a fundamental question at the interface of statistical physics and high-dimensional probability, and several works provide evidence of computational hardness under a variety of settings.

Notably, by treating 𝑄 as the adjacency matrix of a regular graph, [27] establishes that sampling from 𝜈 is impossible for 𝜆 max ( 𝑄 ) -𝜆 min ( 𝑄 ) ≥ 2 + 𝜀 , for any 𝜀 > 0, unless NP = RP . In other words, for poorly conditioned tilt 𝑄 (in the sense that there is a large gap between the smallest and largest eigenvalue), there is no efficient posterior sampling algorithm, even with the knowledge of the prior denoising oracle . The threshold can even be reduced to 1 + 𝜀 by using a weaker notion of computational hardness [42], given by the low-degree polynomial method [4, 43]. Remarkably, this threshold agrees with the current best-known algorithmic results for sampling generic Ising models with Glauber dynamics [25, 1]. Finally, we also mention that when 𝑄 is a random Gaussian symmetric matrix, the associated so-called Sherrington-Kirkpatrick (SK) model, has been analyzed with dedicated algorithms. In this setting, it is also known [24] that 'stable' sampling algorithms fail to sample from the SK model as soon as 𝜆 max ( 𝑄 ) -𝜆 min ( 𝑄 ) > 1. In summary, we have:

Theorem 3 (Computational Hardness of Sampling Ising Models, [42, 27]) . There exist no generalpurpose, efficient posterior sampling algorithms, for 𝑄 sufficiently ill-conditioned, even under the knowledge of the prior denoising oracle.

One could wonder whether this computational hardness comes from the discrete nature of the hypercube. It is not hard to observe that this is not the case: the following proposition, proved in Appendix A, shows a simple reduction from a model where the prior ¯ 𝜋 is replaced by a smooth mixture of Gaussians 𝜋 centered at the corners of the hypercube, with variance 𝛿 .

Proposition 4 (Hardness extends to smooth priors) . Assume a posterior sampler exists for the smooth prior with TV error 𝜖 and 𝛿 = 𝑜 ( 𝑑 -1 / 2 ) . Then there exists a sampler for the associated Ising model with TV error 1 . 1 𝜖 .

In conclusion, one cannot hope for a generic method that leverages the prior denoising oracle to perform efficient posterior sampling, as soon as 𝐴 is mildly ill-conditioned. Thus, in order to perform provable posterior sampling, one needs to either (i) constraint the measurements, or (ii) exploit structural properties of the prior measure. In the following, we focus on (i), namely providing guarantees for well-conditioned 𝐴 that leverage the OU semigroup for generic prior distributions.

4 Posterior Sampling via Tilted Transport

We now present a simple method that reduces the original posterior sampling problem to another posterior sampling problem with more benign geometry, by leveraging the shared quadratic structure of the posterior tilt and the OU semigroup. The power of the denoising oracle to perform sampling of the prior 𝜋 comes from its ability to run the transport equation (3) in either direction, and leveraging the fact that sampling from 𝜋 𝑇 is easy. To transfer this power to posterior sampling, we can thus attempt to replicate this scheme: can we implement a transport between the posterior 𝜈 and a terminal measure 𝜈 𝑇 that is easy to sample, that only relies on the pre-trained prior DO 𝜋 ?

A Motivating Example. Consider first the denoising setting: 𝑦 = 𝑥 + 𝜎𝑤 . According to the forward process, we have 𝑝 ( 𝑋 𝑠 | 𝑋 0 ) 𝑑 = N( 𝑒 -𝑠 𝑋 0 , ( 1 -𝑒 -2 𝑠 ) 𝐼 𝑑 ) . Introduce 𝑇 ∗ > 0 and define ˜ 𝑦 = 𝑒 -𝑇 ∗ 𝑦 = 𝑒 -𝑇 ∗ 𝑥 + 𝑒 -𝑇 ∗ 𝜎𝑤 such that 𝑝 ( ˜ 𝑦 | 𝑥 ) 𝑑 = N( 𝑒 -𝑇 ∗ 𝑥, 𝑒 -2 𝑇 ∗ 𝜎 2 𝐼 𝑑 ) . We match the variance by letting 𝑒 -2 𝑇 ∗ 𝜎 2 = 1 -𝑒 -2 𝑇 ∗ , i.e., 𝑇 ∗ = 1 2 log ( 1 + 𝜎 2 ) , then we have 𝑝 ( ˜ 𝑦 | 𝑥 ) = 𝑝 ( 𝑋 𝑇 ∗ | 𝑋 0 ) , which gives

( 𝑥, ˜ 𝑦 ) 𝑑 = ( 𝑋 0 , 𝑋 𝑇 ∗ ) . Therefore, to perform the posterior sampling 𝑝 ( 𝑥 | ˜ 𝑦 ) , we only need to do the sampling 𝑝 ( 𝑋 0 | 𝑋 𝑇 ∗ ) , which can be achieved through the reverse SDE. Specifically, let 𝑋 𝑇 ∗ = 𝑒 -𝑇 ∗ 𝑦 and run the reverse SDE (5) from 𝑇 ∗ to 0, then 𝑋 0 will be the desired posterior.

Hamilton-Jacobi Equation and Quadratic Tilts. If 𝜋 𝑡 solves the Fokker-Plank eq. (2), then one can verify that the time-varying potentials 𝑓 𝑡 : = log 𝜋 𝑡 solve the viscous Hamilton-Jacobi PDE (HJE)

Now, the posterior 𝜈 = T 𝑄,𝑏 𝜋 creates an additional quadratic term in the potential log 𝜈 = 𝑓 -1 2 𝑥 ⊤ 𝑄𝑥 + 𝑥 · 𝑏 . One could naively hope that this additive quadratic term would still define a solution of the HJE with the tilted initial condition ˜ 𝑓 0 = log 𝜈 -or equivalently that the measure T 𝑄,𝑏 𝜋 𝑡 solves the transport equation (3). Unfortunately, due to the nonlinearity in (8) brought by the terms ∥∇ 𝑓 𝑡 ∥ 2 , this is not the case. However, as we shall see now, this is not far from being true: one just needs to consider time-varying quadratic tilts in order to satisfy the HJE.

Tilt Transport Equation. We consider then a one-parameter family of distributions 𝜈 𝑡 of the form 𝜈 𝑡 = T 𝑄 𝑡 ,𝑏 𝑡 𝜋 𝑡 , with 𝑄 0 = 𝑄 and 𝑏 0 = 𝑏 . As it turns out, one can ensure that log 𝜈 𝑡 solves the HJE associated with the reverse process by asking that 𝑄 𝑡 , 𝑏 𝑡 satisfy the first-order ODE

Theorem 5 (Tilted Transport) . Assume 𝑡 < 𝑇 ∗ such that the ODE (9) is well-defined on [ 0 , 𝑡 ] . By initializing 𝑋 𝑡 ∼ 𝜈 𝑡 and run the reverse SDE (5) from 𝑡 to 0, we have 𝑋 𝑠 ∼ 𝜈 𝑠 for 𝑠 ∈ [ 0 , 𝑡 ] , specifically, 𝑋 0 gives the desired posterior.

Solution to eq. (9). Without loss of generality, we assume 𝑑 ′ ≤ 𝑑 , and the observation operator 𝐴 ∈ ℝ 𝑑 ′ × 𝑑 has a general singular value decomposition form 𝐴 = 𝑈 Σ 𝑉 ⊤ with non-zero singular values 𝜆 1 ≥ 𝜆 2 ≥ · · · ≥ 𝜆 𝑑 ′ > 0. By diagonalizing 𝑄 and solving the scalar ODE / 𝑞 𝑡 = 2 ( 1 + 𝑞 𝑡 ) 𝑞 𝑡

solution is defined up to the blowup time 𝑇 ∗ : = 1 2 log ( 1 + 𝜎 2 / 𝜆 2 1 ) = 1 2 log ( 1 + 𝜆 max ( 𝑄 ) -1 ) . 𝑏 𝑡 can be further solved from the solution 𝑄 𝑡 ; see Appendix B.2 for more details.

With the explicit solution of 𝑄 𝑡 , 𝑏 𝑡 , we can interpret the term exp (-1 2 𝑥 ⊤ 𝑄 𝑡 𝑥 + 𝑥 ⊤ 𝑏 𝑡 ) as the likelihood of the inverse problem with respect to the new prior distribution 𝜋 𝑡 and the corresponding operator. Based on this observation and Theorem 5, we have the following corollary, transforming the original posterior sampling problem to a new posterior sampling problem exactly. We remark that when 𝐴 is identity, the corollary recovers the analysis we have in the motivating example; see Appendix B.2 for the proof and more discussions.

Corollary 6. Fix 𝑡 < 𝑇 ∗ . Sampling from the original posterior is equivalent to a two-step process: first, sample from a new posterior 𝑋 𝑡 ∼ 𝜈 𝑡 , and then run the reverse SDE (5) from time 𝑡 to 0.

5 Quantitative Conditions for Provable Sampling

Now we show that the new posterior sampling problem described above is provably easier to sample than the original posterior sampling problem from two aspects. On the one hand, the (negative) eigenvalues of the quadratic tilt -1 2 𝑥 ⊤ 𝑄 𝑡 𝑥 + 𝑥 ⊤ 𝑏 𝑡 become more negative, essentially meaning that the SNR of the new observation model becomes larger. To be more specific, as 𝑡 → 𝑇 ∗ , 𝜆 min ( 𝑄 𝑡 ) → 1 + 𝜆 max ( 𝑄 ) -1 𝜆 min ( 𝑄 ) -1 -𝜆 max ( 𝑄 ) -1 > 𝜆 min ( 𝑄 ) . On the other hand, the new prior distribution 𝜋 𝑇 ∗ becomes closer to a single-mode Gaussian (recall that KL ( 𝜋 𝑡 | | 𝛾 𝑑 ) = 𝑂 ( 𝑒 -𝑡 ) ), which is also easier to sample. Combining these two arguments, we expect that, as 𝑡 increases, 𝜈 𝑡 becomes easier to sample due to easier prior and easier likelihood:

Let us now quantify the above intuition.

Sufficient Conditions for Strong Log-Concavity of 𝜈 𝑇 ∗ . We start by giving a simple sufficient condition that ensures that 𝜈 𝑇 ∗ is strongly log-concave. As discussed earlier, this ensures fast relaxation of the Langevin dynamics, enabling efficient sampling from 𝜈 𝑇 ∗ - and therefore of 𝜈 as per Corollary 6. For that purpose, given the prior 𝜋 ∈ P( ℝ 𝑑 ) and 𝑡 ≥ 0, we define

where the covariance is given by Cov [ 𝜇 ] = 𝔼 𝑥 ∼ 𝜇 [ 𝑥𝑥 ⊤ ] - ( 𝔼 𝑥 ∼ 𝜇 [ 𝑥 ])( 𝔼 𝑥 ∼ 𝜇 [ 𝑥 ]) ⊤ . 𝜒 𝑡 ( 𝜋 ) thus measures the largest 'spread' of any tilted measure of the form T 𝑡,𝑥 𝜋 . Equipped with this definition, we have the following sufficient condition to ensure that 𝜈 𝑇 ∗ is strongly log-concave:

Proposition 7 (Strong Log-Concavity of 𝜈 𝑇 ∗ ) . 𝜈 𝑇 ∗ is strongly log-concave if

The proof is in Appendix C. It relates two parameters of the measurement process, the condition number of 𝐴 and the signal-to-noise ratio in terms of ∥ 𝑄 ∥ , with a geometric property of the prior, the spread function 𝜒 𝑡 ( 𝜋 ) . While this function is not immediately transparent, the following examples illuminate its behavior in reprsentative high-dimensional settings.

Example 8 (Behavior of 𝜒 𝑡 ( 𝜋 ) ) . We have the following examples

(i) Gaussian measure: 𝜒 𝑡 ( 𝛾 𝑑 ) = 1 1 + 𝑡 .

(ii) Compactly Supported Gaussian Mixture: If 𝜇 is compactly-supported in a ball of radius 𝑅 and 𝛿 ≥ 0 , then 𝜒 𝑡 ( 𝜇 ★ 𝛾 𝛿 ) ≤ GLYPH<0> 𝑅 1 + 𝛿𝑡 GLYPH<1> 2 + 𝛿 1 + 𝛿𝑡 .

(iii) Tensorization: If 𝜇 = 𝜇 1 ⊗ 𝜇 2 · · · ⊗ 𝜇 𝑑 , then 𝜒 𝑡 ( 𝜇 ) = max 𝑖 𝜒 𝑡 ( 𝜇 𝑖 ) .

(iv) Uniform measure on hypercube: If 𝜋 is uniform on the hypercube H 𝑑 , then 𝜒 𝑡 ( 𝜋 ) = 1 .

Ising Models. As a direct consequence of Proposition 7 and Example 8 (iv), we establish a sampling guarantee for Ising models:

Corollary 9 (Tilted Transport for the Ising Model) . Let 𝜋 be the uniform measure on the hypercube, and 𝑄 such that 𝜆 max ( 𝑄 ) -𝜆 min ( 𝑄 ) < 1 . Then 𝜈 𝑇 ∗ is strongly log-concave, and therefore 𝜈 = T 𝑄 𝜋 can be sampled efficiently (in continuous-time).

This result thus establishes that Ising models admit an efficient continuous-time procedure for sampling provided their spectrum satisfies 𝜆 max ( 𝑄 ) -𝜆 min ( 𝑄 ) < 1, thus precisely matching the threshold of [25, 1] achieved by Glauber dynamics, as well as the low-degree prediction from [42]. We remark though that our procedure is not (yet) algorithmic; a careful analysis of the discrete-time complexity and the approximation rates is beyond the current scope, but our next endeavor. If one specializes the previous result to the SK model, the equivalent inverse temperature that guarantees sampling is 𝛽 ∗ = 1 / 4, which remains below 𝛽 = 1, the threshold of the hard phase. For this threshold, dedicated AMP-based sampling succeeds [24, 14]. We also remark that, in itself, it should not come as a surprise that 𝜈 may be sampled under these conditions, since [5] already established an LSI on 𝜈 directly, using an entropy decomposition along the so-called Polchinski renormalization group [6] that refines our Bakry-Emery criterion. In this context, it would be interesting to explore whether this refined criterion could be applied to 𝜈 𝑇 ∗ to improve upon Proposition 7 under appropriate conditions.

Gaussian Mixtures. By applying Proposition 7 to Example 8 (ii), we directly obtain the following guarantee for generic comptactly supported Gaussian mixtures:

Corollary 10 (Tilted Transport for Gaussian Mixtures) . If 𝜋 = 𝜇 ★ 𝛾 𝛿 and diam ( supp ( 𝜇 )) ≤ 𝑅 , then 𝜈 𝑇 ∗ is strongly log-concave if

It also holds when 𝛿 = 0 and the prior 𝜋 is any distribution with a bounded support radius 𝑅 .

Figure 2 displays several contours of the condition in eq. (12) as a function of SNR and 𝜅 ( 𝐴 ) . Each 𝑈 -shaped contour is determined by a combination of 𝛿 and 𝑅 , which uniquely characterizes the prior. For all points ( ( SNR ) , 𝜅 ( 𝐴 ) ) outside of a contour, representing a specific inverse problem, 𝜈 𝑇 ∗ is strongly log-concave and thus easy to sample. Given an observation model where both SNR and 𝜅 ( 𝐴 ) are fixed, it is straightforward to see that the condition in eq. (12) is more readily satisfied as 𝛿 increases and 𝑅 decreases. Figure 2 also confirms this result since as 𝛿 increases or 𝑅 decreases, the 𝑈 -shaped contour shrinks and the region of easy to sample expands. Now we discuss the implications in the reverse scenario where the prior is fixed and the observation model is adjusted. If we look at Figure 2 horizontally, we know that given a prior and 𝜅 ( 𝐴 ) , the target posterior can be reliably sampled if the SNR is either sufficiently low or high, with the region of mid-SNR being challenging. The closer 𝜅 ( 𝐴 ) is to 1, the smaller this challenging region is. When the problem is denoising such that 𝜅 ( 𝐴 ) = 1, the challenging region vanishes, and sampling the posterior is straightforward using the denoising oracle, as previously explained.

10

6

10

4

10

2

10 0

10 2

SNR

1

10

20

(

A

)

Easy

Hard

Easy

=1,

R

=2

=1,

R

=4

=1,

R

=8

=1,

R

=16

10

6

10

4

10

2

10 0

10 2

SNR

1

10

20

(

A

)

Easy

Hard

Easy

=64,

R

=10

=16,

R

=10

=4,

R

=10

=1,

R

=10

Comparisons. 1. (with Langevin dynamics) As introduced above, Langevin dynamics and its discretized version, Langevin Monte Carlo (LMC) [52, 46] serve as ideal baselines for efficient posterior sampling in high SNR regimes where the posterior becomes strongly log-concave. Proposition 7

demonstrates that our tilted transport technique enables provably efficient sampling from a broader range of prior distributions compared to traditional Langevin dynamics without a denoising oracle. Particularly, in low SNR regimes where conventional Langevin dynamics struggle with severe non-log-concavity and slow mixing times, tilted transport can transform the sampling challenge into a tractable problem for log-concave distributions.

2. (with Importance Sampling) In the low SNR regime with a well-conditioned 𝐴 , the posterior measure can be viewed as a small perturbation of the prior. As such, a natural baseline for posterior sampling is importance sampling using the prior as a proposal - for which samples can be efficiently obtained thanks to the denoising oracle and the variance of sample weights is small. However, as detailed in Appendix C.2, the sampling complexity is exponential with the SNR when the SNR is sufficiently large, assuring the failure of the importance sampling on this extreme.

Stability. In the numerical implementation of the boosted posterior, we often encounter specific errors. Appendix C.3 provides a stability analysis of the two-step process outlined in Corollary 6, focusing on initialization error and score error, and demonstrates that the quality of the final samples is robust with respect to these errors.

6 Numerical Experiments

Our theory above demonstrates that 𝜈 𝑇 ∗ is provably easier to sample than the original posterior 𝜈 . Thus, given a baseline sampling algorithm Alg , we can first sample from the boosted posterior and then apply the denoising oracle to obtain the final sample, rather than directly sampling from 𝜈 using Alg . Algorithm 8 provides a complete description of this approach using tilted transport. In this instance, we use Euler discretization with equal time steps to transport samples from from T 𝑄 ˜ 𝑇 ,𝑏 ˜ 𝑇 𝜋 ˜ 𝑇 to T 𝑄,𝑏 𝜋 , though alternative time integrators and grids can also be applied.

We now validate our theoretical results by applying Algorithm 8 to the Gaussian mixture model in high dimensions, using LMC as the baseline algorithm. Same to the models considered in [13], the prior distribution is a mixture of 25 components with known means and variances (see Figure 1 for a 2D visualization and Appendix E for detailed settings). We examine three cases where 𝑑 = 20 , 40, and 80. In each scenario, we set 𝑑 ′ = 𝑑 , fix 𝜅 = 20, and vary the SNR from 10 -5 to 10 -1 . We use the Sliced Wasserstein distance as a principled error metric, computed from samples obtained by our algorithms and samples directly from the analytically computed posterior Gaussian mixture. Figure 3 illustrates the comparison between LMC and LMC boosted by tilted transport. As analyzed earlier, LMC is effective when the SNR is high enough to render the target posterior strongly log-concave, but its error quickly increases as the SNR decreases. In contrast, the tilted transport enhances LMC to perform well in both low and high SNR regimes with small sampling errors. Its performance is weaker in the mid-SNR regime compared to the extremes, as predicted by Corollary 10. However, the tilted transport still improves upon LMC in this challenging regime by boosting effective SNR and simplifying the prior.

We further test tilted transport when 𝑑 ′ < 𝑑 , in which 𝜆 min ( 𝑄 𝑡 ) remains zero but the signal corresponding to the non-zero eigenvalues still gets enhanced. Therefore, although it becomes more

10

5

10

4

10

3

10

2

10

1

SNR

0

2

4

6

8

10

Sliced Wasserstain distance

d

=20

Boosted Langevin

10

5

10

4

10

3

10

2

10

1

SNR

d

=40

10

5

10

4

10

3

10

2

10

1

SNR

d

=80

difficult for 𝜈 𝑇 ∗ to be strongly log-concave, the tilted transport can still make the new posterior easier to sample even if it is not strongly log-concave yet. Detailed results are reported in Appendix E.1.

7 Discussion and Future Work

In this paper, we theoretically investigate posterior sampling using powerful priors provided by denoising oracles. We demonstrate that efficient posterior sampling can be challenging even with a perfect denoising oracle for the prior. To achieve provable posterior sampling, one must either constrain the measurements or leverage the structural properties of the prior. We focus on the former, showing that well-conditioned measurements enable the proposed tilted transport technique to simplify the task significantly, providing a clear, verifiable condition for efficient sampling, as demonstrated on the Ising model. Several questions remain open: Can this approach provably handle poorly-conditioned measurements, such as inpainting? Can it be extended from linear to nonlinear inverse problems? We show in Appendix D how to extend the tilted transport beyond the condition of Proposition 7 via 'iterated tilts', at the expense of introducing approximation errors. On the theory side, the key object underlying the success of the tilted transport is the spread 𝜒 𝑡 ( 𝜋 ) ; in particular, understanding when one can remove dimension dependence is an interesting question. We also aim to systematically evaluate the empirical performance of tilted transport in imaging and scientific computing. Appendix E.2 provides a proof-of-concept for various imaging tasks. We suspect that tilted transport could even improve existing posterior point estimate methods by boosting SNR and enabling proper uncertainty quantification through the reverse process.

References

N. Anari, V. Jain, F. Koehler, H. T. Pham, and T.-D. Vuong. Entropic independence I: Modified log-Sobolev inequalities for fractionally log-concave distributions and high-temperature ising models. arXiv preprint arXiv:2106.04105 , 2021.

B. D. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications , 12(3):313-326, 1982.

D. Bakry and M. Émery. Diffusions hypercontractives. In J. Azéma and M. Yor, editors, Séminaire de Probabilités XIX 1983/84 , pages 177-206. Springer Berlin Heidelberg, Berlin, Heidelberg, 1985.

B. Barak, S. Hopkins, J. Kelner, P. K. Kothari, A. Moitra, and A. Potechin. A nearly tight sum-of-squares lower bound for the planted clique problem. SIAM Journal on Computing , 48 (2):687-735, 2019.

R. Bauerschmidt and T. Bodineau. A very simple proof of the LSI for high temperature spin systems. Journal of Functional Analysis , 276(8):2582-2588, 2019.

R. Bauerschmidt, T. Bodineau, and B. Dagallier. Stochastic dynamics and the Polchinski equation: an introduction. arXiv preprint arXiv:2307.07619 , 2023.

J. Benton, V. De Bortoli, A. Doucet, and G. Deligiannidis. Linear convergence bounds for diffusion models via stochastic localization. arXiv preprint arXiv:2308.03686 , 2023.

J. Benton, V. De Bortoli, A. Doucet, and G. Deligiannidis. Nearly d-linear convergence bounds for diffusion models via stochastic localization. In The Twelfth International Conference on Learning Representations , 2024.

P. Bickel, B. Li, and T. Bengtsson. Sharp failure rates for the bootstrap particle filter in high dimensions. In Pushing the limits of contemporary statistics: Contributions in honor of Jayanta K. Ghosh , volume 3, pages 318-330. Institute of Mathematical Statistics, 2008.

S. G. Bobkov, I. Gentil, and M. Ledoux. Hypercontractivity of Hamilton-Jacobi equations. Journal de Mathématiques Pures et Appliquées , 80(7):669-696, 2001.

H. J. Brascamp and E. H. Lieb. On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation. Journal of functional analysis , 22(4):366-389, 1976.

A. Cabezas, A. Corenflos, J. Lao, and R. Louf. Blackjax: Composable Bayesian inference in JAX, 2024.

G. Cardoso, Y. Janati El idrissi, S. L. Corff, and E. Moulines. Monte Carlo guided denoising diffusion models for Baysian linear inverse problems. In The Twelfth International Conference on Learning Representations , 2024.

M. Celentano. Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energy. The Annals of Probability , 52(3):923-954, 2024.

S. Chatterjee and P. Diaconis. The sample size required in importance sampling. The Annals of Applied Probability , 28(2):1099-1135, 2018.

H. Chen, H. Lee, and J. Lu. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning , pages 4735-4763. PMLR, 2023.

S. Chen, S. Chewi, J. Li, Y. Li, A. Salim, and A. Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations , 2023.

S. Chen, S. Chewi, H. Lee, Y. Li, J. Lu, and A. Salim. The probability flow ODE is provably fast. Advances in Neural Information Processing Systems , 36, 2024.

S. Chen, V. Kontonis, and K. Shah. Learning general gaussian mixtures with efficient score matching. arXiv preprint arXiv:2404.18893 , 2024.

J. Choi, S. Kim, Y. Jeong, Y. Gwon, and S. Yoon. ILVR: Conditioning method for denoising diffusion probabilistic models. arXiv preprint arXiv:2108.02938 , 2021.

H. Chung, B. Sim, and J. C. Ye. Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12413-12422, 2022.

H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations , 2023.

Z. Dou and Y. Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations , 2024.

A. El Alaoui, A. Montanari, and M. Sellke. Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localization. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 323-334. IEEE, 2022.

R. Eldan, F. Koehler, and O. Zeitouni. A spectral condition for spectral gap: fast mixing in high-temperature ising models. Probability theory and related fields , 182(3):1035-1051, 2022.

B. T. Feng, J. Smith, M. Rubinstein, H. Chang, K. L. Bouman, and W. T. Freeman. Score-based diffusion models as principled priors for inverse imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10520-10531, 2023.

A. Galanis, D. Štefankovič, and E. Vigoda. Inapproximability of the partition function for the antiferromagnetic ising and hard-core models. Combinatorics, Probability and Computing , 25 (4):500-559, 2016.

M. Gelbrich. On a formula for the L2 wasserstein metric between measures on euclidean and Hilbert spaces. Mathematische Nachrichten , 147(1):185-203, 1990.

A. Graikos, N. Malkin, N. Jojic, and D. Samaras. Diffusion models as plug-and-play priors. Advances in Neural Information Processing Systems , 35:14715-14728, 2022.

S. Gupta, A. Jalal, A. Parulekar, E. Price, and Z. Xun. Diffusion posterior sampling is computationally intractable. arXiv preprint arXiv:2402.12727 , 2024.

U. G. Haussmann and E. Pardoux. Time reversal of diffusions. The Annals of Probability , pages 1188-1205, 1986.

R. Herbert. An empirical bayes approach to statistics. In Proceedings of the third berkeley symposium on mathematical statistics and probability , volume 1, pages 157-163, 1956.

J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

M. D. Hoffman and A. Gelman. The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research , 15(1):1593-1623, 2014.

R. Holley and D. Stroock. Logarithmic Sobolev inequalities and stochastic ising models. Journal of Statistical Physics , 46(5-6):1159-1194, Mar. 1987.

A. Jalal, M. Arvinte, G. Daras, E. Price, A. G. Dimakis, and J. Tamir. Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems , 34:14938-14954, 2021.

Y. Janati, A. Durmus, E. Moulines, and J. Olsson. Divide-and-conquer posterior sampling for denoising diffusion priors. arXiv preprint arXiv:2403.11407 , 2024.

R. Jordan, D. Kinderlehrer, and F. Otto. The variational formulation of the fokker-planck equation. SIAM journal on mathematical analysis , 29(1):1-17, 1998.

T. Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4401-4410, 2019.

B. Kawar, G. Vaksman, and M. Elad. SNIPS: Solving noisy inverse problems stochastically. Advances in Neural Information Processing Systems , 34:21757-21769, 2021.

B. Kawar, M. Elad, S. Ermon, and J. Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems , 35:23593-23606, 2022.

D. Kunisky. Optimality of Glauber dynamics for general-purpose Ising model sampling and free energy approximation. Proceedings of the 2024 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 5013-5028, 2023.

D. Kunisky, A. S. Wein, and A. S. Bandeira. Notes on computational hardness of hypothesis testing: Predictions using the low-degree likelihood ratio. In ISAAC Congress (International Society for Analysis, its Applications and Computation) , pages 1-50. Springer, 2019.

H. Lee, J. Lu, and Y. Tan. Convergence for score-based generative modeling with polynomial complexity. Advances in Neural Information Processing Systems , 35:22870-22882, 2022.

H. Lee, J. Lu, and Y. Tan. Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory , pages 946-985. PMLR, 2023.

Y.-A. Ma, T. Chen, and E. Fox. A complete recipe for stochastic gradient MCMC. Advances in neural information processing systems , 28, 2015.

M. Mardani, J. Song, J. Kautz, and A. Vahdat. A variational perspective on solving inverse problems with diffusion models. arXiv preprint arXiv:2305.04391 , 2023.

X. Meng and Y. Kabashima. Diffusion model based posterior sampling for noisy linear inverse problems. arXiv preprint arXiv:2211.12343 , 2022.

A. Montanari and Y. Wu. Provably efficient posterior sampling for sparse linear regression via measure decomposition. arXiv preprint arXiv:2406.19550 , 2024.

B. Oksendal. Stochastic differential equations: an introduction with applications . Springer Science & Business Media, 2013.

F. Otto and C. Villani. Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. Journal of Functional Analysis , 173(2):361-400, 2000.

G. O. Roberts and R. L. Tweedie. Exponential convergence of langevin distributions and their discrete approximations. Bernoulli , pages 341-363, 1996.

C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet, and M. Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings , pages 1-10, 2022.

S. Shoushtari, J. Liu, and U. S. Kamilov. Dolph: Diffusion models for phase retrieval. arXiv preprint arXiv:2211.00529 , 2022.

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning , pages 2256-2265. PMLR, 2015.

J. Song, A. Vahdat, M. Mardani, and J. Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations , 2022.

J. Song, Q. Zhang, H. Yin, M. Mardani, M.-Y. Liu, J. Kautz, Y. Chen, and A. Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning , pages 32483-32498. PMLR, 2023.

Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 , 2020.

Y. Song, C. Durkan, I. Murray, and S. Ermon. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems , 34:1415-1428, 2021.

Y. Song, L. Shen, L. Xing, and S. Ermon. Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations , 2022.

Y. Sun, Z. Wu, Y. Chen, B. T. Feng, and K. L. Bouman. Provable probabilistic imaging using score-based generative priors. IEEE Transactions on Computational Imaging , 2024.

L. Wu, B. Trippe, C. Naesseth, D. Blei, and J. P. Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. In Advances in Neural Information Processing Systems , volume 36, pages 31372-31403, 2023.

X. Xu and Y. Chi. Provably robust score-based diffusion posterior sampling for plug-and-play image reconstruction. arXiv preprint arXiv:2403.17042 , 2024.

A Proof of Proposition 4

Let 𝛿 > 0 and ¯ 𝜋 be the uniform measure in the 𝑑 -dimensional hypercube. Consider a Gaussian mixture prior 𝜋 defined as 𝜋 = ¯ 𝜋 ★ 𝛾 𝛿 .

Since both ¯ 𝜋 and 𝛾 𝛿 are product measures, it follows that 𝜋 is also a product measure, and therefore its denoising oracle DO 𝜋 is explicitly given by DO 𝜋 ( 𝑦, 𝑡 ) 𝑖 = 𝜓 ( 𝑦 𝑖 , 𝑡 ) , with

Observe that 𝑞 𝑣,𝑡 is the density of a Gaussian mixture in ℝ of the form 𝛼 N( 𝑏 -, 𝜎 ) + ( 1 -𝛼 )N( 𝑏 + , 𝜎 ) , with parameters

and thus 𝜓 ( 𝑣, 𝑡 ) = 𝛼𝑏 - + ( 1 -𝛼 ) 𝑏 + .

Let us now denote by 𝜇 𝑄 the target Ising model, supported in the 𝑑 -dimensional hypercube, and define the approximation 𝜇 𝜎 𝑄 : = T 𝑄 𝜋 . Suppose that there is an algorithm A that leverages the denoising oracle of 𝜋 that can efficiently sample from 𝜇 𝜎 𝑄 : its law ˆ 𝜇 satisfies TV ( 𝜇 𝜎 𝑄 , ˆ 𝜇 ) ≤ 𝜖 with runtime polynomial in 𝑑 and log ( 𝜖 -1 ) .

Let now 𝑅 ( 𝑥 ) = sign ( 𝑥 ) , and consider the sampler 𝑅 ◦ A , which is now supported in the hypercube. By the triangle and data-processing inequalities, we directly have

It remains to bound the second term in the RHS. We have to compare two measures in the hypercube. For 𝜎 ∈ H 𝑑 : = {± 1 } 𝑑 , they are given respectively by

Applying the Laplace approximation into each integral we obtain, as 𝛿 → 0,

where 𝜎 ⊕ 𝑧 is the XOR, and | 𝜎 -𝑧 | is the Hamming distance. We thus have, for any 𝜎 ∈ H 𝑑 ,

It follows that we can write 𝑅 # 𝜇 𝛿 𝑄 ( 𝜎 ) as

with a relative error

It follows that

and thus if 𝛿 ≪ 1 √ 𝑑 , we have a negligible TV approximation.

B Proofs of Section 4

B.1 Proof of Theorem 5

Proof. We denote the time-dependent score function ∇ log 𝜋 𝑡 ( 𝑥 ) by 𝑠 𝑡 ( 𝑥 ) . As derived in Section 2, if we initialize 𝑋 𝜏 according to density 𝜌 𝜏 and run the reverse SDE eq. (5), the density of 𝑋 𝑡 for 𝑡 ≤ 𝜏 , denoted by 𝜌 𝑡 , satisfies the backward PDE:

We need to verify that 𝜈 𝑡 satisfies the above PDE.Note that a general positive function 𝜌 𝑡 satisfies this PDE is equivalent to that ℎ 𝑡 = log 𝜌 𝑡 satisfies the following Hamilton-Jacobi PDE

By definition, we know ℎ 𝑡 = log 𝜋 𝑡 satisfies the above PDE, and we need to prove ℎ 𝑡 = log 𝜈 𝑡 = log 𝜋 𝑡 -1 2 𝑥 ⊤ 𝑄 𝑡 𝑥 + 𝑥 ⊤ 𝑏 𝑡 + 𝐹 ( 𝑡 ) satisfies this PDE as well. Here 𝐹 ( 𝑡 ) denotes the normalizing constant. Taking the difference between two equations, we need

which can be satisfied by the ODE dynamics (9).

B.2 Derivation of Solution to eq. (9)

Sanity Check for the Motivating Example. In the denoising setting, we have 𝑄 0 = 1 𝜎 2 𝐼 𝑑 , 𝑏 0 = 1 𝜎 2 𝑦 . The ODE (9) has the explicit solution

Note that this solution has a finite blow-up time when 1 + 𝜎 2 -𝑒 2 𝑡 → 0 + , which is exactly at 𝑇 ∗ = 1 2 log ( 1 + 𝜎 2 ) , as derived in the main text by matching the SNR. As 𝑡 → 𝑇 ∗ , 𝑄 𝑡 →∞ 𝐼 𝑑 ,

To see the limit of 𝑄 -1 𝑡 𝑏 𝑡 , we only need to consider the ODE for each component since 𝑄 is diagonal. So we view the above ODE as scalar ODEs. Considering the dynamics of

■

gives 𝑄 -1 𝑡 𝑏 𝑡 = 𝑒 -𝑡 𝑄 -1 0 𝑏 0. Therefore

which matches the initial condition derived in the main text for the denoising case. Furthermore, we can explicitly verify that the intermediate distribution of 𝑋 𝑡 by running the reverse SDE from 𝜈 ∗ 𝑇 is 𝜈 𝑡 :

To match the form of 𝜈 𝑡 , we have 𝑄 𝑡 = 1 𝑒 2 ( 𝑇 ∗ -𝑡 ) -1 = 𝑒 2 𝑡 1 + 𝜎 2 -𝑒 2 𝑡 , 𝑄 -1 𝑡 𝑏 𝑡 = 𝑒 -𝑡 𝑦 , which are the solutions of the ODE (9).

Solution to eq. (9). We recall that the observation operator 𝐴 ∈ ℝ 𝑑 ′ × 𝑑 has a general singular value decomposition form 𝐴 = 𝑈 Σ 𝑉 ⊤ with non-zero singular values 𝜆 1 ≥ 𝜆 2 ≥ · · · ≥ 𝜆 𝑑 ′ > 0. By definition, we have 𝑄 0 = 𝑉 diag ( 𝜆 2 1 / 𝜎 2 , · · · , 𝜆 2 𝑑 ′ / 𝜎 2 , 0 , · · · , 0 ) 𝑉 ⊤ . By left multiplying 𝑉 ⊤ and right multiplying 𝑉 to the first ODE in (9), we can diagonalize it to scalar equations / 𝑞 𝑡 = 2 ( 1 + 𝑞 𝑡 ) 𝑞 𝑡 for each diagonal entry. Solving this ODE gives

Here we explain how to solve 𝑏 𝑡 from eq. (9). We denote 𝑉 = [ 𝑣 1 , · · · , 𝑣 𝑑 ] , in which 𝑣 𝑖 are eigenvectors of 𝑄 (and 𝑄 𝑡 as well), and denote the eigenvalues of 𝑄 𝑡 (0 ≤ 𝑡 < 𝑇 ∗ ) by



By definition, we know ˜ 𝜆 𝑖 satisfies the ODE

We rewrite the solution 𝑏 𝑡 = ˝ 𝑑 𝑖 𝜉 𝑖 ( 𝑡 ) 𝑣 𝑖 and aim to solve 𝜉 𝑖 ( 𝑡 ) . From 𝑏 0 = 𝑉 ( 1 𝜎 2 Σ ⊤ 𝑈 ⊤ 𝑦 ) , we have the initial condition



Taking the inner product between 𝑣 𝑖 and both sides of the ODE / 𝑟 𝑡 = ( 𝐼 + 2 𝑄 𝑡 ) 𝑏 𝑡 , we have

Therefore, for 𝑑 ′ + 1 ≤ 𝑖 ≤ 𝑑 , 𝜉 𝑖 ( 𝑡 ) = 0. For 1 ≤ 𝑖 ≤ 𝑑 ′ , same to the derivation in eq. (32), we have

which gives

Proof of Corollary 6. Given Theorem 5, we only need to prove that sampling from

is equivalent to sampling from a new posterior. Taking 𝜋 𝑡 as the corresponding prior, we only need to show that the factor exp GLYPH<16> -1 2 𝑥 ⊤ 𝑄 𝑡 𝑥 + 𝑥 ⊤ 𝑏 𝑡 GLYPH<17> is the likelihood of certain observation model in the form of ˜ 𝑦 = 𝐴 𝑡 𝑥 + 𝑤 with 𝑤 ∼ 𝛾 𝑑 . To end, we need to ensure

Choosing 𝐴 𝑡 in the standard SVD form 𝐴 𝑡 = Σ 𝑡 𝑉 ⊤ where the singular values of 𝐴 𝑡 (the diagonal entries of Σ 𝑡 ) are 𝑒 𝑡 ( 1 + 𝜎 2 / 𝜆 2 𝑖 -𝑒 2 𝑡 ) 1 / 2 for 1 ≤ 𝑖 ≤ 𝑑 ′ , the quadratic term is matched. Matching the first order term requires 𝑏 𝑡 = 𝐴 ⊤ ˜ 𝑦 = 𝑉 Σ ⊤ 𝑡 ˜ 𝑦 . Further matching the coefficients in the basis of 𝑉 requires that

It is easy to verify that ˜ 𝑦 = Σ ′ 𝑡 𝑈 ⊤ 𝑦 with Σ ′ 𝑡 = diag GLYPH<18> 1 ( 𝜎 2 + 𝜆 2 1 ( 1 -𝑒 2 𝑡 ) ) 1 / 2 , . . . , 1 ( 𝜎 2 + 𝜆 2 𝑑 ′ ( 1 -𝑒 2 𝑡 ) ) 1 / 2 GLYPH<19> satisfies the above requirement.

Remark 11. Also, one can directly use the backward transport equation (3) to generate samples with the probability flow ODE [58] backward in time

from 𝑋 ← 𝑇 ∼ 𝛾 𝑑 . Combining the fact that 𝜈 𝑡 satisfies the PDE (5) and Δ 𝜈 𝑡 = ∇·(∇ 𝜈 𝑡 ) = ∇·( 𝑠 𝑡 -𝑄 𝑡 𝑥 + 𝑏 𝑡 ) , wehavethat 𝜈 𝑡 also satisfies the transport equation 𝜕 𝑡 𝑝 𝑡 = ∇·(( 𝑠 𝑡 +( 𝐼 + 𝑄 𝑡 ) 𝑥 -𝑏 𝑡 ) 𝑝 𝑡 ) . Therefore, for any 𝑡 < 𝑇 ∗ , by initializing 𝑋 𝑡 ∼ 𝜈 𝑡 and run the reverse ODE d 𝑋 ← 𝑡 = (-( 𝐼 + 𝑄 𝑡 ) 𝑋 ← 𝑡 -∇ log 𝜋 𝑡 ( 𝑋 ← 𝑡 ) + 𝑏 𝑡 ) d 𝑡 then 𝑋 ← 0 also gives the desired posterior. However, we note that unlike the reverse SDE case, the corresponding transport PDE and the vector field in the reverse ODE case are different from those used in the prior data generation. As discussed below, both 𝑄 𝑡 and 𝑏 𝑡 are singular near 𝑇 ∗ . Therefore running the reverse SDE might be preferrable for better numerical stability.

C Proofs of Section 5

C.1 Proof of Proposition 7

Proof of Proposition 7. By definition, we need to show

From the argument in the main text, we know 𝑇 ∗ = 1 2 log ( 1 + 𝜆 max ( 𝑄 ) -1 ) , and thus

From Corollary 13, we have

Let 𝑚 = 𝜆 min ( 𝑄 ) . Therefore, we can guarantee that 𝜈 𝑇 ∗ is strongly log-concave if

■

or equivalently

Lemma 12 (Hessian of Gaussian Mixture Potential) . Let 𝜋 = 𝜇 ★ 𝛾 Σ be a Gaussian mixture. Then ∇ 2 log 𝜋 ( 𝑥 ) = Σ -1 GLYPH<0> Cov GLYPH<2> T Σ -1 , Σ -1 𝑥 𝜇 GLYPH<3> Σ -1 -𝐼 GLYPH<1> .

Proof. Let us first compute the score ∇ log 𝜋 . By definition we have

where we defined Cov [ 𝜇 ] = 𝔼 𝑥 ∼ 𝜇 [ 𝑥𝑥 ⊤ ] - ( 𝔼 𝑥 ∼ 𝜇 𝑥 )( 𝔼 𝑥 ∼ 𝜇 𝑥 ) ⊤ .

Corollary 13. In particular, we have

Proof. From Lemma 12 and 𝜋 𝑡 = C 1 -𝑒 -2 𝑡 ( D 𝑒 𝑡 𝜋 ) = D 𝑒 𝑡 𝜋 ★ 𝛾 1 -𝑒 -2 𝑡 , we directly have

Now, using the commutation property between the isotropic tilt and the dilation D 𝛼 T 𝜂,𝜃 = T 𝛼 2 𝜂,𝛼𝜃 D 𝛼 , we have

■

and thus and therefore

Using that 𝑒 2 𝑇 ∗ -1 = ∥ 𝑄 ∥ -1 , we thus obtain

■

Lemma 14 (Isotropic Tilt of a Gaussian Mixture) . If 𝜋 = 𝜇 ★ 𝛾 𝛿 , then

Proof. By definition, we have

which gives the desired result after performing the affine change of variables from 𝑦 to ˜ 𝑦 .

■

where 𝜎 -2 = 𝛿 -1 + 𝑡 and

By expressing we have

Proof of Examples 8. The first example is immediate, after observing that T 𝑡 𝛾 is a Gaussian of variance ( 1 + 𝑡 ) -1 . For the Gaussian mixture example, we observe from Lemma 14 that T 𝑡 ( 𝜇★𝛾 𝛿 ) is a Gaussian mixture of variance ( 𝑡 + 𝛿 -1 ) -1 , where the mixture distribution is supported in a ball of radius 𝑅 𝛿 -1 𝛿 -1 + 𝑡 . Moreover, the covariance of a homogeneous mixture of the form 𝜇★𝛾 Σ is Σ + Cov ( 𝜇 ) .

If 𝜇 is a product measure, we observe that the isotropic tilt T 𝑡 𝜇 is also a product measure, and therefore its covariance is diagonal. Finally, by the previous argument, if 𝜋 is the uniform measure on the hypercube, then 𝜒 𝑡 ( 𝜋 ) = 𝜒 𝑡 ( 1 2 ( 𝛿 -1 + 𝛿 + 1 )) = 1. ■

Proof of Corollary 10. We plug the spread function 𝜒 𝑡 ( 𝜋 ) = 𝜒 𝑡 ( 𝜇 ★ 𝛾 𝛿 ) ≤ GLYPH<0> 𝑅 1 + 𝛿𝑡 GLYPH<1> 2 + 𝛿 1 + 𝛿𝑡 from Example 8 (ii) into eq. (12) to get (we use 𝜅 to denote 𝜅 ( 𝐴 ) for simplicity

■

C.2 Exponential Complexity of Importance Sampling in High SNR Regime

As shown in the main text, the proposed boosted posterior provably works for both low SNR and high SNR regimes. In this section, we formally argue that the importance sampling method is a nature baseline for posterior sampling with a large noise (low SNR regime) , but can suffer from exponential complexity when the SNR is high.

In order to estimate an integral of a function 𝑓 with respect to the posterior measure 𝜈 :

the idea of importance sampling is to independently sample 𝑋 1 , . . . , 𝑋 𝑛 from the prior 𝜋 and calculate

where 𝜏 ( 𝑥 ) is the observation likelihood exp (-1 2 𝑥 ⊤ 𝑄𝑥 + 𝑥 ⊤ 𝑟 ) . With the Denoising Oracle, we can sample from the prior efficiently. Intuitively, one can think if the posterior and prior are very similar, for example, when 𝜎 is large such that the ratio 𝜏 is close to 1, 𝐼 𝑛 ( 𝑓 ) computed from prior samples can efficiently approximate 𝐼 ( 𝑓 ) . On the contrary, if 𝜎 is small, 𝜏 ( 𝑥 ) can have very large variance and the importance sampling can be inefficient since many prior proposals have very small weights. The work [15, Theorem 1.2] proves that, in a fairly general setting, a sample of size approximately exp ( KL ( 𝜈 | | 𝜋 )) is necessary and sufficient for accurate estimation by importance sampling, where KL ( 𝜈 | | 𝜋 ) is the Kullback-Leibler divergence of 𝜋 from 𝜈 :

This result confirms one part of the intuition above: if 𝜎 is sufficiently small, then the magnitude of 𝑄 and 𝑟 will be sufficiently small, and so is ( 𝜈 | | 𝜋 ) and the number of samples needed in the importance sampling. Next we show that for a fairly generic prior distribution 𝜋 , when the SNR is large, KL ( 𝜈 | | 𝜋 ) will be also large such that we need approximately O( 𝑒 𝑑 · SNR ) examples to implement importance sampling, which is unachievable.

Without loss of generality, we assume the covariance of the prior 𝜋 is 𝐼 𝑑 .

Proposition 15 (Importance Sampling Sample Complexity Lower Bound) . Assume ∇ log 𝜋 ( 𝑥 ) is 𝐿 -Lipschitz:

Then, when SNR > 𝐿 + 2 , we have

and therefore the sample complexity of IS is exponential in dimension.

Proof. We wish to lower bound KL ( 𝜈 | | 𝜋 ) with tools of functional inequalities for the concentration of measure. By the celebrated work [51, 10] that the log-Sobolev inequality implies the Talagrand transport-entropy inequality, we have

Here 𝑊 ( 𝜈, 𝜋 ) denotes the Wassertain distance between 𝜈 and 𝜋 :

where Γ ( 𝜈, 𝜋 ) denotes the set of probability measures on ℝ 𝑑 × ℝ 𝑑 with marginals 𝜈 and 𝜋 .

First by Equation (63), we have which gives

by Bakry-Emery criterion [3]. Furthermore, we have the lower bound for the Wasserstain distance [28]

wherethesecond-to-last equality comes from the fact that the trace remains unchanged under orthogonal transformation. By (66) and Brascamp-Lieb Inequality (Theorem 16), we have Std ( 𝜈 ) 𝑖 ≤ √︃ 1 SNR -𝐿 . With the condition SNR -𝐿 > 2,

Collecting eqs. (65), (67) and (72), we obtain our final estimate

■

Theorem 16 (Brascamp-Lieb Inequality, [11]) . If 𝜋 is a strongly-log-concave measure on ℝ 𝑑 , i.e., of the form 𝜋 = 𝑒 -𝑓 with ∇ 2 𝑓 ( 𝑥 ) ⪰ 𝛼 Id for all 𝑥 ∈ ℝ 𝑑 , then ∥ Cov ( 𝜋 )∥ ≤ 𝛼 -1 .

C.3 Stability Analysis

In the numerical implementation of boosted posterior, we typically encounter certain errors. Especially, we may have imperfect score subject to certain 𝐿 2 errors, and we may not be able to sample the boosted posterior 𝜈 𝑡 exactly. Suppose that instead of starting from 𝜈 𝑡 at 𝑡 and run the exact reverse SDE (5), we start from an approximate distribution 𝑞 𝑡 ≈ 𝜈 𝑡 and run the reverse SDE (5) with approximating score 𝑠 𝜃 ( 𝑥, 𝑡 ) ≈ ∇ log 𝜋 𝑡 ( 𝑥 ) where 𝜃 denote the parameters parametrizing the score. Denote the distribution of the final samples by 𝑞 0, we have the following error estimate

Proposition 17. Suppose 𝜈 𝑡 , 𝑞 𝑡 , ∇ log 𝜋 𝑡 , 𝑠 𝜃 ( 𝑥, 𝑡 ) has enough regularities such that the reverse SDEs exist, if the Novikov's condition 𝔼 h exp ( ∫ 𝑡 0 ∥∇ log 𝜋 𝜏 ( 𝑥 ) -𝑠 𝜃 ( 𝑥, 𝜏 )∥ 2 d 𝜏 ) i < ∞ , then

The above proposition ensures that if both the initialization error and score error (over the posterior paths) are small, then the distribution of our final samples is close to the target posterior. The proof is provided in Appendix C.3. Note that we consider the reverse dynamics in continuous-time without time discretization error. There are various works [44, 45, 17, 8] analyzing the time discretization error and those techniques can be further incorporated into the above error estimate.

The proof is similar to that in [59]. Here we provide the proof in our posterior sampling context for completeness.

Proof. Consider the following two reverse dynamics needed for the error estimate: one is based on the exact score and starts from the exact boosted posterior

and another one is based on the approximate score and starts from the approximation to the boosted posterior √

Note that these two dynamics are defined backwardly for 𝜏 ∈ [ 0 , 𝑡 ] and we drop the superscript ← for notation simplicity. We denote the path measure of { 𝑋 𝜏 } 𝜏 ∈[ 0 ,𝑡 ] and { ˜ 𝑋 𝜏 } 𝜏 ∈[ 0 ,𝑡 ] by ν and q , respectively. Then 𝜈 and 𝑞 0 are the marginal distributions of the two path measures at 𝑡 = 0. By data processing inequality and chain rule of KL divergence, we have

Given the Novikov's condition, we can apply the Girsanov theorem [50] to eq. (75)76 to compute the second term above

■

D Iterated Tilted Transport

We have shown that posterior sampling of 𝜈 = T 𝑄 𝜋 can be reduced to sampling from 𝜈 𝑇 ∗ by running the reverse SDE. While 𝜈 𝑇 ∗ is easy to sample under the conditions presented in Section 5, these may

not be verified in several situations of interest. In this context, a natural question is whether one could still leverage the tilted transport, at the expense of introducing sampling error. This is what we address in this section.

Let 𝜆 1 , . . . 𝜆 𝑑 be the eigenvalues of 𝑄 . Let us assume for simplicity that 𝑏 = 0 and all eigenvalues have multiplicity 1, so 𝜆 𝑖 > 𝜆 𝑖 + 1. We define the events 𝑇 ∗ 𝑗 for 𝑗 = 1 . . . 𝑑 given by

where 𝜆 𝑗 ( 𝑡 ) = 𝑒 2 𝑡 /( 1 + 𝜆 -1 𝑗 -𝑒 2 𝑡 ) is the solution to the ODE / 𝑞 𝑡 = 2 ( 1 + 𝑞 𝑡 ) 𝑞 𝑡 . By abusing notation, we denote by ¯ 𝑄 𝑡 , 𝑡 ≥ 𝑇 ∗ the matrix that shares eigenvectors with 𝑄 , and with eigenvalues ( ¯ 𝜆 1 ( 𝑡 ) , . . . , ¯ 𝜆 𝑑 ( 𝑡 )) . Denote by 𝑉 𝑘 = [ 𝑣 𝑑 -𝑘 . . . 𝑣 𝑑 ] ∈ ℝ 𝑑 × 𝑘 the orthogonal projection onto the last 𝑘 eigenvectors.

While previously we considered only the transport between 𝜈 and 𝜈 1 : = 𝜈 𝑇 ∗ 1 , now we can consider the sequence 𝜈 𝑘 : = T ¯ 𝑄 𝑇 ∗ 𝑘 𝜋 𝑇 ∗ 𝑘 for 𝑘 = 1 , . . . , 𝑑 . Observe that 𝜈 𝑘 is a measure supported on a subspace Ω 𝑘 of dimension 𝑑 -𝑘 ; in other words, where 𝑘 directions are singular, corresponding to the eigenvectors associated with the ∞ eigenvalues of ¯ 𝑄 𝑇 ∗ 𝑘 , and thus Ω 𝑘 = { 𝑥 ∈ ℝ 𝑑 ; 𝑉 ⊤ 𝑘 𝑥 = y 𝑘 } for some y 𝑘 ∈ ℝ 𝑘 .

Now, let us consider 𝑘 ∗ = min { 𝑘 ; 𝜈 𝑘 is s.l.c. } ; that is, the first 𝑘 such that 𝜈 𝑘 is strongly logconcave, and therefore efficiently sampleable by Langevin dynamics. Under the same assumptions as Corollary 10, and by defining 𝜅 𝑘 : = 𝜆 𝑘 𝜆 𝑑 as the condition number of the truncated 𝑄 , we immediately obtain the bound

For 𝑘 < 𝑘 ∗ , assume first that one had sampling access to 𝜈 𝑘 + 1. Running the reverse tilted transport for time 𝜂 𝑘 = 𝑇 ∗ 𝑘 + 1 -𝑇 ∗ 𝑘 would produce samples from a tilted measure ˜ 𝜈 𝑘 : = T ˜ 𝑄 𝑘 𝜋 𝑇 ∗ 𝑘 , where ˜ 𝑄 𝑘 has eigenvalues ( 𝜓 ( 𝜂 𝑘 ) , . . . , 𝜓 ( 𝜂 𝑘 ) , 𝜆 𝑘 + 1 ( 𝑇 ∗ 𝑘 ) , . . . , 𝜆 𝑑 ( 𝑇 ∗ 𝑘 )) , where we defined 𝜓 ( 𝑡 ) : = ( 𝑒 -2 𝑡 -1 ) -1 as the inverse of 𝜆 ↦→ 1 2 log ( 1 + 𝜆 -1 ) . It is thus a non-singular measure in ℝ 𝑑 , capturing the fact that the denoising oracle driving the reverse dynamics is isotropic, and thus oblivious to the existence of the singular support of 𝜈 𝑘 .

We thus need a procedure to transform samples from ˜ 𝜈 𝑘 to samples of 𝜈 𝑘 . The easiest procedure is to simply marginalize the coordinates ( 𝑥 1 , . . . , 𝑥 𝑘 ) = 𝑉 ⊤ 𝑘 𝑥 ∈ ℝ 𝑘 , ie

and then 'lift' this measure in the subspace Ω 𝑘 , i.e.,

where we defined x 𝑘 = ( 𝑥 1 , . . . , 𝑥 𝑘 ) and x -𝑘 = ( 𝑥 𝑘 + 1 , . . . , 𝑥 𝑑 ) .

We can then iteratively run the tilted transport backwards, from 𝑘 = 𝑘 ∗ -1 to 𝑘 = 0, as illustrated in Algorithm 2:

Algorithm 2 Sampling Using Iterated Tilted Transport

1: Start by sampling 𝑋 𝑘 ∗ ∼ 𝜈 𝑘 ∗ .

2: for 𝑘 = 𝑘 ∗ -1 to 0 do

3: Run tilted transport starting at 𝑋 𝑘 + 1 for time 𝜂 𝑘 , resulting in ˜ 𝑋 .

4: Set 𝑋 𝑘 = ( y 𝑘 ; ˜ X -𝑘 ) .

5: end for

6: return 𝑋 0

Denote by

By the data-processing inequality, the TV error will accumulate linearly at each step. Denoting ˆ 𝜈 the law of 𝑋 0, we have

This bound can be interpreted as the accumulation of errors arising from conditioning a measure by marginalizing over its first components. To the extent that 𝜓 ( 𝜂 𝑘 ) is large, these variables are nearly deterministic, so one would expect that marginalization is a good approximation of conditioning. The outstanding question is to understand conditions when this error guarantee can be quantified.

Inspired by [18], a natural extension of this simple iterative procedure is to apply 'thermalization' towards the stationary measure 𝜈 𝑘 after line 4 of Algorithm 2 above, by running Langevin dynamics in Ω 𝑘 with score ∇ log 𝜈 𝑘 :

The drift of this diffusion is available, since both 𝑄 𝑇 ∗ 𝑘 and ∇ log 𝜋 𝑇 ∗ 𝑘 are known, so is ∇ log 𝜈 𝑘 .

Denote by ˇ 𝜈 𝑘 the law of 𝑋 𝑡 after time 𝑡 = 𝐵 𝑘 . While the time to relaxation of such Langevin dynamics is generally not quantitative (otherwise 𝑘 ∗ ≤ 𝑘 ), even a short amount of thermalization is able to improve upon the previous method. Indeed, by the reverse transport inequality [10, Lemma 4.2], a weaker Wasserstein guarantee 𝑊 2 ( ˆ 𝜈 𝑘 , 𝜈 𝑘 ) can be 'upgraded' to a TV guarantee of the form TV ( ˇ 𝜈 𝑘 , 𝜈 𝑘 ) = 𝑂 ( √ 𝐿 𝑘 𝑊 2 ( ˆ 𝜈 𝑘 , 𝜈 𝑘 )) by running Langevin dynamics for time 𝐵 𝑘 = Θ ( 1 / 𝐿 𝑘 ) , where 𝐿 𝑘 = sup 𝑥 𝜆 max (∇ 2 log 𝜈 𝑘 ( 𝑥 )) > 0 is the largest eigenvalue of ∇ 2 log 𝜈 𝑘 , which is positive by definition of 𝑘 ∗ and 𝑘 < 𝑘 ∗ . In summary, the 'thermalized' iterated tilted transport satisfies an error bound of the form

E Experimental Details

E.1 Gaussian mixture models

For a given dimension 𝑑 with 𝑑 mod2 = 0, we consider prior data a mixture of 25 Gaussian distributions, the same as considered in [13]. The Gaussian distribution has mean ( 8 𝑖, 8 𝑗 , · · · , 8 𝑖, 8 𝑗 ) ∈ ℝ 𝑑 for ( 𝑖, 𝑗 ) ∈ {-2 , -1 , 0 , 1 , 2 } 2 and unit variance. Each (unnormalized) mixture weight is independently drawn according to a 𝜒 2 distribution.

For the measurement model considered in Figure 3, we generate 𝐴 in the following way. We first sample a 𝑑 × 𝑑 matrix with each entry sampled from the standard normal and compute its SVD to get 𝑈 and 𝑉 for 𝐴 . The singular value is given by [ 1 , · · · , 1 / 20 ] where each component in between is independently sampled from Unif ([ 1 / 20 , 1 ]) such that the condition number of 𝐴 is 20. The observation noise is then determined by SNR. For the measurement model considered in Table 1, the matrix 𝑈 and 𝑉 for the SVD form of 𝐴 is the same to the above. Each singular value in 𝑆 is independently sampled from Unif ([ 0 , 1 ]) , and 𝜎 is sampled from Unif ([ 0 . 2 max 𝑆, max 𝑆 ]) .

For all the experiments we run the boosted posterior from 𝑇 ∗ -0 . 01 such that the ODE solution 𝑄, 𝑏 is well-defined. We use BlackJAX [12] to implement the No-U-turn sampler.

Besides results reported in Figure 3, we further test tilted transport when 𝑑 ′ < 𝑑 . In this setting, 𝜆 min ( 𝑄 𝑡 ) remains zero but the signal corresponding to the non-zero eigenvalues still gets enhanced. Therefore, although it becomes more difficult for 𝜈 𝑇 ∗ to be strongly log-concave, the tilted transport can still make the new posterior easier to sample even if it is not strongly log-concave yet. As shown in Table 1, when 𝑑 ′ = 0 . 9 𝑑 , 10% percent eigenvalues of 𝑄 𝑡 are zero, our tilted transport technique still reduces the statistical distance of the posterior samples significantly. We also consider an even more challenging case where 𝑑 ′ = 1 such that the target posterior is still heavlily multimodal (as visualized in the 2D example in Figure 2). In this case, LMC suffers from the local maxima of the potential and thus cannot explore the multimodal distribution efficiently. We use the No-U-turn sampler[34], a Hamilton Monte Carlo (HMC) method, as the baseline method, which can move among different modes more efficiently than Langevin. We find that the tilted transport technique can still boost the performance of HMC in this challenging setting. We also verify Theorem 5 by sampling from the boosted posterior directly from its analytical formula and running the reverse SDE, and the obtained samples approximate the target posterior well, as reported in Table 1.

E.2 Imaging Problems

We perform four inverse tasks on the Flickr-Faces-HQ Dataset (FFHQ) [39] to demonstrate the application of the tilted transport technique on imaging data as a proof of concept. To apply the proposed tilted transport technique to these problems, we still need to select a baseline method for sampling from the boosted posterior 𝜈 𝑇 ∗ . In the case of ill-conditioned problems, sampling 𝜈 𝑇 ∗ may still be challenging for principled algorithms like LMC, and we still need to rely on heuristic methods for imaging tasks. However, as noted in the introduction, most existing heuristic methods primarily facilitate conditional generation based on the measurement, lacking principled guarantees for posterior sampling. Consequently, we lack a principled interpretation for enhancing these methods with tilted transport. Nevertheless, we can still experiment with such methods as a proof of concept. We chose Diffusion Model Based Posterior Sampling (DMPS) [48] as the baseline method for the following reasons: The main assumption of DMPS in approximating the time-dependent conditional score is that the prior 𝜋 is uninformative (flat) with respect to 𝑋 𝑡 , such that 𝑝 ( 𝑋 0 | 𝑋 𝑡 ) ∝ 𝑝 ( 𝑋 𝑡 | 𝑋 0 ) . This assumption only holds approximately in early phases of the forward diffusion, and hopefully a higher SNR provided by tilted transport makes the effect of this approximation error smaller.

We conducted four tasks: (a) denoising; (b) inpainting with random masks from [53]; (c) 4 × super-resolution; and (d) deblurring using a Gaussian kernel. Our algorithm was implemented using the NVIDIA codebase [47] with 1000 diffusion steps for posterior sampling, and utilized the score function from a pretrained diffusion model [20]. Similar to our Gaussian mixture model experiments where we adjusted the timing for the boosted posterior to avoid the singularity of 𝑄 𝑡 , we shifted 6 10 timesteps for setting the boosted posterior. Experiments show that the final performance is robust with respect to the number of shifted steps. Figure 4 showcases examples from the inpainting task, demonstrating how tilted transport enhances the baseline DMPS method. Additionally, we report various sample statistics including peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Learned Perceptual Patch Similarity (LPIPS). However, it is important to note that while these statistics assess the quality of prior data generation, they may not accurately reflect the quality of posterior samples.

Original

Observed

DMPS

DMPSfromBoosted

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

(i) Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: This work focuses on provable sampling of posterior distribution with denoising oracles and the proposed tilted transport technique.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

(ii) Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See the last section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

(iii) Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All the details of proof are provided in Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

(iv) Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) Werecognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

(v) Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code for Gaussian mixture is uploaded in a single zip file. Our results on imaging tasks in Appendix E.2 are mainly for a proof-of-concept.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

(vi) Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: See the main text, Appendix E and code.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

(vii) Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In each experiment, we report percentiles or standard deviation.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

(viii) Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our experiments with the Gaussian mixture model require only a few seconds per run on a laptop.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

(ix) Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We conform the the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

(x) Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our paper mainly focuses provable sampling of the posterior distribution.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

(xi) Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The work poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

(xii) Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: We use open-source packages.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should state which version of the asset is used and, if possible, include a URL.

The authors should cite the original paper that produced the code package or dataset.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

(xiii) New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [No]

Justification: No new asset is released.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

(xiv) Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

(xv) Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies posterior sampling using unconditional diffusion models, or denoising oracles. The authors propose a \"tilted transport\" technique to combine a linear inverse problem and denoising oracles into a \"boosted\" posterior sampling problem. The authors study the feasibility of \"boosted\" posterior sampling with respect to properties (e.g. singular values) of $A$ (or $A^\\top A$), the measurement operator. They devise conditions on these properties for which \"boosted\" posterior sampling is tractable.\n\nSTRENGTHS:\n* The subject (posterior sampling from denoising oracles) is interesting and relevant to the ML community.\n\nWEAKNESSES:\n* The presentation of this manuscript is very poor. First, the formulation is very different from other diffusion (or generative) models papers, with unusual notations and concepts, some of which are undefined in the text (e.g. quadratic tilt or $\\ll$ at line 99). Second, it feels like the authors use this different formulation to purposely obfuscate their contribution. If I understand correctly, the \"tilted transport\" technique is an annealing of the likelihood $p(y | x_t)$ with respect to the added noise $\\sigma_t^2 = 1 - \\exp(-2t)$, such that it converges to $p(y | x)$ as $t$ goes to $0$. This is similar to previous works [1-4], but because the notations are so different, it is impossible for the reader to compare them. The authors should provide a thorough comparison of their method (preferably using standard formulation) with these previous works [1-4].\n\n* In the introduction (lines 40-46), the error of previous works using heuristics (including [1-4]) is not \"uncontrollable\" but not analyzed. In fact, Rozet et al. [3] propose a sampling scheme that is exact for enough LMC (Langevin Monte Carlo) corrections steps. I don't see how this \"contrasts with the principled sampling used in prior data generation\".\n\n* There is no algorithm to concisely describe the method. The provided code does not help as it is undocumented and frankly messy.\n\n* The tilted transport technique is not applicable to any distribution $\\pi$. In particular, sampling from $\\nu_{T^*}$ is only possible in contrived settings. For most distributions $\\pi$ and most measurement matrix $A$, sampling from $\\nu_{T^*}$ is as hard as sampling from $\\nu_0$.\n\n* The experiments are not relevant for most general linear inverse problems. The prior distributions are low-dimensional ($d \\leq 80$) mixture of few (25 components) Gaussians. The condition number of $A^\\top A$ is controlled ($\\kappa = 20$).\n\n* In appendix, the authors compare their method with Meng et al. [5] which is a quite poor (and not peer-reviewed) baseline. Better baselines [1-4] are not considered.\n\n[1] Song et al. \"Pseudoinverse-Guided Diffusion Models for Inverse Problems\". In International Conference on Learning Representations. 2023.\n\n[2] Finzi et al. \"User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical System\". In International Conference on Machine Learning. 2023.\n\n[3] Rozet et al. \"Score-based Data Assimilation\". In Advances in Neural Information Processing Systems. 2023.\n\n[4] Adam et al. \"Posterior samples of source galaxies in strong gravitational lenses with score-based priors\". 2022.\n\n[5] Meng et al. \"Diffusion model based posterior sampling for noisy linear inverse problems\". 2022.\n\nQUESTIONS:\n* To my understanding, simulating the reverse SDE Eq. (3) from $T$ to $0$ only generates samples from $\\pi_0$ if $\\pi_t$ is the measure defined by applying the Ornstein-Uhlenbeck process to $\\pi_0$, i.e. $\\pi_t = C_{\\beta_t} D_{\\alpha_t} \\pi_0$. Therefore, for $\\nu_t = T_{Q_t, b_t} \\pi_t$ to generate samples from $\\nu_0 = T_{Q, b} \\pi_0$ with the reverse SDE, it would be necessary that $\\nu_t = C_{\\beta_t} D_{\\alpha_t} \\nu_0$. Is this the case?\n\n* What is the relationship of $\\nu_t$ with previous heuristics [1-4] ?\n\n### Typos\n\n* Line 110, $w \\sim \\gamma_{d'}$\n* Line 122, $Q = \\sigma^{-2} I_d$\n* Line 222, $p(\\tilde{y} | x) = p(X_{T^*} | X_0)$\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a theory regarding posterior sampling in linear inverse problems with additive Gaussian noise. It provides two major contributions:\n\n* A proof that no general algorithm exists for posterior sampling of Ising models when the degradation operator is ill-conditioned.\n* A transport equation for the coefficients of the quadratic form of the posterior distribution in the linear setup, transforming it into a new distribution that is easier to sample from. This includes conditions on SNR and condition number for strong log-concavity of the transformed probability, allowing for efficient sampling.\n\nThe authors illustrate their theoretical results with a toy example with Gaussian mixture model.\n\nSTRENGTHS:\n* The paper effectively presents the problem and necessary background, along with detailed and rigorous derivations.\n* In the context of the linear setup, the paper presents impactful and theoretically sound results regarding posterior sampling.\n\nWEAKNESSES:\n* Figure 1 and its description are not clear. I would expect it to clearly summarize the essence of the work, particularly highlighting the second contribution.\n\n* While the paper contains rigorous derivations, it lacks intuitive explanations and simplified examples, making it difficult for readers to understand the overall direction and implications of the work as they read.\nThis lack of accessibility limits the paper's potential audience.  \n\n* The focus on linear problems with additive Gaussian noise limits the practicality and potential impact of the work.\n\n* The lack of a practical algorithm for implementing the tilted transport map limits the applicability of this work.\n\n* The authors' decision to place the extension to nonlinear setups and imaging experiments to the appendix is puzzling, as these results are compelling and could strengthen the paper's main arguments if presented in the main body.\n\nQUESTIONS:\nPlease address weaknesses.\n\nLIMITATIONS:\nThe limitations of the work, e.g. the focus on linear setups, are only briefly mentioned in the discussion section. This warrants a more extensive and dedicated discussion to adequately address their implications.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
This paper proposes a novel posterior sampling algorithm specialized to linear inverse problems. The proposed algorithm relies on a sequence of intermediate and **tractable** posterior distributions $\nu_t$ that satisfy a transport equation up to some time $T^*$. The algorithm then operates as follows; first draw approximate samples from the distribution $\vu_{T^*}$ using any approximate inference algorithm, and then run the reverse SDE from $T^*$ to $0$ starting at the obtained samples. Of course, the computational hardness is transferred to $\nu_{T^*}$ and the authors derive sufficient conditions that ensure it is strongly log concave. The study of the algorithm is then specialized to both the Ising model and Gaussian mixtures. They show for example that $\nu_{T^*}$ is strongly log-concave for the Ising model under the condition that the eigenvalues difference is smaller than 1, thus providing a new efficient continuous time procedure for sampling from it.

STRENGTHS:
Overall, this paper is highly original and I greatly appreciated reading it. It has quite a few contributions and the proposed main algorithm is interesting. The writing is clean so that the paper is relatively easy to understand, while still sufficiently detailed. The properties of the algorithm are well investigated throughout the Ising model and Gaussian mixture examples.

WEAKNESSES:
There isn’t any serious weakness; perhaps I would have liked to see more challenging experiments and comparisons with denoising diffusion based posterior sampling algorithms, but in my opinion this is optional for this paper, since it is more on the theoretical side and has strong contributions.

QUESTIONS:
N/A

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-12

comments: SUMMARY:
The paper "Provable Posterior Sampling with Denoising Oracles through Tilted Transport" analyses the possibility of posterior sampling utilizing denoising oracles, such as score-based diffusion models. To demonstrate the possibilities the authors focus on linear inverse problems and propose the tilted transport technique. It uses the quadratic log-likelihood structure of linear inverse problems together with the oracle to transform the original posterior into one easier to sample. Based on the example of the Ising model the authors show that well-conditioned measurements allow the transport to simplify the task. Furthermore, first empirical tests for image tasks indicate promising practical use.

STRENGTHS:
Well written and well structured with proofs. Potential for impact is high

WEAKNESSES:
The topic is far from my expertise. I cannot see obvious weaknesses for this analysis. It has its limitations, but it is clearly stated and argued by the authors.

QUESTIONS:
What is the reason for the peak with large percentiles for 10^-3 SNR in Figure 3?

LIMITATIONS:
yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-14