Title: Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion
Venue: NeurIPS 2024
Accepted: yes
Keywords: Non-logconcave sampling

Monte Carlo sampling

Denoising Diffusion Model
Abstract: This paper considers the problem of sampling from non-logconcave distribution, based on queries of its unnormalized density. It first describes a framework, Denoising Diffusion Monte Carlo (DDMC), based on the simulation of a denoising diffusion process with its score function approximated by a generic Monte Carlo estimator. DDMC is an oracle-based meta-algorithm, where its oracle is the assumed access to samples that generate a Monte Carlo score estimator. Then we provide an implementation of this oracle, based on rejection sampling, and this turns DDMC into a true algorithm, termed Zeroth-Order Diffusion Monte Carlo (ZOD-MC). We provide convergence analyses by first constructing a general framework, i.e. a performance guarantee for DDMC, without assuming the target distribution to be log-concave or satisfying any isoperimetric inequality. Then we prove that ZOD-MC admits an inverse polynomial dependence on the desired sampling accuracy, albeit still suffering from the curse of dimensionality. Consequently, for low dimensional distributions, ZOD-MC is a very efficient sampler, with performance exceeding latest samplers, including also-denoising-diffusion-based RDMC and RSDMC. Last, we experimentally demonstrate the insensitivity of ZOD-MC to increasingly higher barriers between modes or discontinuity in non-convex potential.

Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion

Abstract

This paper considers the problem of sampling from non-logconcave distribution, based on queries of its unnormalized density. It first describes a framework, Denoising Diffusion Monte Carlo (DDMC), based on the simulation of a denoising diffusion process with its score function approximated by a generic Monte Carlo estimator. DDMC is an oracle-based meta-algorithm, where its oracle is the assumed access to samples that generate a Monte Carlo score estimator. Then we provide an implementation of this oracle, based on rejection sampling, and this turns DDMC into a true algorithm, termed Zeroth-Order Diffusion Monte Carlo (ZODMC). We provide convergence analyses by first constructing a general framework, i.e. a performance guarantee for DDMC, with out assuming the target distribution to be log-concave or satisfying any isoperimetric inequality. Then we prove that ZOD-MC admits an inverse polynomial dependence on the desired sampling accuracy, albeit still suffering from the curse of dimensionality. Consequently, for low dimensional distributions, ZOD-MC is a very efficient sampler, with performance exceeding latest samplers, including also-denoising-diffusion-based RDMC and RSDMC. Last, we experimentally demonstrate the insensitivity of ZOD-MC to increasingly higher barriers between modes or discontinuity in nonconvex potential.

1 Introduction

The problem of drawing samples from a distribution based on unnormalized density ∝ exp( -V ) (described by the potential V ) is a fundamental statistical and algorithmic problem. This classical problem nevertheless remains as a research frontier, providing pivotal tools to applications such as decision making, statistical inference / estimation, uncertainty quantification, data assimilation, and molecular dynamics. Worth mentioning is that machine learning could benefit vastly from progress in sampling as well, not only because of its connection to inference, optimization and approximation, but also through modern domains such as diffusion generative modeling & differential privacy.

Recent years have seen rapid developments of sampling algorithms with quantitative and nonasymptotic theoretical guarantees. Many of the results are either based on discretizations of diffusion processes [12, 13, 50, 16, 34, 33] or gradient flows [38, 10, 22]. In order to develop such guarantees, it is necessary to make assumptions about the target distributions, for instance, that it satisfies an isoperimetric property, where standard requirements are log-concavity or functional inequalities

[12, 56, 21, 8, 48]. However, there is empirical evidence that the corresponding algorithms struggle to sample from targets that have high barriers between modes that create metastability. Overcoming such issues is highly nontrivial and researchers have continued to develop new methods to tackle these problems.

Diffusion models have lately shown remarkable ability in the generative modeling setting, with applications including image, video, audio, and macromolecule generations. This created a wave of theoretical work that showed the ability of diffusion models to sample from distributions under minimal assumptions [14, 57, 5, 32, 29, 4, 11, 3]. However, these works all started with the assumption that there is access to an approximation of the score function with some accuracy. This is a reasonable assumption for the task of generative modeling when one spends enough efforts on the training of the score, but the task of sampling is different. A natural question is: can we leverage the insensitivity of diffusion models to multimodality to efficiently sample from unnormalized, non-log-concave density? This would require approximating the score, which is then used as an inner loop inside an outer loop that integrates reverse diffusion process to transport, e.g., Gaussian initial condition, to nearly the target distribution.

The seminal works by [26, 27, 20] try to answer this question using Monte Carlo estimators of the score function and to provide theoretical guarantees. We also mention earlier work by [53] which learns parameterized scores and more work along the same line by [58, 44, 54, 55], whose theoretical guarantees are less clear but are based other interesting ideas. [26] proposed Reverse Diffusion Monte Carlo (RDMC), which estimates the score via LMC algorithm and relaxes the isoperimetric assumptions in the analysis of traditional sampling algorithms. [20] proposed a similar method, stochastic localization via iterative posterior sampling (SLIPS), which approximate the score via Metropolis-adjusted Langevin algorithm (MALA). However, both methods rely on the usage of a small time window where isoperimetric properties hold. This leaves the problem of finding a good initialization for the diffusion process. To alleviate this issue, [27] developed an acceleration of RDMC, the Recursive Score Diffusion-based Monte Carlo (RSDMC), which improves the nonasymptotic complexity to be quasi-polynomial in both dimension and inverse accuracy and gets rid of any isoperimetric assumption. Such work provides strong theoretical guarantees, however it requires a lot of computational power to get a high accuracy sampler. Additionally, RDMC, SLIPS and RSDMC are all based on first-order queries (i.e. gradients of V ), which brings extra computational and memory costs, in addition to requiring a continuous differentiable V . Motivated by these two observations, we create a sampler that only makes use of zeroth-order queries without assuming any isoperimetric conditions on the target distribution. Our contributions can be summarized as follows.

We introduce an oracle-based meta-algorithm DDMC (Denoising Diffusion Monte Carlo) and provide a non-asymptotic guarantee in KL-divergence in Theorem 1. Our result provides theoretical insight on the choice of optimal step-size in DDMC as well as in denoising diffusion models (Sec. 3.2).

We develop a novel algorithm ZOD-MC (Zeroth Order Diffusion-Monte Carlo) that uses zeroth-order queries and the global minimal value of the potential function to generate samples approximating the target distribution. In Corollary 3.1, we establish a zeroth-order query complexity upper bound for general target distributions satisfying mild smoothness and moment conditions. Our result is summarized and compared to other sampling algorithms in Table 5.

The advantages of our algorithm are experimentally verified for non-log-concave target distributions. We demonstrate the insensitivity of our algorithm to various high barriers between modes, and the ability of correctly account for discontinuities in the potential.

2 Preliminaries

2.1 Diffusion Model

Diffusion model generates samples that are similar to training data, by requiring the generated data to follow the same latent distribution p as the training data. To do so, it considers a forward noising

O

O

process that transforms a random variable into Gaussian noise. One most commonly used forward process is (a time reparameterization of) the Ornstein-Uhlenbeck (OU) process, given by the SDE:

where { B t } t ≥ 0 is the standard Brownian motion in R d . The OU process that solves (1) is in distribution equivalent to a sum of two independent random vectors: X t = e -t X 0 + √ 1 -e -2 t Z where ( X 0 , Z ) ∼ p ⊗ γ d and γ d is the standard Gaussian distribution in R d . Denote p t = Law ( X t ) for all t ≥ 0 . If we consider a large, fixed terminal time T of (1), then p T is close to γ d . Then, the denoising or backwards diffusion process, { ¯ X t } 0 ≤ 0 ≤ T , can be constructed by reversing the OU process from time T , meaning that Law ( ¯ X t ) := Law ( X T -t ) for all t ∈ [0 , T ] . By doing so we obtain the denoising diffusion process which solves the following SDE:

where { ¯ B t } 0 ≤ t ≤ T is a Brownian motion in R d , independent of { B t } 0 ≤ t ≤ T and ∇ ln p t is usually referred as the score function for p t . Although the denoising process initializes at p T , we can't generate exact samples from p T . In practice, people consider the standard Gaussian initialization γ d due to the fact that p T is close to γ d when T is large. The denosing process with the standard Gaussian initialization is given by

By simulating this denoising process (3), we can achieve the goal of generating new samples. However, the denoising process (3) can't be simulated directly due to the fact that the score function is not explicitly known. A widely applied method to solve this issue is to learn the score function through denoising score matching [51, 24, 52]. Given a learned score, denoted as s ( t, x ) , one can simulate the denoising diffusion process using discretizations like the Euler Maruyama or some exponential integrator. From a theoretical perspective, assuming the learned score satisfies

non-asymptotic convergence guarantees for diffusion models are obtained in [5, 3, 4, 11]. For instance, in [3], polynomial iteration complexities were proved without assuming any isoperimetric property of the data distribution and only assuming the data distribution has a finite second moment and a score estimator satisfying (4) is available.

In this work, we consider instead the sampling setting, in which no existing samples from the target distribution is available. Our sampling algorithm and theoretical analysis are motivated from the denoising diffusion process given by (2) and its corresponding discretization through the exponential integrator in Algorithm 1. In particular, we first introduce an oracle-based meta-algorithm, DDMC, which integrates Algorithm 1 and Algorithm 2, where the exponential integrator scheme of (2) is applied to generate samples and the score function is approximated by a Monte Carlo estimator assuming independent samples from a conditional distribution are available.

2.2 Rejection Sampling and Restricted Gaussian Oracle

Rejection sampling is a popular Monte Carlo method for sampling a target distribution, p , based on the zeroth-order queries of the potential V . It requires that we have access to the potential function

Algorithm 1:

V µ of some other distribution µ , such that µ is easy to sample from and exp( -V ) ≤ exp( -V µ ) globally. Such a distribution µ is typically called an envelope for the distribution p . With an envelope ν , rejection sampling generates samples from p by running the following algorithm till acceptance:

Sample X ∼ µ ,

The rejection sampling is considered as a high-accuracy algorithm as it outputs a unbiased sample from the target distribution. However, despite such a remarkable property, it has drawbacks. First, it is a nontrivial task to find an envelope for a general target distribution. Second, rejection sampling usually suffers from 'curse of dimensionality'. Even for strongly logconcave target distributions, the complexity of the rejection sampling increases exponentially fast with the dimension: in expectation it requires κ d/ 2 many rejections before one acceptance, where κ is the condition number for the potential V , see [9].

Accept X with probability exp( -V ( X ) + V µ ( X )) .

The Restricted Gaussian Oracle (RGO), which was first introduced in [31], assumes that an accurate sample from distribution π ( ·| y ) ∝ exp ( -V ( · ) -1 2 η ∥· -y ∥ 2 ) can be generated for any y ∈ R d , η > 0 and any potential V . Implementing the RGO is challenging. It is usually done by rejection sampling. However, most proposed methods [36, 18], are only suitable for small η .

Our proposed sampling algorithm, ZOD-MC applies the rejection sampling (Algorithm 3) to implement the RGO with a large value of η . Details on ZOD-MC are introduced in Section 3.1.

3 Denoising Diffusion Monte Carlo Sampling

In this section, we first introduce DDMC and ZOD-MC in Section 3.1. Then we provide a convergence guarantee for DDMC in Section 3.2. Last, in Section 3.3, we establish the zeroth-order query complexity of ZOD-MC. Note DDMC is a meta-algorithm that still requires an implementation of its oracle, and ZOD-MC is an actual algorithm that contains such an implementation. The theoretical guarantee of ZOD-MC (Sec. 3.3), therefore, is based on the analysis framework of DDMC (Sec. 3.2).

3.1 Denoising Diffusion Monte Carlo and Zeroth-Order Diffusion Monte Carlo

Denoising Diffusion Monte Carlo (DDMC). Let's start with a known but helpful lemma on score representation, derivable from Tweedie's formula [45].

Lemma 1. Let { X t } t ≥ 0 be the solution to the OU process (1) and p t = Law ( X t ) . Then for all t > 0 ,

where p 0 | t ( ·| x ) ∝ exp ( -V ( · ) -1 2 ∥ ·-e t x ∥ 2 e 2 t -1 ) is the distribution of X 0 conditioned on { X t = x } .

This lemma was for example applied in [26] to do sampling based on the denoising diffusion process in (2). For the sake of completeness, we include its proof in Appendix C.5.

Due to (5), to approximate the score function ∇ ln p t ( x ) , it suffices to generate samples that approximate p 0 | t ( ·| x ) . [26, 27] proposed to use Langevin-based algorithms to sample from p 0 | t ( ·| x ) . The first step of our work is to generalize this, with refined and more general theoretical analysis later on, by considering an oracle algorithm, DDMC, which assumes independent samples { z t,i } n ( t ) i =1 that

Algorithm 2: Monte Carlo Score Estimation

Input : t ∈ (0 , T ] , x ∈ R d , n ( t ) ∈ Z + , δ ( t ) > 0 . Output: s ( t, x ) . Oracle : generate independent { z t,i } n ( t ) i =1 such that W 2 (Law( z t,i ) , p 0 | t ( ·| x )) ≤ δ ( t ) . s ( t, x ) ← 1 n ( t ) ∑ n ( t ) i =1 x - e - t z t,i 1 - e - 2 t .

approximate p 0 | t ( ·| x ) are available. The Monte Carlo score estimator in Algorithm 2 is given by

where n ( t ) is the number of samples and δ ( t ) is such that W 2 (Law( z t,i ) , p 0 | t ( ·| x )) ≤ δ ( t ) for all i . In Section 3.2, we will discuss how the performance of sampling depends on n ( t ) , δ ( t ) .

Zeroth-Order Diffusion Monte Carlo (ZOD-MC). Noticing that in Lemma 1, the conditional distribution has a structured potential function: a summation of the target potential and a quadratic function. Therefore, implementing the oracle in DDMC is equivalent to implementing RGO with y = e t x and η = e 2 t -1 . Based on this, we propose ZOD-MC, a novel methodology based on rejection sampling and DDMC. Rejection samplng (Algorithm 3) can generate i.i.d. Monte Carlo samples required in Algorithm 2. Therefore, ZOD-MC, as a combination of rejection sampling and DDMC, can efficiently sample from non-logconcave distributions. See Appendix B for more details.

Algorithm 3: Rejection Sampling: generating z n ( t )

{ t,i } i =1 in Algorithm 2 Input : x ∈ R d , zeroth-order queries of V . Output: z . while TRUE do Generate ( ξ, u ) ∼ γ d ⊗ U [0 , 1] ; z ← e t x + √ e 2 t -1 ξ ; return z if u ≤ exp( -V ( z ) + V ∗ ) ; end

Remark 1. (Remark on the optimization step) In theory, we assume an oracle access to the minimum value of V . However, in practice we use Newton's method to find a local minimum. Throughout the sampling process we update the local minimum as we explore the search space.

Remark 2. (Parallelization) Notice that Algorithm 3 can be run in parallel to generate all the n ( t ) samples required to compute the score. Contrary to methods like LMC that have a sequential nature, this allows our method to be more computationally efficient and reduce the running time. This is a feature that RDMC or RSDMC doesn't benefit as much from.

3.2 Convergence of DDMC

Our oracle-based meta-algorithm, DDMC, provides a framework for designing and analyzing sampling algorithms that integrate the denoising diffusion model and the Monte Carlo score estimation. In this section, we first present an error analysis to the Monte Carlo score estimation in Proposition 3.1, whose proof is in Appendix C.3. After that, we leverage our result in Proposition 3.1 and provide a non-asymptotic convergence result for DDMC in Theorem 1, whose proof is in Appendix C.4.

Proposition 3.1. Let { X t } t ≥ 0 be the solution of the OU process (1) and p t = Law( X t ) for all t > 0 . If we define s ( t, x ) = 1 n ( t ) ∑ n ( t ) i =1 e -t z t,i -x 1 -e -2 t with { z t,i } n ( t ) i =1 being a sequence of independent random vectors such that W 2 ( Law ( z t, 1 ) , p 0 | t ( ·| x )) ≤ δ ( t ) for all t > 0 and x ∈ R d , then we have

Choice of δ ( t ) and n ( t ) . The error bound in (7) helps choose the accuracy threshold δ ( t ) and the number of samples n ( t ) to control the score estimation error over different time. In fact, when

t increases, it requires less samples and allows larger sample errors to get a good Monte Carlo score estimator. If we assume Cov p ( x ) = O ( d ) for simplicity, then when t is small, the factor e -2 t (1 -e -2 t ) 2 = O ( t -2 ) and the choice of δ ( t ) = O ( tε ) and n ( t ) = Ω( dt -2 ε -2 ) will lead to the L 2 -error of order O ( ε 2 ) . When t is large, the factor e -2 t (1 -e -2 t ) 2 = O ( e -2 t ) and it only requires δ ( t ) = O ( e t ε ) and n ( t ) = Ω( de -2 t ε -2 ) to ensure the L 2 -error is of order O ( ε 2 ) . In the latter case, the δ ( t ) is of a larger order and n ( t ) is of a smaller order than the first case.

We now analyze the convergence of DDMC. Recall that Algorithm 1 is an exponential integrator discretization scheme of (2) with the time schedule 0 = t 0 < t 1 < · · · < t N = T -δ for some δ > 0 . In each iteration, x k +1 = e t k +1 -t k x k +2( e t k +1 -t k -1) s ( T -t k , x k ) + √ e 2( t k +1 -t k ) -1 ξ k , where ξ k ∼ γ d and s ( T -t k , · ) is the Monte Carlo score estimator generated by Algorithm 2. The trajectory of Algorithm 1 can be piece-wisely characterized by the following SDEs: for all t ∈ [ t k , t k +1 ) ,

Therefore, the convergence of DDMC is equivalent to the convergence of the process { ˜ X t } 0 ≤ t ≤ t N , which could be quantified under mild assumptions on the target distribution. Next, we present the moment assumption on the target distribution and our non-asymptotic convergence theorem.

Assumption 3.1. The distribution p has a finite second moment: E x ∼ p [ ∥ x ∥ 2 ] = m 2 2 < ∞ .

Theorem 1. Assume that the target distribution satisfies Assumption 3.1. Let { X t } t ≥ 0 be the solution of (1) with p t := Law( X t ) and { ˜ X t } t ≥ 0 be the solution of (8) with q t := Law( ˜ X t ) . For any δ ∈ (0 , 1) and T > 1 , let 0 = t 0 < t 1 < · · · < t N = T -δ be a time schedule such that γ k = Θ( γ k -1 ) for all k = 0 , 1 , · · · , N -1 , where γ k := t k +1 -t k . Then where δ ( t ) , n ( t ) are parameters in Algorithm 2.

Remark 3. In Theorem 1, we characterize KL ( p δ | q t N ) instead of KL ( p | q t N ) due to the fact that KL ( p | q t N ) is not well-defined when the target distribution p is not smooth w.r.t. the Lebesgue measure. It turns out that p δ is an alternative distribution to look at because p δ is smooth for all δ > 0 and p δ is close to p when δ is small (see Proposition C.1). This is a standard treatment, referred to as early stopping, in the score-based generative modeling literature [e.g., 5, 3].

The terms I, II, III in (9) correspond to the three types of errors in Algorithm 1, the initialization error, the score estimation error and the discretization error, respectively. Such a decomposition is very common in analyses of diffusion models, see [5, 6, 3]. Since we consider the sampling setting, the score estimation error is derived from the error analysis to the Monte Carlo score estimator, i.e., Proposition 3.1. A detailed discussion on these three types of errors is provided in Appendix C.4.

Compared to existing analyses on diffusion models, Theorem 1 extends result in [3] from exponentialdecay step-size to general choices of step-size, and recovers their sharp linear dimension dependence in the discretization error, with minimal assumptions on the target distribution. By assuming two consecutive step sizes are of the same order, we perform asymptotic estimation on the accumulated discretization errors and obtain a bound that depends on the step-size. Such kind of result helps to understand the optimal time schedule, as we will discuss soon.

Discussion on the choices of time schedule. Under different choices of step-size, the discretization errors in the denoising diffusion model have the same linear dependence on d , but different dependence on δ . The linear dimension dependence improves the results in [4], where O ( d 2 ) discretization error bounds are proved for different choices of step-size. It is also a extension of the result in [3], where O ( d ) discretization error is only proved for the exponential-decay step-size. In fact, Theorem 1 implies that the exponential-decay step-size induces an optimal discretization error up to some constant in term of the inverse early-stopping time δ -1 . Detailed discussions on different choices of time schedules is provided in Appendix C.5.

3.3 Complexity of ZOD-MC

With the convergence result for DDMC in Theorem 1, we introduce the query complexity bound of ZOD-MC. Our analysis assumes a relaxation of the commonly used gradient-Lipschitz condition on the potential. The formal statement is presented in Corollary 3.1, whose proof is provided in Appendix C.6.

Assumption 3.2. There exists a constant L > 0 such that for any x ∗ ∈ arg min y ∈ R d V ( y ) and x ∈ R d , V satisfies V ( x ) -V ( x ∗ ) ≤ L 2 ∥ x -x ∗ ∥ 2 .

2 ε KL , ε 2 ε -1

Corollary 3.1. Under the assumptions in Theorem 1 and Assumption 3.2, if we set T = 1 ln( d +m 2 2 )

γ k = κ min(1 , T -t k ) , δ = min( W 2 d , W 2 m 2 ) , κ = Θ ( T +ln( δ ) N ) , then to obtain an output (with distribution q t N ) in ZOD-MC such that W 2 ( p, p δ ) ≲ ε W 2 and KL ( p δ , q t N ) ≲ ε KL, the zeroth-order query complexity is of order

where the ˜ O hides polylog ( d +m 2 2 ε W 2 ) factors.

Remark 4. If we assume WLOG that the minimizer of the potential is at the origin, i.e., x ∗ = 0 , and further make reasonable assumptions that m 2 2 , L and {∥ x k ∥ 2 } are all of order O ( d ) , where { x k } are the iterates in Algorithm 1, then the query complexity of ZOD-MC is of order exp ( ˜ O ( d ) log( ε -1 KL ) ) . Even though this complexity bound has an exponential dimension dependence, it only depends polynomially on the inverse accuracy. Since it applies to any target distribution satisfying Assumptions 3.1 and 3.2, this complexity bound suggests that with the same overall complexity, ZOD-MC can generate samples more accurate than other algorithms in Table 5, for a large class of low-dimensional non-logconcave target distributions.

Comparison to LMC, RDMC and RSDMC. When no isoperimetric condition is assumed, we compare convergence for ZOD-MC to convergence for LMC, RDMC and RSDMC.

In the absence of the isoperimetric condition, [1] demonstrated that LMC is capable of producing samples that are close to the target in FI assuming the target potential is smooth. However, FI is a weaker divergence than KL divergence/ Wasserstein-2 distance. It has been observed that, in certain instances, the KL divergence/Wasserstein-2 distance may still be significantly different from zero, despite a minimal FI value. This observation implies that the convergence criteria based on FI may not be as stringent as our result which is based on KL divergence/Wasserstein-2 distance. [26] proved that RDMC produces samples that are ε -close to the target in KL divergence with high probability. Assuming the potential is smooth and a tail-growth condition, the first order oracle complexity is shown to be of order exp( ε -1 log d ) . [27] introduced RSDMC as an acceleration of RDMC. They were able to show that if the potential is smooth, RSDMC produces a sample that is ε -close to the target in KL divergence with high probability. The first order oracle complexity is shown to be of order exp(log 3 ( d/ε )) . Compared to RDMC and RSDMC, our result on ZOD-MC doesn't require the potential to be smooth as our Assumption 3.2 is only a growth condition of the potential. This indicates that our convergence result applies to targets with non-smooth, or even discontinuous potentials. Our result in Corollary 3.1 shows the zeroth-order oracle complexity for ZOD-MC is of order exp( d log( ε -1 )) , which achieves a better ε -dependence compared to RDMC and RSDMC, at the price of a worse dimension dependence. This suggests that, for any low-dimensional target, ZOD-MC produces a more accurate sample than RDMC/RSDMC when the overall oracle complexity are the same. Last, zeroth-order queries cost less computationally than first-order queries in practice, which also makes ZOD-MC a more suitable sampling algorithm when the gradients of the potential are hard to compute.

4 Experiments

We will demonstrate ZOD-MC on three examples, namely Gaussian mixtures, Gaussian mixtures plus discontinuities, and Müller-Brown which is a highly-nonlinear, nonconvex test problem popular in computational chemistry and material sciences. Multiple Gaussian mixtures will be considered, for showcasing the robustness of our method under worsening isoperimetric properties. The baselines we consider include RDMC [26], RSDMC [27], SLIPS [20], the proximal sampler [37], annealed

importance sample [42], sequential Monte Carlo [15], a parallel tempering approach with MALA proposals [30] and naive unadjusted Langevin Monte Carlo. All the experiments are conducted using a NVIDIA GeForce RTX 4070 Laptop GPU with 8 GB of VRAM and Pytorch.

4.1 Results for Gaussian Mixtures

Matched Oracle Complexity. We modify a 2 D Gaussian mixture example frequently considered in the literature to make it more challenging, by making its modes unbalanced with non-isotropic variances, resulting in a highly asymmetrical, multi-modal problem. We include the full details of the parameters in Appendix D. We fix the same oracle complexity (total number of 0 th and 1 st order V queries) for different methods, and show the generated samples in Figure 2. Note matching oracle complexity puts our method at a disadvantage, since other techniques require querying the gradient, which results in more function evaluations. Despite this, we see in Figure 1a that our method achieves both the lowest MMD and W 2 using the least number of oracle complexity.

0

2000

4000

6000

8000

10000

Oracle Complexity

0

1

2

3

4

5

MMD

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

0

2000

4000

6000

8000

10000

Oracle Complexity

4

6

8

10

12

14

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

1

2

3

4

5

6

7

Dimension

0

100

200

300

400

500

600

700

800

Error in estimation of

E

[

f

(

x

)]

ZOD-MC

RDMC

RSDMC

SLIPS

1

2

3

4

5

6

7

Dimension

0

5

10

15

20

25

ZOD-MC

RDMC

RSDMC

SLIPS

(a) Sampling accuracy against oracle complexity. For anyfixed oracle complexity, ZOD-MC has the least error both in MMD and W 2 . Note diffusion based methods do not have an initialization. Thus curves don't start at the same y -value.

Robustness Against Mode Separation. Now let's further separate the modes in the mixture to investigate the robustness of our method to increasing nonconvexity/metastability. More precisely, we scale the means of each mode by a constant factor to have a mode located at (0 , R ) ; doing so increases the barriers between the modes and exponentially worsens the isoperimetric properties of the target distribution [49]. Figure 4a shows our method is the most insensitive to mode separation. Being the only one that can successfully sample from all modes, as observed in Figure 3, ZOD-MC suffers less from metastability. Note there is still some dependence on mode separation due to the x k dependence in the complexity bound in Corollary 3.1.

Dimension Dependence Against Other Diffusion Based Methods. One drawback of our method, is its bad dimension dependence when compared to diffusion based methods. For instance, RDMC and RSDMC have a dependence of exp( O (log( d )) ˜ O ( ε -1 )) and exp( O (log 3 ( dε -1 ))) respectively, in comparison to our exp( ˜ O ( d ) O ((log( ε -1 )))) . Despite this theoretical disadvantage, we find empirically that these methods don't scale well with dimension either. To demonstrate this we sample 5 points on the positive quadrant and use them as means for a GMM. We then evaluate statistics on the generated samples and W 2 as a function of dimension. We observe in Figure 1b that under a fixed number of function evaluations our method results in the lowest W 2 . More details are in Appendix D. Discontinuous Potentials. The use of zeroth-order queries allows ZOD-MC to solve problems that would be completely infeasible to first order methods. To demonstrate this, we modify the

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

smc

proximal

parallel

30

30

30

30

30

OE

30

30

30

30

20

20

20

20

20

20

20

20

20

10

10

10

10

10

10

10

10

10

10

D

10

20

OE

10

20

30

10

20

30

10

20

30

0

10

20

30

10

20

30

0

10

30

10

20

30

0

10

2030

10

20

30

potential in Figure 2. We consider V ( x ) + U ( x ) where U is a discontinuous function given by U ( x ) = 8 ⌊∥ x ∥⌋ ✶ { 5 < ∥ x ∥ < 11 } This creates an annulus of much lower probability and a strong potential barrier. In the original problem, the mode centered at the origin was chosen to have the smallest weight ( 0 . 1 ), but adding this discontinuity significantly changes the problem. As observed in Figure 5, our method is still able to correctly sample from the target distribution, while other methods not only continue to suffer from metastability but also fail to see the discontinuities. We quantitatively evaluate the sampling accuracy by using rejection sampling (slow but unbiased) to obtain ground truth samples, and then compute MMD and W 2 . See Appendix D.2 for details.

0

5

10

15

20

25

Radius

0

1

2

3

4

5

6

7

MMD

ZOD-MC

RDMC

RSDMC

Slips

Smc

Proximal

Parallel

0

5

10

15

20

25

Radius

0

5

10

15

20

25

ZOD-MC

RDMC

RSDMC

Slips

Smc

Proximal

Parallel

0

5

10

15

20

25

Radius

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Mass on Center Mode

ZOD-MC

RDMC

RSDMC

Slips

Smc

Proximal

Parallel

True

Weight

0

1

2

3

4

Time

-

1

0

1

2

3

4

5

6

7

E

p

t

[

‖

s

(

x, t

)

-∇

log

p

(

x, t

)

‖

]

ZOD-MC

RDMC

RSDMC

SLIPS

(b) Average Score Error as a function of time . Shaded is the standard deviation of the errors.

Figure 4: Accuracies of generated samples against dimension and Score Error . On the right, the result for SLIPS is not directly comparable as it has a different forward process.

Score Approximation of Diffusion Based Methods. One explanation of our method's great success in comparison with RDMC and RSDMC is the ability to approximate the score correctly. We select an unbalanced assymetrical 5 d GMM and evaluate the average L 2 score error between methods. On Figure 4b we show that the best approximations of the score are found by using ZODMC as an estimator as opposed to other methods. Even as t increases and the approximation gets harder we are able to retain accuracy and therefore generate high quality samples.

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

a15

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

5

5

5

口

0

D

5

10

0

5

10

10

5

10

5

10

5

10

5

10

10

5

10

10

4.2 Results of Müller Brown Potential

The Müller Brown potential is a toy model for molecular dynamics. Its highly nonlinear potential has 3 modes despite of being the sum of 4 exponentials. The original version has 2 of its modes corresponding to negligible probabilities when compared to the 3rd, which is not good to visualization and comparison across different methods. Thus we consider a balanced version [35] and further translate and dilate x and y so that one of the modes is centered near the origin. The details of the potential can be found in Appendix D.5. Our method is the only one that can correctly sample from all 3 modes as observed in Figure 6 (note they are leveled).

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

2

2

2

2

2

2

2

0

0

0

2

-2

-2

-2

2

2

~2

-4

-4

-4

4

-4

4

-6

6

=6

-6

-6

-6

-6

-H

L

-8

-8

-8

-8

-8

L

-8

Acknowledgments and Disclosure of Funding

The authors are grateful for the partially support by NSF DMS-1847802, Cullen-Peck Scholarship, and GT-Emory Humanity.AI Award. We thank the anonymous reviewers for their helpful comments.

References

K. Balasubramanian, S. Chewi, M. A. Erdogdu, A. Salim, and S. Zhang. Towards a theory of non-log-concave sampling: first-order stationarity guarantees for langevin monte carlo. In Conference on Learning Theory , pages 2896-2923. PMLR, 2022.

C. J. Bélisle, H. E. Romeijn, and R. L. Smith. Hit-and-run algorithms for generating multivariate distributions. Mathematics of Operations Research , 18(2):255-266, 1993.

J. Benton, V. De Bortoli, A. Doucet, and G. Deligiannidis. Linear convergence bounds for diffusion models via stochastic localization. ICLR , 2024.

H. Chen, H. Lee, and J. Lu. Improved analysis of score-based generative modeling: Userfriendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning , pages 4735-4763. PMLR, 2023.

S. Chen, S. Chewi, J. Li, Y. Li, A. Salim, and A. Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. ICLR , 2022.

Y. Chen, S. Chewi, A. Salim, and A. Wibisono. Improved analysis for a proximal algorithm for sampling. In Conference on Learning Theory , pages 2984-3014. PMLR, 2022.

Y. Chen and R. Eldan. Localization schemes: A framework for proving mixing bounds for markov chains. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 110-122. IEEE, 2022.

Y. Chen and K. Gatmiry. A simple proof of the mixing of metropolis-adjusted langevin algorithm under smoothness and isoperimetry. arXiv preprint arXiv:2304.04095 , 2023.

S. Chewi. Log-concave sampling . 2023. Book draft available at https://chewisinho. github.io/ .

S. Chewi, T. Le Gouic, C. Lu, T. Maunu, and P. Rigollet. Svgd as a kernelized wasserstein gradient flow of the chi-squared divergence. Advances in Neural Information Processing Systems , 33:2098-2109, 2020.

G. Conforti, A. Durmus, and M. G. Silveri. Score diffusion models without early stopping: finite fisher information is all you need. arXiv preprint arXiv:2308.12240 , 2023.

A. S. Dalalyan and A. Karagulyan. User-friendly guarantees for the langevin monte carlo with inaccurate gradient. Stochastic Processes and their Applications , 129(12):5278-5311, 2019.

A. S. Dalalyan and L. Riou-Durand. On sampling from a log-concave density using kinetic langevin diffusions. Bernoulli , 26(3):1956-1988, 2020.

V. De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. TMLR , 2022.

P. Del Moral, A. Doucet, and A. Jasra. Sequential monte carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology , 68(3):411-436, 2006.

R. Dwivedi, Y. Chen, M. J. Wainwright, and B. Yu. Log-concave sampling: Metropolis-hastings algorithms are fast. Journal of Machine Learning Research , 20(183):1-42, 2019.

M. Dyer, A. Frieze, and R. Kannan. A random polynomial-time algorithm for approximating the volume of convex bodies. Journal of the ACM (JACM) , 38(1):1-17, 1991.

J. Fan, B. Yuan, and Y. Chen. Improved dimension dependence of a proximal algorithm for sampling. In The Thirty Sixth Annual Conference on Learning Theory , pages 1473-1521. PMLR, 2023.

A. Garbuno-Inigo, F. Hoffmann, W. Li, and A. M. Stuart. Interacting langevin diffusions: Gradient structure and ensemble kalman sampler. SIAM Journal on Applied Dynamical Systems , 19(1):412-441, 2020.

L. Grenioux, M. Noble, M. Gabrié, and A. Oliviero Durmus. Stochastic localization via iterative posterior sampling. In Proceedings of the 41st International Conference on Machine Learning , volume 235, pages 16337-16376. PMLR, 21-27 Jul 2024.

Y. He, K. Balasubramanian, and M. A. Erdogdu. On the ergodicity, bias and asymptotic normality of randomized midpoint sampling method. Advances in Neural Information Processing Systems , 33:7366-7376, 2020.

Y. He, K. Balasubramanian, B. K. Sriperumbudur, and J. Lu. Regularized stein variational gradient flow. arXiv preprint arXiv:2211.07861 , 2022.

Y. He, T. Farghly, K. Balasubramanian, and M. A. Erdogdu. Mean-square analysis of discretized itô diffusions for heavy-tailed sampling. Journal of Machine Learning Research , 25(43):1-44, 2024.

J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33:6840-6851, 2020.

D. Holzmüller and F. Bach. Convergence rates for non-log-concave sampling and log-partition estimation. arXiv preprint arXiv:2303.03237 , 2023.

X. Huang, H. Dong, Y. Hao, Y. Ma, and T. Zhang. Reverse diffusion monte carlo. ICLR , 2024.

X. Huang, D. Zou, H. Dong, Y. Ma, and T. Zhang. Faster sampling without isoperimetry via diffusion-based monte carlo. COLT , 2024.

M. A. Iglesias, K. J. Law, and A. M. Stuart. Ensemble kalman methods for inverse problems. Inverse Problems , 29(4):045001, 2013.

H. Lee, J. Lu, and Y. Tan. Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory , pages 946-985. PMLR, 2023.

H. Lee and Z. Shen. Improved bound for mixing time of parallel tempering. arXiv preprint arXiv:2304.01303 , 2023.

Y. T. Lee, R. Shen, and K. Tian. Structured logconcave sampling with a restricted gaussian oracle. In Conference on Learning Theory , pages 2993-3050. PMLR, 2021.

G. Li, Y. Wei, Y. Chen, and Y. Chi. Towards faster non-asymptotic convergence for diffusionbased generative models. ICLR , 2024.

R. Li, M. Tao, S. S. Vempala, and A. Wibisono. The mirror Langevin algorithm converges with vanishing bias. In International Conference on Algorithmic Learning Theory , pages 718-742. PMLR, 2022.

R. Li, H. Zha, and M. Tao. Sqrt(d) Dimension Dependence of Langevin Monte Carlo. In ICLR , 2021.

X. H. Li and M. Tao. Automated construction of effective potential via algorithmic implicit bias. arXiv preprint arXiv:2401.03511 , 2024.

J. Liang and Y. Chen. A proximal algorithm for sampling. arXiv preprint arXiv:2202.13975 , 2022.

J. Liang and Y. Chen. A proximal algorithm for sampling. arXiv preprint arXiv:2202.13975 , 2022.

Q. Liu. Stein variational gradient descent as gradient flow. Advances in neural information processing systems , 30, 2017.

L. Lovász and M. Simonovits. The mixing rate of markov chains, an isoperimetric inequality, and computing the volume. In Proceedings [1990] 31st annual symposium on foundations of computer science , pages 346-354. IEEE, 1990.

L. Lovász and S. Vempala. Hit-and-run from a corner. In Proceedings of the thirty-sixth annual ACM symposium on Theory of computing , pages 310-314, 2004.

K. L. Mengersen and R. L. Tweedie. Rates of convergence of the hastings and metropolis algorithms. The annals of Statistics , 24(1):101-121, 1996.

R. M. Neal. Annealed importance sampling. Statistics and computing , 11:125-139, 2001.

L. Pardo. Statistical inference based on divergence measures . CRC press, 2018.

L. Richter, J. Berner, and G.-H. Liu. Improved sampling via learned diffusions. ICLR , 2024.

H. E. Robbins. An empirical bayes approach to statistics. In Breakthroughs in Statistics: Foundations and basic theory , pages 388-394. Springer, 1992.

G. O. Roberts and R. L. Tweedie. Geometric convergence and central limit theorems for multidimensional hastings and metropolis algorithms. Biometrika , 83(1):95-110, 1996.

A. Roy, L. Shen, K. Balasubramanian, and S. Ghadimi. Stochastic zeroth-order discretizations of langevin diffusions for bayesian inference. Bernoulli , 28(3):1810-1834, 2022.

A. Salim, L. Sun, and P. Richtarik. A convergence theory for svgd in the population limit under talagrand's inequality t1. In International Conference on Machine Learning , pages 19139-19152. PMLR, 2022.

A. Schlichting. Poincaré and log-sobolev inequalities for mixtures. Entropy , 21(1):89, 2019.

R. Shen and Y. T. Lee. The randomized midpoint method for log-concave sampling. Advances in Neural Information Processing Systems , 32, 2019.

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. ICML , 2015.

Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations , 2021.

F. Vargas, W. Grathwohl, and A. Doucet. Denoising diffusion samplers. arXiv preprint arXiv:2302.13834 , 2023.

F. Vargas, A. Ovsianas, D. Fernandes, M. Girolami, N. D. Lawrence, and N. Nüsken. Bayesian learning via neural schrödinger-föllmer flows. Statistics and Computing , 33(1):3, 2023.

F. Vargas, S. Padhy, D. Blessing, and N. Nüsken. Transport meets variational inference: Controlled monte carlo diffusions. In The Twelfth International Conference on Learning Representations , 2024.

S. Vempala and A. Wibisono. Rapid convergence of the unadjusted langevin algorithm: Isoperimetry suffices. Advances in neural information processing systems , 32, 2019.

K. Yingxi Yang and A. Wibisono. Convergence of the inexact langevin algorithm and scorebased generative models in kl divergence. arXiv e-prints , pages arXiv-2211, 2022.

Q. Zhang and Y. Chen. Path integral sampler: a stochastic control approach for sampling. ICLR , 2022.

A Related Works on Zeroth-Order Sampling

The zeroth-order sampling algorithms have been widely studied in the past decades. There is a class of zeroth-order sampling algorithms, including the Ensemble Kalman Inversion [28] and the Ensemble Kalman Sampler [19], that are based on moving a set of easy-to-sample particle according to certain dynamics. However, these methods require (noisy) observations from the target distribution rather than queries of the potential function. Within the zeroth-order sampling algorithms using queries of the potential function, one type of methods make use of the zeroth-order queries to approximate the gradient and apply it to some first-order sampling algorithm [12, 47, 23]. Since it is based on the first-order methods, the analysis of this type of algorithms assumes the target distribution satisfies certain isoperimetric property in general. The other type of methods utilize the zeroth-order queries directly without relating to the gradient. Such methods include the Rejection sampling algorithm, the Metropolized Random Walk (MRW) [41, 46], Ball Walk [39, 17], Hit-and-Run algorithm [2, 40]. The rejection sampling algorithm requires to finding an envelope function which is easy to sample from. This could be difficult. MRW requires sufficient smooth and light tail of the target distribution to mix fast. Ball walk and Hit-and-Run algorithms assume the target distribution is compactly supported. In this paper, we develop a zeroth-order sampling algorithm based on the reverse OU process. Our algorithm does not suffer from the difficulties in the rejection sampling and MRW, and our analysis does not assume isoperimetric property and compact support of the target distribution.

[25] also studies the complexity of a zeroth-order sampling algorithm that combines an approximation technique and rejection sampling. For target distributions that are m -differentiable with compact support, [25][Theorem 12] implies a complexity of order Ω d ( ε -d/m ) to reach an ε -accuracy in KL-divergence, where the dimension dependence is implicit. Compare to our result in Corollary 3.1, both complexities are polynomial in ε and exponential in d . Our complexity is smaller for less smooth targets ( m< 2 ) while their complexity is smaller for smoother targets ( m> 2 ). However, result in [25] only applies to smooth targets ( m> 0 ) with compact supports, while our Corollary 3.1 applies to more general target distributions which can be with full support with even discontinuous potentials.

B More Details on ZOD-MC

In this section, we provide more details on how rejection sampling (Algorithm 3) in ZOD-MC implements the oracle in DDMC, i.e., generating Monte Carlo samples required in Algorithm 2. Construction of an envelope. If we have V ∗ as a minimum value of V , then by noting that:

We are able to construct an envelope for rejection sampling. In particular we propose a samples z from N ( · ; e t x, e 2 t -1) and accept proposal z with probability exp( -V ( z ) + V ∗ ) .

Sampling from the target distribution. Algorithm 3, implements the oracle in Algorithm 2 with δ ( t ) = 0 . When n ( t ) increases, Algorithm 2 outputs unbiased Monte Carlo score estimators with smaller variance, hence closer to the true score. We will quantify the convergence of DDMC next and consequently demonstrate that ZOD-MC can sample general non-logconcave distributions.

C Proofs

C.1 Properties of the OU-Process

In this section, we introduce and prove some useful properties of the OU-process. Throughout this section, we denote { X t } t ≥ 0 as the solution of (1) with p t := Law( X t ) . For any s, t > 0 , p t | s denotes the conditional probability measure of X t given the value of X s .

Proposition C.1. (Decay along the OU-Process) Let { X t } t ≥ 0 be the solution of (1) with p t := Law( X t ) . Assume that the initial distribution p satisfies Assumption 3.1. Then we have

Proof of Proposition C.1. The proof for (11) is based on the fact that the solution to (1) can be represented by

We have

Next, to prove (12), we have

where the inequality follows from the convexity of KL divergence. According to (13), p t | 0 ( ·| y ) is a Gaussian measure with mean e -t y and covariance matrix (1 -e -2 t ) I d . According to [43], we have

As a result, we get

where the last inequality follows from the fact that ln(1 + x ) x for all x > 0

≤ .

Proposition C.2. (Stochastic Dynamics along the OU-Process) Let { X t } t ≥ 0 be the solution of (1) . Define m t ( X t ) := E X 0 ∼ p 0 | t ( ·| X t ) [ X 0 ] and Σ t ( X t ) := Cov X 0 ∼ p 0 | t ( ·| X t ) ( X 0 ) = E X 0 ∼ p 0 | t ( ·| X t ) [( X 0 -m t ( X t )) ⊗ 2 ] . Then we have for all t ≥ 0 ,

The above proposition is known in stochastic localization literature [7] and diffusion model literature [3]. We present its proof for the sake of completeness.

Proof of Proposition C.2. For any T > 0 , from (13), we have the conditional distribution

where { X t } 0 ≤ t ≤ T is the solution of (1). Noticing that the solution of (2), { ¯ X t } 0 ≤ t ≤ T is the reverse process of { X t } 0 ≤ t ≤ T and it satisfies ¯ X t = X T -t in distribution for all t ∈ [0 , T ] . Therefore, it suffices to study

where the normalization constant Z t = ∫ R d exp( h t ( x )) p (d x ) . We have q 0 | t (d x | ¯ X t ) = p 0 | T -t (d x | X T -t ) in distribution for all t ∈ [0 , T ] and

where the above two identities hold in distribution. For simplicity, we denote σ t = √ 1 -e -2 t . Then h t ( x ) = -1 2 ( σ -2 T -t -1) ∥ x ∥ 2 + σ -1 T -t √ σ -2 T -t -1 ⟨ x, ¯ X t ⟩ is a stochastic process linearly depending on

{ ¯ X t } t ≥ 0 . The conditional measure { q 0 | t ( x | ¯ X t ) } t ≥ 0 is a measure-valued stochastic process, whose dynamics can be studied by applying Itô's formula. First we have

Since { ¯ X t } 0 ≤ t ≤ T solves (2), according to Lemma 1, it satisfies that

Based on (17), (18) and (19), we have

If we define R t ( ¯ X t ) = q 0 | t (d x | ¯ X t ) p (d x ) = Z -1 t exp( h t ( x )) , then apply Itô's formula again and we have

Now combine the results in (17), (18), (20) and (21), we can derive the differential equation of ¯ m t ( ¯ X t ) :

Utilize (19) and the definition of σ t , all terms with factor d t in the above equation cancel and (22) can be simplified as

Last, we derive the differential equation that E X t ∼ p t [ Σ t ( X t ) ] satisfies. Let f ( t ) := E X t ∼ p t [ Σ t ( X t ) ] and g ( t ) := E X t ∼ p t [ Σ t ( X t ) 2 ] be two deterministic functions on [0 , T ] . According to (16) and (23), we have

where the last inequality follows from the Itô isometry. Proposition C.2 is then proved by reverse the time in f and g .

C.2 Proofs of Section 3.1

Proof of Lemma 1. Based on (13), we have p t = ( e -t ) # p ∗ ( √ 1 -e -2 t ) # γ d where ( e -t ) # p is the pushforward measure of p via map x ∈ R d ↦→ e -t x and ( √ 1 -e -2 t ) # γ d is the pushforward measure of γ d via map x ∈ R d ↦→ √ 1 -e -2 t x ∈ R d . The pushforward measures ( e -t ) # p and ( √ 1 -e -2 t ) # γ d can be written as

respectively. Therefore the score function ∇ ln p t ( x ) can be written as

where the last step follows from the Bayesian rule and

C.3 Proofs of Section 3.2

Proof of Proposition 3.1. With the score estimator given in Algorithm 2, we have

where { x t,i } n ( t ) i =1 is a sequence of i.i.d. samples following p 0 | t ( ·| x ) that are chosen such that E [ ∥ x t,i -z t,i ∥ 2 | x ] = W 2 ( p 0 | t ( ·| x ) , Law( z t,i )) for all t > 0 and i = 1 , 2 , · · · , n ( t ) . Based on Lemma 1, { e -t x t,i -x 1 -e -2 t } n ( t ) i =1 is a sequence of unbiased i.i.d. Monte Carlo estimator of ∇ ln p t ( x ) for all t > 0 and x ∈ R d . Therefore, we get

The first term in the above equation, N 1 , is related to the covariance of p 0 | t ( ·| X t ) , which is studied in Proposition C.2. We have

where the inequality follows from Proposition C.2 indicating that t ↦→ E x ∼ p t [ trace ( Σ t ( x ) )] is a increasing function.

The second term N 2 characterize the bias from the Monte Carlo samples and the bias can be measured by the Wasserstein-2 distance:

(4) follows from the estimation on N 1 and N 2 .

C.4 Proof of Theorem 1

In this section, we introduce the proof of our main convergence results, Theorem 1. Recall that in the convergence result in Theorem 1, three types of errors appear in the upper bound: the initialization

error, the discretization error and the score estimation error. Our proof compares the trajectory of { ˜ X t } 0 ≤ t ≤ T that solves (8) and the trajectory of { ¯ X t } 0 ≤ t ≤ T that solves (2). We denote the path measures of { ¯ X t } 0 ≤ t ≤ T and { ˜ X t } 0 ≤ t ≤ T by P p T , and Q γ d , respectively. Next, we introduce a high level idea on how the three types of errors are handled .

Initialization error: the initialization error comes from the comparison between { ˜ X t } 0 ≤ t ≤ T and { ˜ X p T t } 0 ≤ t ≤ T . To characterize this error, we introduce the intermediate process { ˜ X p T t } 0 ≤ t ≤ T

in (24) and denote the path measure of { ˜ X p T t } 0 ≤ t ≤ T by Q p T . Both processes are driven by the estimated scores and only the initial conditions are different. We factor out the initialization error from KL ( p δ | q t N ) by the following argument:

where the inequality follows from the data processing inequality and the last identity follows from the fact that d Q p T d Q γ d = d Q p T 0 d Q γ d 0 = d p T d γ d , which is true because the processes { ˜ X t } 0 ≤ t ≤ T and { ˜ X p T t } 0 ≤ t ≤ T have the same transition kernel function. KL ( p T | γ d ) is the initialization error and it is bounded based on (12) in Proposition C.1.

Discretization error: the dicretization error arises from the evaluations of the scores at the discrete times. We factor out the discretization error from the KL ( P p T | Q p T ) via the Girsanov's Theorem.

We bound the discretization error term in the above equation by checking the dynamical properties of the process {∇ ln p t ( ¯ X t ) } 0 ≤ t ≤ T . Similar approach was used in the analysis of denoising diffusion models, see [3]. For the sake of completeness, we include the proof in Appendix C.7.

Score estimation error: as discussed in the discretization error, the score estimation error is the accumulation of the L 2 -error between the true score and score estimator at the time schedules { T -t k } N -1 k =0 . In the analysis of denoising diffusion models, [5, 4, 3], it is usually assumed that such a L 2 score error is small. In this paper, we consider to do sampling via the Reverse OU-process and score estimation. One of our main contribution is that we prove the L 2 score error can be guaranteed small for the class of Monte Carlo score estimators given in Algorithm 2. The L 2 score error upper bound is stated in Proposition 3.1.

Proof of Theorem 1. First we can decompose KL ( p δ | q t N ) into summation of the three types of error.

where the first inequality follows from the data processing inequality. The second inequality follows from Girsanov's theorem and [6, Section 3.1]. According to Proposition C.1 and the assumption that T > 1 , the initialization error satisfies

According to Lemma 2, the discretization error satisfies

Reordering the summation in the second term and we have

Recall that Σ t ( X t ) = Cov( X 0 | X t ) for all 0 ≤ t ≤ T . We have

where the last identity follows from (13). (29) and (28) implies that E [ trace ( Σ t ( X t ) )] ≲ (1 -e -2 t )( d +m 2 2 ) for all 0 ≤ t ≤ T . Therefore, from (26) and (27), the overall discretization error can be bounded as

Last, according to Proposition 3.1, the score estimation error satisfies

and (9) follows from (25), (30) and (31).

C.5 Discussion on the Step-size

In this section, we first state error bounds of DDMC under different choices of step-size. Then we provide the detailed calculations. Last we compare our results in Theorem 1 to existing results on convergence of denoising diffusion models.

In the following discussion, we assume δ ( T -t k ) 2 ≤ dγ k e 2( T -t k ) and n ( T -t k ) ≥ γ -1 k e -2( T -t k ) for all k = 0 , 1 , · · · , N -1 , so that the score estimation error is dominated by the discretization error. For different choices of step-size, we discuss the parameter dependence of the error bound in (9) under the assumptions on δ ( t ) and n ( t ) .

constant step-size: the constant step-size is widely considered in sampling algorithms and denoising diffusion generative models. It requires γ k = γ for all 0 ≤ k ≤ N -1 . Then

linear step-size: the linear step-size is considered by [6] as an interpretation of the uniform discretization of a diffusion model with non-constant diffusion coefficient [52]. It requires t k = T -( δ +( N -k ) γ ) 2 with γ = √ T -δ N for all 0 ≤ k ≤ N -1 . Then

exponential-decay step-size: the exponential-decay step-size is considered to be optimal in SGMs [6, 3]. It requires γ k = κ min(1 , T -t k ) for some κ ∈ (0 , 1) . Then

The purple terms are denoting the discretization errors. For all of the above choices of step-size, the error bounds have the same linear dimension dependence and different dependence on the early stopping parameter δ . Next, we provide a detailed calculation of these error bounds and a derivation of optimal δ -dependence.

constant step-size: when γ k = γ for all k = 0 , 1 , · · · , N -1 , we have T -t k = δ +( N -k ) γ and

Therefore

linear step-size: when T -t k = ( δ +( N -k ) γ ) 2 with γ = √ T -δ N , we have γ k = (2 δ +(2 N -2 k -1) γ ) γ = Θ( √ T -t k γ ) and

Therefore

exponential-decay step-size: when γ k = κ min(1 , T -t k ) with κ = T +ln(1 /δ ) N , we have

Therefore

Optimality of the exponential step-size: assuming that γ k = Θ( γ k -1 ) for all k = 0 , 1 , · · · , N -1 , the exponential step-size actually provides the optimal order estimation for the error terms. Noticing that the error terms all depend on the quantity γ k 1 -e -2( T -t k ) which is of order

Therefore

Noticing that x ↦→ x 2 and x ↦→ x 3 are both convex functions on the domain x ∈ (0 , ∞ ) . Since ∑ 1 <T -t k <T γ k = T -1 is fixed, according to Jensen's inequality, ∑ 1 <T -t k <T γ 2 k and ∑ 1 <T -t k <T γ 3 k reach their minimum when γ k are constant-valued for all k such that T -t k > 1 . Similarly, let β k = ln ( T -t k T -t k +1 ) ∈ (0 , ∞ ) . Then γ k T -t k = 1 -e -β k and ∑ δ<T -t k < 1 β k = ln(1 /δ ) is fixed. Since x ↦→ (1 -e -x ) 2 and x ↦→ (1 -e -x ) 3 are both convex functions on the domain x ∈ (0 , ∞ ) , according the Jensen's inequality, ∑ δ<T -t k < 1 γ 2 k ( T -t k ) 2 and ∑ δ<T -t k < 1 γ 3 k ( T -t k ) 3 reach their minimum when β k are constant-valued for all k such that T -t k < 1 .

Comparison to convergence results in denoising diffusion models. (9) in Theorem 1 bounds the error of DDMC by I, II, III, which reflect the initialization error, the discretization error and the score estimation error, respectively. Assuming the score estimation error is small, (9) reduces to the same type of results that study the error bound for the denoising diffusion models (Algorithm 1). In [5], the discretization error is proved to be of order O ( d ) assuming the score function is smooth along the trajectory and a constant step-size. In [4], they get rid of the trajectory smoothness assumption and prove a O ( d 2 ) discretization error bound with early-stopping. In [3], the discretization error bound is improved to O ( d ) with early-stopping and exponential-decay step-size, and without the trajectory smoothness assumption. Compared to these works, our result in Theorem 1 also implies a O ( d ) discretization error without the trajectory smoothness assumption and it applies to any choice of step-size with early stopping, as we discussed above. As shown in [3], the O ( d ) is the optimal for the discretization error. Therefore, our results indicates that with early-stopping, the denoising diffusion model achieves the optimal linear dimension dependent error bound.

C.6 Proofs of Section 3.3

To prove the query complexity of ZOD-MC, we first look at query complexity of Algorithm 3.

Query complexity of Algorithm 3 . The query complexity of Algorithm 3 is essentially the number of proposals we need so that n ( t ) of them can be accepted. Intuitively, to get one sample accepted, the number of proposals we need is geometrically distributed with certain acceptance probability [9]. We state this formally in the following proposition, for which it suffices to assume a relaxation of the commonly used gradient-Lipschitz condition on the potential.

Proposition C.3. Under Assumption 3.2, the expected number of proposals for obtaining n ( t ) many exact samples from p 0 | t ( ·| x ) defined in Lemma 1 via Algorithm 3, is

Remark 5. Our complexity bound in Proposition C.3 exponentially depends on the dimension. This is due the curse of dimensionality phenomenon in the rejection sampling: the acceptance rate and algorithm efficiency decreases significantly when the dimension increases.

Proof of Proposition C.3. For each t ∈ [0 , T ] , the expected number of iterations in the rejection sampling to get one accepted sample is

To get n ( t ) many samples, the expected number of iterations we need is N ( t ) = n ( t ) M ( t ) .

Query complexity of ZOD-MC. The query complexity of ZOD-MC, denoted as ˜ N , is essentially the sum of query complexities in Proposition C.3 over the discretized time points, i.e. ˜ N = ∑ N -1 k =0 N ( T -t k ) .

Proof of Corollary 3.1. First, for δ = Θ ( min( ε 2 W 2 d , ε W 2 m 2 ) ) , it follows from Proposition C.1 that W 2 ( p, p δ ) ≤ ε W 2 .

Next, under the exponential-decay step size, according to Theorem 1, if we set n ( T -t k ) = γ -1 k e -2( T -t k ) , then

By choosing T = 1 2 ln( d +m 2 2 ε KL ) , N = Θ ( max( ( d +m 2 2 ) 1 2 ( T +ln( δ -1 )) 3 2 ε 1 2 KL , d ( T +ln( δ -1 )) 2 ε KL ) ) and κ =

Θ ( T +ln( δ ) N ) , we have KL ( p δ | q t N ) ≲ ε KL. Last, it follows from Proposition C.3 that

-1

By plugging in δ, T, t k and N , (10) is proved.

C.7 Side Lemmas

Lemma 2. Let { ¯ X t } 0 ≤ t ≤ T be the solution to (2) . Then for any 0 ≤ k ≤ N -1 , we have

where { Σ t ( X t ) } 0 ≤ t ≤ T is defined in Proposition C.2.

Proof of Lemma 2. For fixed s , consider the process {∇ ln p T -t ( ¯ X t ) } 0 ≤ t ≤ T , denoted as { L t } 0 ≤ t ≤ T , and a function E s,t := E [ ∥ L t -L s ∥ 2 ] . It is shown by Itô's formula in [3, Lemma 3] that

and as a result, (32) implies that

Apply (32) and Itô's formula again, we have

Therefore (33) can be rewritten as

Let { X t } 0 ≤ t ≤ T be the solution of (1). Since ¯ X t = X T -t in distribution for all t ∈ [0 , T ] , E [ ∥ L s ∥ 2 ] and E [ ∥ ∥ ∇ 2 ln p T -t ( ¯ X t ) ∥ ∥ 2 F ] can both be represented by the covariance matrix defined in Proposition C.2. It is proved in [3, Lemma 6] that

and

Now we choose s = t k in (34) and integrate from t k to t . According to (35) and (36), we have

where the last identity follows from integration by parts. According to Proposition C.2, t ↦→ E [ trace ( Σ T -t ( X T -t ) )] is positive and decreasing. Therefore, we have for all t ∈ [ t k , t k +1 ] ,

Integrate again from t = t k to t = t k +1 , we get

0

2000

4000

6000

8000

10000

Gradient Complexity

10

1

10 0

10 1

10 2

Time (s)

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

D More Experiments

D.1 Samples from 2D GMM at different Oracle Complexities

We sample from a Gaussian Mixture model with 4 modes, the following summarizes the parameters of the GMM.

We display the generated samples at different oracle complexities in Figures 8, 9, 10. Notice that the mode located at the origin holds less weight, and as the oracle complexity increases our method becomes better at sampling from other modes, as opposed to the corresponding baselines.

We detail the hyperparameters in Table 2 and the wall clock time of different methods in Figure 15.

(a) Generated samples at

200

oracle complexity per score evaluation

(b) Generated samples at

1200

oracle complexity per score evaluation

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

D

10

10

0

10

10

10

10

10

10

10

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

aIS

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

5

0

10

10

10

10

10

10

10

10

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

囍

10

10

10

10

10

10

10

10

10

10

10

10

OT

10

10

10

10

10

(c) Generated samples at

2200

oracle complexity per score evaluation

(a) Generated samples at

3200

oracle complexity per score evaluation

(b) Generated samples at

4200

oracle complexity per score evaluation

(c) Generated samples at

5200

oracle complexity per score evaluation

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ails

Smc

parallel

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

dlS

parallel

16

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

smc

oroximal

parallel

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

als

sImC

(d) Generated samples at

6200

oracle complexity per score evaluation

(a) Generated samples at

7200

oracle complexity per score evaluation

(b) Generated samples at

8200

oracle complexity per score evaluation

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

0

10

10

10

10

OT

10

10

10

10

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

5

0

10

10

-5

10

10

10

10

10

10

-5

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

sie

proximal

parallel

10

10

囍

10

10

10

10

101

o

10

。

10

10

OT

OT

10

10

(c) Generated samples at

9200

oracle complexity per score evaluation

D.2 Samples from Discontinuous 2D GMM at different Oracle Complexities

We display the generated samples at different oracle complexities in Figures 11 ,12, 13. At the end we show the W 2 and MMD for this example in Figure 14.

We detail the hyperparameters in Table 3 .

(a) Generated samples at

200

oracle complexity per score evaluation

(b) Generated samples at

1200

oracle complexity per score evaluation

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

10

OT

10

10

10

40

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

aIS

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

10

-5

10

10

10

10

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

OT

10

10

10

10

10

10

10

10

10

10

10

OT

10

10

(c) Generated samples at 2200 oracle complexity per score evaluation

(a) Generated samples at

3200

oracle complexity per score evaluation

(b) Generated samples at

4200

oracle complexity per score evaluation

(c) Generated samples at

5200

oracle complexity per score evaluation

(d) Generated samples at

6200

oracle complexity per score evaluation

(a) Generated samples at

7200

oracle complexity per score evaluation

(b) Generated samples at

8200

oracle complexity per score evaluation

Ground Truth

ZOD-MC

RDMC

RSDMC

sllps

DUUs

angevin

parallel

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

5

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

ais

proximal

parallel

10

10

10

10

10

10

10

10

10

10

OT

10

10

10

10

1.0

GroundTruth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

10

10

10

10

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

Smc

proximal

parallel

10

10

10

OT

10

10

10

10

10

10

10

10

OL

10

10

10

10

(c) Generated samples at 9200 oracle complexity per score evaluation

0

2000

4000

6000

8000

10000

Oracle Complexity

1

2

3

4

5

6

MMD

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

0

2000

4000

6000

8000

10000

Oracle Complexity

6

8

10

12

14

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

0

2000

4000

6000

8000

10000

Gradient Complexity

10

1

10 0

10 1

10 2

Time (s)

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

D.3 Samples from different radius

We display the generated samples at different radius in Figures 16, 18 and the hyperparameters in Table 4.

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

8

8

8

6

6

6

6

6

6

642

4

4

4

4

42

4

N

2

2

N

2

2

0

0

。

0

-2

2

2

2

2

2

0

5

S

0

5

0

0

5

5

5

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

10

10

10

10

10

10

10

10

10

10

5

5

5

0

0

0

0

0

0

0

0

10

10

D

L

10

10

0

10

10

10

10

10

0

10

Ground Truth

ZOD-MC

RRRXRRRR

RDMC

RSDMC

slips

ais

smc

proximal

parallel

15

15

15

10

10

10

10

10

10

10

20

Ground Truth

201510

ZOD-MC

2

RDMC

20 153 05

RSDMC

slips

aIS

smc

proximal

parallel

15

10

20

10

20

20

OT

10

Ground Truth

ZOD-MC

RDMC

XXXXXXR

RSDMC

slips

aIS

Smc

proximal

parallel

25

21510

20

20

10

20

20

Ground Truth

ZOD-MC

RDMC

RSDMC

slips

ais

smc

proximal

parallel

30

30

30

30

30

30

30

30

30

30

20

20

20

20

20

20

20

20

20

20

10

10

10

10

10

10

10

10

10

10

0

0

0

0

0

0

0

0

0

Q

10

20

30

10

20

30

10

20

30

0

10

20

30

0

10

20

30

Q

10

20

30

10

20

30

Q

10

20

30

10

20

30

Q

10

20

30

(a) Generated samples at R = 26

D.4 Higher Dimensional Examples

Score Error Approximation Details. We use the following 5 d Gaussian mixture to measure the error of the score approximation:

Randomized Gaussian Mixtures To generate the results in Figure 1b we proceed as follows. For a given dimension we take:

Where z ∼ U [0 , 1] d , additionally we sample σ 2 ∼ U [ . 3 , 1 . 3] . We then consider the Gaussian target N ( µ, σ 2 I ) . We repeat this 5 times and create a Gaussian mixture with equally weighted modes. We plot the the 2 d marginals of the target distribution as long as the generated samples.

D.5 Muller Brown Potential Details

The potential is given by V ( x, y ) = β · ( V m ( x, y ) + V q ( x, y ))

where V m corresponds to the original Müller Brown and V q ( x, y ) = 35 . 0136( x -x ∗ c ) 2 +59 . 8399( y -y ∗ c ) 2 , with ( x ∗ c , y ∗ c ) is approximately the minimizer at the center of the middle potential well, and V q is a correction introduced so that the depths of all three wells are.

D.6 Score error at t = T

One natural concern is that the sampling problem at t = T could be nearly as hard as sampling from the target distribution. Therefore, only a small number of samples could be accepted and the score error would be high. We display the score error at t = T to show that this is not necessarily the case.

0

20000

40000

60000

80000

Oracle Complexity

-

1

0

1

2

3

4

E

p

t

[

‖

s

(

x, t

)

-∇

log

p

(

x, t

)

‖

]

ZOD-MC

RDMC

RSDMC

SLIPS

(a) Score error at t = T = 4 . for different diffusion samplers, we used t ≈ 1 for SLIPS. Results are for the 2 d GMM in the main paper

0

20000

40000

60000

80000

Oracle Complexity

-

1

0

1

2

3

4

E

p

t

[

‖

s

(

x, t

)

-∇

log

p

(

x, t

)

‖

]

ZOD-MC

RDMC

RSDMC

SLIPS

D.7 Further details on number of accepted samples

We present the number of accepted samples from our rejection sampler as a function of time. Specifically we consider 1000 trajectories of the diffusion and for every intermediate step we sample 10 K samples. We present the number of accepted samples in Table 5.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Section 1, Table 1.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Section 3.3, Remark 5 and the discussion after

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Main results: Proposition 3.1, Theorem 1, Corollary 3.1. Proofs are included in Appendix B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Code is provided with instructions to run it. Scripts are provided so that a one line script can reproduce all the plots found in the paper.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code is provided with instructions to run it. Scripts are provided so that a one line script can reproduce all the plots found in the paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes, the target distributions are well described, furthermore every hyperparameter can be easily checked in the configuration files or the corresponding script for each experiment.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In Figure 4b we present the standard deviation of the errors in the score approximation. In other experiments we didn't find it insightful to add these statistics.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes, we give this information at the beginning of section 4

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our paper studies the fundamental theory of sampling, with no specified applications.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: We didn't use any dataset or asset from another party during these experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Our paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
The authors investigate a popular reverse-diffusion sampling process, when the scores are estimated not from target samples but using the target's unnormalized density, which is the setup in energy-based modelling. Specifically, the authors consider a recent Monte-Carlo estimator of the scores which requires sampling from a possibly multimodal distribution (product of target and Gaussian densities). To do this, the authors use rejection sampling and obtain an upper bound on the convergence of their method. They also empirically validate their method in low dimensions with thorough experiments that monitor and benchmark against other methods:

- **convergence** in MMD, W2, and generic statistics vs. **cost** in dimension or target queries) in Figure 1 
- quantifying **mode coverage** in Figure 4

as well as other visual diagnostics of convergence.

STRENGTHS:
The paper is clearly written and the results are interesting. Their experiments are thorough, I appreciate the use of the $W2$ in Fig 1.a. as it is sensitive to mode coverage which is less the case of some other sampling metrics.

WEAKNESSES:
The authors are clear about the limitations of their method, for example the complexity in Corollary 3.1. is exponential not only in the dimension $d$ but also in the smoothness $L$ of the target (this holds even in one dimension).

QUESTIONS:
-

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors are interested in the problem of sampling from an arbitrary non-logconcave probability distribution (namely, multi-modal distribution) with only access to its unnormalized density. While most of popular sampling methods rely on queries of the score of the target, i.e. the gradient of the log-density (1st-order methods), the proposed approach is a 0th-order sampling method, as it relies on queries of the unnormalized target density itself. Inspired by the performance of denoising diffusion models for generative tasks, they propose a sampling algorithm, Zeroth-Order Diffusion Monte Carlo (ZOD-MC), which simulates a discretized and approximate version of the time-reversal of the standard Ornstein-Uhlenbeck diffusion process (the ideal scheme being known to converge to the target). In this algorithm, rejection-sampling is used to provide a Monte Carlo estimate of the intractable scores of the process marginals that appear in the recursion. This is made possible by the Tweedie's formula, that links the score of the marginals to an expectation over the posteriors of the model. As rejection sampling is known to suffer from the curse of dimensionality, the authors acknowledge that ZOD-MC cannot address sampling problems with relatively high dimensions ($d\\geq10$). To support their algorithm, they provide a convergence analysis which requires a weak assumption on the target (bounded second order moment) and a control of the Monte Carlo estimation error for the posterior distributions at any time of the process. Under an extra-assumption on this control, they derive a readable oracle complexity of ZOD-MC that quantifies the number of queries of the target density in the whole sampling procedure. Finally, they conduct numerical experiments on low dimensional settings (mostly $d=2$), by considering multi-modal distributions which increasing higher energy barriers or with discontinuous potential. They compare ZOD-MC to recent approaches based on denoising approaches [1,2] and standard MCMC approaches such as Langevin or parallel tempering. In the considered settings, ZOD-MC exhibits the best performance at equivalent computational budget (namely, same oracle complexity)\n\n[1] Reverse Diffusion Monte Carlo. Huang et al. 2023.\n\n[2] Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo. Huang et al. 2024\n\nSTRENGTHS:\n- The paper is didactic in the sense that the motivations and the bricks to build the algorithm ZOD-MC are exposed in a logical way.\n- The authors pay attention to bring intuition on their theoretical results.\n- The mathematical results are clear to understand, especially Theorem 1.\n- Although the numerical experiments are conducted on low dimensional settings, they consider interesting cases, i.e., sampling from multi-modal distributions where the modes are increasingly further from each other and Gibbs distributions with discontinuous potential.\n\nWEAKNESSES:\nIn my opinion, this paper suffers from 3 main weaknesses, which explains my score.\n\n1. The comparison to the related work is incomplete. The authors do not cite the work [1] (although it was published approximately at the same time as [2]), where the authors propose a 1st-order sampling method, SLIPS, based on stochastic localization, that can be linked to a denoising diffusion model. In SLIPS, samples from the target distribution are obtained by following a diffusion process with intractable drift, which is nothing less than an expectation of a posterior distribution; in practice, the authors estimate this term by MCMC method (Langevin). In contrast to RDMC or ZOD-MC, SLIPS may exhibit multiple advantages:  (i) it can be applied to a flexible class of stochastic localization schemes, whereas RDMC and ZOD-MC only consider the standard Ornstein-Uhlenbeck diffusion process, (ii) it provides an exact finite-time setting for sampling (which is not the case for RDMC or ZOD-MC),  (iii) it is shown to scale well in dimension in their experiments (up to dimension 100), (iv) it has a small oracle complexity per score evaluation compared to the numerics presented here (32 vs 200 at least in the numerics of the current paper). Since the current approach has the same spirit as SLIPS (diffusion-based approach with MC estimation), it should be compared with it. Moreover, no numerical comparison with AIS or SMC (which are considered gold-standard to sample from low-dimensional multi-modal distributions) is provided. For me, this justifies why this paper does not meet the standards of NeurIPS. \n\n2. In my opinion, the proposed approach misses a fundamental point, which lies in the MC estimation of the denoiser (i.e. the conditional expectation over the posterior distribution): it is linked to the ability to be able to provide an accurate MC estimate near at $t_0=0$, equivalently $t=T$, namely when $p_t$ is fully noised. When $t\\approx T$, it is clear that the corresponding posterior distribution is approximately equal to the target distribution itself (since we have full noise, the quadratic term vanishes). Therefore, applying rejection sampling on the posterior distribution at the very beginning of the sampling process in ZOD-MC is equivalent to do rejection sampling on the target distribution itself ! Hence, in practice, **the whole diffusion-based approach presented in ZOD-MC to sample from the target distribution via an annealing scheme is actually as hard as directly sampling from the target distribution**, which hurts its usefulness. This remark is intimately linked to the notion of duality of log-concavity presented in [1], which explains that it is required to start the diffusion process further such that the posterior distribution is \"smoother\", and easier to sample from. On the theoretical side, I have a concerns that is linked to my point:  it seems to me that the authors make an assumption on the MC error $\\delta(t)$ to estimate the denoiser  (which is not explicited in the main) in order to derive the total complexity cost (hidden in Section B.5 in the appendix); as said above, this control is actually crucial in practice and it is not straightforward to verify an upper bound on it.\n\n3. The section on numerical experiments is incomplete for several reasons: no indication of tuning of the hyperparameters ($T$, $N$, step-size) either for ZOD-MC or other approaches, no ablation study on these hyperparameters to exhibit robustness, display of the estimation error of the true score for a large $t$ (where it becomes harder) and a variety of levels of oracle complexity, the plots in Figure 1 and Figure 4.a do not display any variance, missing related work as stated in my claim 1.\n\n[1] Stochastic Localization via Iterative Posterior sampling. Grenioux et al. 2024\n\n[2] Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo. Huang et al. 2024\n\nQUESTIONS:\n- I have a question about Algorithm 1: to obtain the sampling recursion with the MC estimator, you first discretize the SDE with exponential integration and then replace the score by its MC estimator. Would it bring less discretization error to first replace the score in the SDE by the term involving the conditional expectation and then apply exponential integration upon it ? It seems to me that exponential integration would still be tractable in this case. \n- To support my claim 2 in the 'Weaknesses' section, could the authors provide (i) the value of the average score error between the true score and the approximated score near $t=T$, while varying the number of queries to the target distribution (ie oracle complexity per score evaluation)  and (ii) results of rejection sampling directly applied on the target distribution with same oracle complexity as other methods ?\n\nLIMITATIONS:\nThe main (and crucial) limitation of ZOD-MC is the fact that it can only be applied to small dimensional settings ($d\\leq 5$), as rejection sampling is notoriously known to suffer from the curse of dimensionality (which is the reason why it is not a popular sampling method in practice). Although the authors acknowledge this limitation in the abstract, it is not well highlighted at all in the main of the paper: this limitation appears explicitly in Remark 5 in Section 3.3 in the Appendix, but not in the main paper. As suggested by NeurIPS guidelines, a section/paragraph 'Limitations' should be given in the main.\n\nOn the other hand, ZOD-MC requires to know the location of local minima of the energy landscape, i.e., the location of the modes of the target distribution. As indicated by the authors, this can be done by applying Newton's method on the target potential (see Remark 1) before or while sampling; however, this is a 2nd-order method, which violates the framework chosen by the authors (0-th order method). Although it is a common assumption to have access to the location of the modes in realistic settings (which turns out to be another challenge of sampling procedures), this requirement should be much more highlighted. Otherwise the title of the paper is misleading.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel zero-order sampling algorithm (ZOD-MC) when the target distribution is beyond the log-concavity and even isoperimetry. Different from first-order diffusion-based Monte Carlo proposed previously, this paper only requires the zero-order information. Besides, this paper shows the good performance of ZOD-MC from both theoretical and practical perspectives.\nThis paper's key step is to introduce rejection sampling to implement the score estimation in a reverse OU process. Specifically, the authors note that the envelope for the rejection sampling is a Gaussian-type distribution when the minimum negative log density of the target distribution is given as $V_*$.\n\nSTRENGTHS:\n- This is the first diffusion-based Monte Carlo method which only requires the bounded second moment and a relaxation of the commonly used gradient-Lipschitz condition. Such assumptions are even much weaker than the first-order diffusion-based Monte Carlo, e.g., RDMC and RSDMC.\n- Although zeroth-order sampling methods cannot circumvent the curse of dimensionality, this paper gets rid of the exponential dependence on error tolerance, i.e., $\\epsilon$. Combined with nearly no smoothness requirements in this paper, such complexity is acceptable.\n- This paper greatly simplifies the proof techniques used to estimate upper-bound score estimation errors, making them easy for readers to follow. Specifically, previous work introduces complicated concentration properties to provide the error bound. While this paper replace it with some monotonicity (Proposition B.2) used in stochastic localization.\n\nWEAKNESSES:\n- Some related work is missed in the authors’ survey, e.g., [1]. Theorem 7 of [1] provides a similar zeroth-order complexity ($\\exp(O(d)\\log \\epsilon^{-1})$) for achieving a minimal optimal error. I suggest the authors compare the theoretical results with [1].\n- The implementation of Algorithm 3 requires the minimum of $V$, which is denoted as $V_*$. How do we obtain this minimum? Since the function $V$ is highly irregular, calculating the minimum may also be difficult.\n- Although the experiments seem to be comprehensive, some details are not reported. For example, what is the meaning of oracle complexity? Does it include only zeroth-order oracle, first-order oracle, or both? If only counting the first-order oracle, is the comparison fair to first-order methods? What results are if the x-axis is set as the wall clock time? What are the hyper-parameters of ZOD-MC and baselines, and how do we choose these hyper-parameters? I hope these details will be included in the appendix.\n\n[1] Convergence Rates for Non-log-concave Sampling and Log-partition Estimation.\n\nQUESTIONS:\n- In Corollary 3.1, we can find the zeroth-order query complexity will depend on $\\epsilon_{\\mathrm{KL}}^{-d}$. But I would be curious to know if we can achieve a complexity whose order of $\\epsilon$ is independent with $d$?   What are the barriers to obtaining this complexity?\n\nLIMITATIONS:\nThis is a theoretical paper. It can hardly find potential negative societal impact in their work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of sampling from an unnormalized density by combining techniques developed from score-based generative modeling and non-log-concave sampling. Specifically, based on the Reverse Diffusion Monte Carlo (RDMC) framework proposed in [1], which is a meta-algorithm based on an oracle that estimates score function at any time, the authors proposed one way to implement such oracle via techniques developed for sampling non-log-concave distributions (Alternating Sampling Framework, where the Restricted Gaussian Oracle is implemented via Rejection Sampling) and arrived at an implementable and derivative-free version of RDMC.\n\nSTRENGTHS:\nThis is a technically solid paper with rigorous proof. Also, a complete set of numerical experiments are included to justify the main claims. Furthermore, this paper is the first work turning RDMC into an implementable sampler, which might be useful for sampling real-world distributions. Moreover, for the theory part of this paper, no strong assumption like log-concavity or isoperimetric inequality are made in the proof.\n\nWEAKNESSES:\n1. One possible drawback of the proposed sampler, just as the authors stated in the paper, is that its iteration complexity depends linearly on the data dimension $d$. Hence, it will probably be appealing only for low-dimensional distributions. Therefore, it might be meaningful to do a numerical investigation to see how the sampler proposed in this paper behaves on high dimensional distributions. \n2. For the sake of completeness, it might be necessary for the authors to include a section of related work on derivative-free methods for optimization and sampling from unnormalized densities. Some classical examples include the Ensemble Kalman Filter (EnKF) and Ensemble Kalman Inversion (EKI).\n\nQUESTIONS:\nThe reviewer's main concern is that when using rejection sampling to implement the Restricted Gaussian Oracle (RGO) in the Alternating Sampling Framework (ASF), one can only obtain the expected number of executions for rejection sampling (Proposition B.3). Therefore, it might be fair to compare the expected time complexity of this sampler with the exact time complexity of other samplers listed in Table 1?\n\nLIMITATIONS:\nIt seems to be the reviewer that the name \"Diffusion Monte Carlo\" and its abbreviation \"DMC\" are not good choices for naming the sampling methodology here. They happen to coincide with the one of the quantum monte carlo methods developed in computational quantum physics. Maybe one possible way to resolve such issue is to use the name \"Diffusion-Based Monte Carlo (DBMC)\" or the name \"Score-Based Monte Carlo (SBMC)\" instead. \n\nReferences:\n\n[1] Huang, X., Dong, H., Yifan, H. A. O., Ma, Y., & Zhang, T. (2023, October). Reverse diffusion monte carlo. In The Twelfth International Conference on Learning Representations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-25"}