Denoising MCMC for Accelerating Diffusion-Based Generative Models
Authors: Beomsu Kim
Jong Chul Ye
Venue: Submitted to ICLR 2023 (2023)
Keywords: Markov Chain Monte Carlo
Diffusion Models
Score-Based Models

Abstract: Diffusion models are powerful generative models that simulate the reverse of diffusion processes using score functions to synthesize data from noise. The sampling process of diffusion models can be interpreted as solving the reverse stochastic differential equation (SDE) or the ordinary differential equation (ODE) of the diffusion process, which often requires up to thousands of discretization steps to generate a single image. This has sparked a great interest in developing efficient integration techniques for reverse-S/ODEs. Here, we propose an orthogonal approach to accelerating score-based sampling: Denoising MCMC (DMCMC). DMCMC first uses MCMC to produce samples in the product space of data and variance (or diffusion time). Then, a reverse-S/ODE integrator is used to denoise the MCMC samples. Since MCMC traverses close to the data manifold, the computation cost of producing a clean sample for DMCMC is much less than that of producing a clean sample from noise. To verify the proposed concept, we show that Denoising Langevin Gibbs (DLG), an instance of DMCMC, successfully accelerates all six reverse-S/ODE integrators considered in this work on the tasks of CIFAR10 and CelebA-HQ-256 image generation. Notably, combined with integrators of Karras et al. (2022) and pre-trained score models of Song et al. (2021b), DLG achieves state-of-the-art results. In the limited number of score function evaluation (NFE) settings on CIFAR10, we have $3.86$ FID with $\approx 10$ NFE and $2.63$ FID with $\approx 20$ NFE. On CelebA-HQ-256, we have $6.99$ FID with $\approx 160$ NFE, which beats the current best record of Kim et al. (2022) among score-based models, $7.16$ FID with $4000$ NFE.

Under review as a conference paper at ICLR 2023
DENOISING MCMC FOR ACCELERATING DIFFUSION- BASED GENERATIVE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Diffusion models powerful generative models that simulate the reverse of dif- fusion processes using score functions to synthesize data from noise. The sam- pling process of diffusion models can be interpreted as solving the reverse stochas- tic differential equation (SDE) or the ordinary differential equation (ODE) of the diffusion process, which often requires up to thousands of discretization steps to generate single image. This has sparked a great interest in developing effi- cient integration techniques for reverse-S/ODEs. Here, we propose an orthogonal approach to accelerating score-based sampling: Denoising MCMC (DMCMC). DMCMC first uses MCMC to produce initialization points for reverse- S/ODE the product space of data and variance (or diffusion time). Then, reverse-S/ODE integrator is used to denoise the initialization points. Since MCMC traverses close the data manifold, the cost of producing clean sample for DMCMC is much less than that of producing a clean sample from noise. To verify the proposed con- cept, we show that Denoising Langevin Gibbs (DLG), instance of DMCMC, successfully accelerates all six reverse-S/ODE integrators considered in this work on the tasks of CIFARIO and CelebA-HQ- HQ-2556 image generation. Notably, com- bined with integrators Karras et al. (2022) and pre-trained score models of SSog et af.] (2021b), DLG achieves state-of ofate-of-the-ar results among score-based models. In the limited number of score function evaluation (NFE) settings on CIFAR 10, we have 3.86 FID with Rs 10 NFE and 2.63 FID with x 20 NFE. On CelebA-HQ- 256, we have 6.99 FID with s 160 NFE, which beats the current best record of Kim et al. (2022) among score-based models, 7.16 FID with 4000 NFE.
INTRODUCTION
Sampling from probability distribution given its score function, i.e., the gradient of the log-density, is an active area of research in machine learning. Its applications range far and wide, from Bayesian learning (Welling & Tch 2011 learning energy-based models (Song Kingmal 2021), synthe- sizing new high-quality data (Dhariwal & Nichol 2021), ), and so on. Typical examples of traditional score-based samplers Markov chain Monte Carlo (MCMC) methods such as Langevin dynamics [angevin 1908) and Hamiltonian Monte Carlo (Neal) 2011).
Recent developments in score matching with deep neural networks (DNNs) have made it possible to estimate scores of nigh-dimensional distributions such as those of natural images (Song & Ermon 2019) However, natural data distributions are often sharp and multi-modal, rendering naive appli- cation of traditional MCMC methods impractical. Specifically, MCMC methods tend to skip over or get stuck at local high-density modes, producing biased samples (Levy et al., 2018).
Diffusion models (Sohl-Dickstein et al. 2015 Ho et al., 2020: Song et al.) 2021a) depart from MCMC and use the concept diffusion, the process of gradually corrupting data into noise, to generate samples. (Song et al. (202lb) observed that for each diffusion process, there is reverse stochastic differential equation (SDE) and an ordinary differential equation (ODE). Hence, given a noise sample, integrating the reverse-/OODS produces data sample. Only time-dependent score function of the data during the diffusion process is required to simulate the reverse process.
This discovery generated great interest in finding better ways to integrate reverse- S/ODDe. For instance, [Song et al. (202Tb) uses black-box ODE solvers with adaptive stepsizes to accelerate sam- pling. Furthermore, multitude of recent works score-hased generative modeling focus improv-

---

Under review as a conference paper at ICLR 2023
MCMC Clain
Rov. S/ODE DMCMC
Density
Noise Samples
Rev. S/ODE from Noise
Data Space
Naise level
Prict Note Sumples
MCMC Chain
318
200e 200 Poe
Sumpling From Noise
Denoising MCMC
Figure 1: Top: conceptual illustration of VE diffusion model sampling process and DMCMC sampling process. VE diffusion models integrate the reverse-S/OD starting from maximum diffu- sion time maximum noise level. So, samples often noisy with small computation budget due large truncation error. DMCMC produces an MCMC chain which travels close to the image mani- fold (compare the noise level r). So, the MCMC samples be denoised produce high-quality data with relatively little computation. Bottom: Visualization of sampling processes without (left) and with (right) DMCMC on Cel2bA-HQ-256 under fixed computation budget.
ing reverse-S/ODE integrators (Jolicoeur-Martineauu et Zhang & Chen 2022).
2021
2022
Karras al. 2022
In this work, develop orthogonal approach accelerating score-based sampling. Specifically, we propose Denoising MCMC (DMCMC) which combines MCMC with reverse-S/ODE integra- tors. MCMC used generate initialization points ((x,, {n) } in the product space of data and variance exploding (VE) diffusion time t noise level a (see Fig. [ top panel). Since all modes are connected the product space, MCMC mixes well. Then, the reverse-Sie PreesseS/ODDE starting at from time f = f, to & = 0. Since MCMC explores high-density regions, the MCMC chain stays close to the data manifold, so in tends to be close to i.e., noise level tends to be small (see Fig. top and bottom panels). Thus, integrating the reverse-S/ODE from much faster than integrating the reverse-S/ODE from maximum time t T to
Diverse-S/DDE integrator solves
t tn to t 0 starting from noise. This leads to significant acceleration of the sampling process.
Our contributions can be summarized as follows.
We introduce the product space of data and diffusion time, and develop a novel acore-based sampling framework called Denoising MCMC on the product space. Our framework general, any MCMC, any VE process noise-conditional score function, and any reverse- S/ODE integrator can be used a plug-and-play manner.
We develop Denoising Langevin Gibbs (DLG), which an instance of Denoising MCMC that is simple implement and is scalable. The MCMC part of DLG alternates between data update step with Langevin dynamics and a noise level prediction step. so all that DLG requires is pre-trained noise-conditional score network and noise level classifier.
2

---

Under review as a conference paper at ICLR 2023
We verify the effectiveness of DLG by accelerating six reverse-S/ODE integrators. No- tably, combined with the integrators of Karras et al. (022). DLG achieves state-of-the-an results among score- baseebaaed models. On CIFARIO in the limited number of score function evaluation (NFE) setting, we obtain 3.86 FID with v 10 NFE and 2.63 FID with x 20 NFE. On CelebA-HQ-256, result with score- based models. The computation cost of evaluating a noise level classifier is negligible, so we obtain acceleration essentially for free.
have 6.99 FID with x 160 NFE, which is currently the best
BACKGROUND
2.1
DENOISING SCORE MATCHING
Given a distribution p(z). a noise level a, and perturbation kernel Pa ()) = Naxx ã,, T2
solving the denoising score matching objective (Vincent 2011)
min Ep(a) Ep. (al=)
[==) V Po x)
yields score model (x)) which approximates the score of Pa T)p( ( dē. Denoising score matching was then extended to train Noise Conditional Score Networks (NCSNs) s0(x, which approximate the score of data smoothed at general set of noise levels by solving
min Ex(e) pp(a) E,, (m|a) [I|s0(2, o) V log x)
where X(ar can be a discrete or a continuous distribution over (A max) (Song Ermon 2019
Song et al. 2021bb. We note f Par (a: x p)p(T) ( dē approaches p(ær) as + 0, since the perturbation kernel Pa Ø σ.) converges to the Dirac delta function centered at æ.
2.2
MARKOV CHAIN MONTE CARLO (MCMC)
Given an unnormalized version of p(z) or the score function V~ x log p(z), MCMC constructs Markov chain in the data space whose stationary distribution is p(x). An MCMC which uses the unnormalized density is the DettrpolissHasstine MCMC (Metropolis et al. 1953 Hastings 1970) that builds a Markov chain by sequentially accepting or rejecting proposal distribution samples les ac- cording density ratio. A popular score-based MCMC is Langevin dynamics (Langevin) 1908)
Langevin dynamics generates a Markov Chain {z,, 1os1 using the iteration
1n+1 = xn + (n/2) Vn logp(æ.) + /ij
where 6 ~ ~N I).
(x:n 1000
n= converges to p(x²) in distribution for an appropriate choice of r).
To sample from a joint distribution p(a:, y), may resort Gibbs sampling (Geman & Geman 1984). Given a current Markov chain state (I.,, yn). Gibbs sampling produces En++1 by sampling Trom p( yn) and y by sampling from p(y (n41)) The sampling steps may be replaced with MCMC. Hence, Gibbs sampling is useful when conditional distributions are amenable to MCMC.
Annealed MCMC. Despite their diversity, MCMC methods often have difficulty crossing low- density regions in high-dimen nigh-dimensional multimodal distributions. For Langevin dynamics, at a low- density region, the score function vanishes in Eq. (3). resulting in a meaningless diffusion. More- over, natural data often lies on fow-dimensional manifold. Thus, once Langevin dynamics leaves the data manifold, becomes impossible for Langevin dynamics to find its way back.
One way to remedy this problem is to use annealing, i.e., constructing sequence of increasingly smooth and wide distributions and running MCMC different levels of smoothness. As smoothness is increased, disjoint modes merge, so MCMC can cross over to other modes. Annealing has been used to empower various types of MCMC (Geyer & Thompson, 1995 Neal (000). In this work, shall refer to the collection of MCMC that annealing as annealed MCMC.
An instance of annealed MCMC is annealed Langevin dynamics (ALD) (Song & Ermon 2019).
For sequence of increasing noise levels {o,, | Langevin dynamics is sequentially executed with J Po. (x: T ()) dē in place of p(x) in Eq. B for i = N, - 1,..., 1. Since p(x) smoothed at large noise level has wide support and connected modes, ALD overcomes the pitfalls of vanilla Langevin dynamics. However, ALD has the drawback that thousands of iterations are required to produce single batch of samples.
3

---

Under review as a conference paper at ICLR 2023
2.3 DIFFUSION MODELS
Diffusion models and differential equations. Diffusion models opened up a new avenue towards fast sampling with score functions via SDEs and ODEs (Song et al. 202Tb). Suppose data dis- tributed in R4, Given a diffusion process of data sample To ~ p(æ)) into a sample from a simple prior noise distribution, the trajectory of data during diffusion can be described with an Itō SDE
dæ = f(z t) dt + g(t)) dw
(4)
for some drift coefficient f IR x [0. T] + Rd, diffusion coefficient g [0, + R, and Brownian motion w. Here, T' is the diffusion termination time. With initial condition =(0) = z0, integrating Eq. (4) from time = 0 to f T produces a sample from the prior distribution.
For each diffusion SDE, there exists corresponding reverse-SDE:
dæ = [f (()) (r)) dt + g(t) dw
(5)
where (a(x:) is the density of (t) evolving according to Eq. (4 and t0 is Brownian motion if time
flows from 1. = T to L = 0. Given a sample æy from the prior distribution, integrating Eq. (5) with initial condition az (T) = øt from 1 T to 1 results in sample from p(æ)) Moreover, each
reverse-SDE, there exists corresponding deterministic reverse-ODE
dx (f (1/2) v log px(]]] dt
which also can be integrated from / = T to ( = 0 to produce samples from p(æ)).
Diffusion models generate data by simulating the reverse of the diffusion process, i.e., by solving the SSverse-SOODE of the diffusion process. Initial works on diffusion models (Sohl-Dickstein et al.,
2013 Ho et al. 2020) used computationally expensive ancestral sampling to solve the reverse dif- ferential equations. Later works discovered that using adaptive numerical integrators to solve the reverse-S/ODE could accelerate the sampling process. This led to great attention on developing bet-
ter S/ver integrators (Jolicoeur- Martineau et al. 2021: Song et al., 20216 Lu et al.) [222]] Karras et al. 2022 Zhang & Chen 2022). Our work is orthogonal to such works as focus on finding good initialization points for integration via MCMC. Hence, better integration technique directly translates to even better generative performance when plugged into Denoising MCMC.
Variance exploding (VE) diffusion model. A VE diffusion model considers the diffusion process
dx (1)) dw. dt
where (t) increases from Jmin = =0) to o mmax = (T). Distribution of (t) evolves as
(x)) Po (2) .)p(ā) dē
(8)
so if min is sufficiently small, (2) ss p(x), and if max is sufficiently large, so variance explodes, pr (x) Re (x 0, I). If we have a score model sx, o) trained with Eq. (2). V log Pr (x) se so(x, (t)). It follows that with Er ~ N(x 0, 0. I), we may integrate the Reverse-S/ODE corresponding to Eq. 77) with xTT) xT from T to O using score model to generate data. In the next section, we bridge MCMC and Reverse-S/ODE integrators with VE diffusion to form a novel sampling framework that improves both MCMC and diffusion models.
DENOISING MARKOV CHAIN MONTE CARLO (DMCMC)
We now develop a general framework called Denoising MCMC (DMCMC) which combines MCMC with Peverse-S/ODD integrators. We denote the data space as X' C Rd and the noise level space as (omin: m- The construction of DMCMC is comprised of two steps. In the first step, we build MCMC to generate initialization points in the product space X' x S, i.e., X' augmented by the smoothness parameter a. Since (t)) a monotone increasing function, this is equivalent to augmenting the data space with diffusion time t. In the second step, we incorporate denoising steps, where we denoise the generated initialization points via reverse- SSODE integrators. Note that (PC) method (Song al. (021b) also uses MCMC, PC uses MCMC to
while the predictor-correcct improve the denoising process. Thus, DMCMC and PC contribute to orthogonal aspects of diffusion sampling. More discussion the differences between DMCMC and PC is in Appendix D
4

---

Under review as a conference paper at ICLR 2023
3.1 CONSTRUCTION STEP 1: MCMC ON THE PRODUCT SPACE X' X
Suppose p(æ) is a high-dimensional multimodal distribution, supported on a low-dimensional man- ifold. If the modes are separated by wide low-density regions, MCMC can have difficulty moving between the modes. Indeed, convergence time for such distributions can grow exponential in di- mension d (Roberts & Rosenthal] 200I). Intuitively, for MCMC move between disjoint modes, the Markov Chain would have to step off the data manifold. However, once MCMC leaves the data manifold, the density or the score vanishes. Then, most random directions produced by the pro- posal distribution do not point to the manifold. Thus, MCMC gets lost in the ambient space, whose volume grows exponentially in d.
Annealing via Gaussian smoothing, used in both ALD and VE diffusion, circumvents this problem. As p(æ) smoothed with perturbation kernel po @) increasing a, the modes of p(æ) grow wider and start to connect. Thus, MCMC can easily transition between modes. However, running MCMC in the manner of ALD is inefficient since we do not know how many itcrations within each noise level is sufficient. To address this problem, we propose augment X' with the smoothness scale a and run MCMC in the product space X' x such that MCMC automatically controls the value of a. Below, we formally describe MCMC X' x S.
Let us define the o-conditional distribution
p( o) = Po (x T)p(T) dē.
(9)
We also define a prior p(o) on S. Then by the Bayes' Rule,
p(æ, = p(x a) -(o).
(10)
In Appendix E.2
discuss the effect of varying the prior. MCMC with p(æ, will produce samples (( in X' x S such that
On p(a),
In p(x o..).
(11)
Hence, on >> Tmin Fn will be a noisy sample, i.e. a sample corrupted with Gaussian noise of variance o and if dr R mmin E, will resemble a sample from p(æ).
Since our goal is to generate samples from p(æ), naively, we can keep samples (æn, an) with on R mmin and discard other samples. However, this could lead to large waste of computation resources. In the next section, we incorporate reverse- S/ODE integrators to avert this problem.
3.2 CONSTRUCTION STEP 2: INCORPORATING DENOISING STEPS
Let us recall that integrating the reverse-S/ODE for the VE diffusion SDE Eq. 7 from time / = T to = sends samples from pr pr to samples from Po(x) se p(x). In general. integrating the reverse-SDE or ODE from time t t2 to t for f1 < t2 sends samples from Pt, (x) to samples from pt, (x) (Song et al. 2021b). We use this fact to denoise MCMC samples from ((x,o).
Suppose we given a sample (In, on) $(z, o). With f, = a- (on), Eq. (11 tells us
Fr ~ Pt, (x)
(12)
so integrating the reverse-S/ODE with initial condition att,,) = H, from t = tn to t = 0 produces sample from (=( x p(z). Here, we note that any reverse-S/ODY solver may be used carry out the integration. In Appendix F.2 we show the necessity of the denoising step.
Let us briefly explain how DMCMC accelerates sampling. Given an MCMC chain {(rn, an)) in A' S and a prior ((or) which places high mass near min. MCMC traverses high probability regions, so nn & mmxx for most n, i.c., fn < T for most n. This also means the sequence {æ,) generally stays close to the data manifold. So, the average length of integration intervals (0, (n) will tend to be much shorter than T. Thus, integrating the reverse-S/ODE over (0, f,) to denoise H, is much faster than integrating the reverse-S/ODE over (0. T), i.e., standard diffusion sampling. This idea is illustrated in Figure and more detailed explanation given Appendix E.T
DENOISING LANGEVIN GIBBS (DLG)
In Section 3 described an abstract framework, DMCMC, for accelerating ccore-based sampling by combining MCMC and SwverseSSODDE integrators. We now develop a concrete instance of DM- CMC. As the second construction step of DMCMC is simple, only describe the first step.
5

---

Under review as a conference paper at ICLR 2023
Naively, we could extend denoising score matching Eq. estimate the score $0 o) R
R - R2 x IR of p(z, 0) and apply Langevin dynamics in the first step of DMCMC. But, this would
prevent us from using pre-trained score models, as we would have to solve (for some small v > 0)
min Ee (000 pp(e.)))))))))))))))))))) [ - -l:d; - Ed+1) E e/v2
(13)
6
which differs from Eq. (2). Gibbs sampling provides a simple path around this problem. Let us recall that given a previous MCMC iterate (2n, Gibbs sampling proceeds by alternating between an update step xn+1 ~ p(x σn) and a a update step 0++1 ~ -p(rr Tn++1). Below, we describe our score-based sampling algorithm, Denoising Langevin Gibbs (DLG) (pseudocode in Appendix C
Updating æ. Suppose we are given an MCMC iterate (In: on)) and score model sø (ax, σ) from Eq. We generate xn++1 by Langevin dynamics step on pia: on). Specifically, by Eq. (
V log p( (n) x se (zz, Fn)
(14)
and so an Langevin dynamics update on æ, according to Eq. 3 is
En+1 x,, + (1/22 (a nn) + V for <~ No, I). Here, we call 1y the step size.
(15)
Updating a. We now have In+1 and need to sample xx+1 ~ pia In+1). To this end, we first train a DNN noise level classifier (o x) to approximate piar x) by solving max o [log ( x)].
(16)
Specifically, we discretize [mmin: max into M levels T1 Fmin < T2 < TM = 0 max. Given Tm where 1 < m S. M,, m serves as the label and clean training data corrupted by Gaussian noise of variance 7m serves as the classifier input. The classifier is trained predict m by minimizing the cross entropy loss. Having trained noise level classifier, we sample at+1 by drawing an index m according to the classifier output probability for 2n+1 and setting xx+1 = Tm- In practice, using the index of largest probability worked fine. We denote this process xx+1 Lo In+1). In Appendix E. we verify whether DLG with the approximated conditional works as intended.
4.1 PRACTICAL CONSIDERATIONS
Computation cost of a prediction. We found that using shallow neural networks for the noise clas- sifier ds was sufficient to accelerate sampling. Concretely, using a neural net with four convolution layers and one fully connected layer the classifier, one evaluation of do was around 100 ~ 1000 times faster than one evaluation of the score model sg. So, when comparing sampling methods, we only count the number of score function evaluations (NFE). We also note that the training time ds was negligible compared to the training time of 80- For instance, on CelebA-HQ-256 training Ze with the aforementioned architecture for 100 epochs took around 15 minutes
RTX 2080 Ti.
Starting points for DLG. Theoretically, an MCMC chain {(I,,, on) h will converge to p(x, or) regardless of the starting point (æo, oo). However, theory shows that setting starting points close to the stationary distribution, i.e., using "warm start", can significantly accelerate convergence of the Markov chain (Dalalyan) 2017: Dwivedi al. 2019). Thus, we set 2g by generating a clean data sample with a reverse-S/ooe S/ODE solver starting from prior noise, adding some Gaussian noise to the clean data sample, and running Gibbs sampling for few iterations. Pseudocode is shown in Appendix C The NFE involved in generating 2o is included in the final per-sample average NFE computation for DLG when comparing methods Section 5 But, we note that this cost vanishes in the limit of infinite sample size.
Reducing autocorrelation. Autocorrelation MCMC chains, i.e., correlation between consecutive samples in the MCMC chain, could reduce the sample diversity of MCMC. A typical technique to reduce autocorrelation is to use every nekkp--th -th samples of the MCMC chain for some Taakip > 1. For DMCMC, this means we denoise every nakip -t sample. So. if we 7den NFE to denoise MCMC samples, the average NFE for generating a single sample is around N2,kip + R-den-
Choosing iterates to apply denoising. The MCMC chain can be partitioned into blocks which consist of nskip consecutive samples. Using every nskiigthh sample of the MCMC chain corresponds to denoising the last iterate of each block. Instead, to further shorten the length of integration, within each block, we apply denoising to the sample of minimum noise scale o.
Choice of prior p(a). We use (()) x 1/a to drive the MCMC chain towards small values of a.

---

Under review as a conference paper at ICLR 2023
Mode Coverage
Sample Class Ratio
Class Seq, Autocorrelation
1000
1.0
1.0
Graund Truth
500
Langevin
DLG
0.5
0.5
0.0
0.0
200
400
1000
2
5
60
60
100
Chain Length
Class
1ag
Figure 2: Ablation study of a update step in DLG.
KAR1
Probability Flow (log scale)
RK45
FID
WD DLG
500
5
With DLG
200
6-
100
4.15
50
3
2.44
10
5.25
9.51
400
800
175
KAR2
Reverse Diffusion (log scale)
Euler-Maruyama (log scale)
F10
F1D
R10
7
500
500
200
100
100
5-
50
50
20
20
3
3.04
10
2:19
7.27
2
NFE-D
200
400
500
800
1000
200
400
600
800
1000
0
200
400
600
800
1000
Figure 3: Sampling acceleration of DI.G on CIFAR10. FID of notable points are written in the corresponding color. Top row: deterministic integrators. Bottom row: stochastic integrators.
EXPERIMENTS
5.1 MIXING OF DMCMC CHAINS
Here we provide experimental proof that DMCMC capable of visiting diverse modes as a con- sequence of running MCMC in the product space X' x S. To this end, we compare DLG with and without a updates. DLG without a updates is just Langevin dynamics fixed a, so we run fifty Langevin dynamics chains and fifty DLG chains on mixture of Gaussians (MoG) with 1k modes at CIFARIÓ images. All chains are initialized at a single mode, For each method, we compute the mode coverage of the samples, the class distribution of the samples, and the autocorrelation of sam- ple image class sequence. Since the noise conditional score function can be calculated analytically for MoGs, this setting decouples sampler performance from score model approximation error.
Figure 2 shows the results. In the left panel, we observe that Langevin dynamics is unable to escape the initial mode. Increasing the step size 7) of Langevin dynamics caused the chain diverge. On the other hand, DLG successfully captures all modes of the distribution. DLG samples cover all 1 modes at chain length 432. Middle panel provides evidence that DLG samples correctly reflect the statistics of the data distribution. Finally, the right panel indicates that the DLG chain moves freely between classes, i.c., distant modes. These observations validate our claim that DLG mixes well.
5.2 ACCELERATING IMAGE GENERATION WITH SCORE NETWORKS
We compare six integrators with and without DLG on CIFAR10 and CelebA- image gener- ation. The deterministic integrators are: the deterministic integrator of Karras et al. (2022) (KARI), the probability flow integrator of (ong al. (0221b), and the RK45 solver. The stochastic integra- tors are: the stochastic integrator of Karras et al. (2022) (KAR2), the reverse diffusion integrator of Song et al. (2021b). and the Euler-Maruyana method. We use the Fréchet Inception Distance (FID) (Heusel et al. 2)17) to measure sample quality. For CIFARIO, we generate 50k samples, and for CelebA-HQ-256, we generate 10k samples. We use pre-trained score models of Song et al. (202lb).
7

---

Under review as a conference paper at ICLR 2023
Deterministic Integrators
Stochastic Integrators
FID
FID
400
KAR1
427
250
KAR2
240
Prob. Flow
Reverse Diffusion
300
RK45 tol=1e-5
150
Euler-Maruyama
100
100
126
50
23
0
15.2
29
0
19.02
NFE-0
100
150
200
250
300
50
100
150
200
250
300
350
Figure 4: Sampling acceleration of DLG on CelebA-HQ-256. A dot indicates an integrator without DLG, and a cross of the same color indicates corresponding integrator combined with DLG. Dotted lines indicate performance improvement due to DLG.
420)
DIG UNFE 150)
M0R3
Froh. Flow
R045
XO92
Rac DEr
E.lentarr.
Figure 5: Non-cherry-picked samples on CelebA-HQ-256 using the settings for Fig. shows samples for an integrator without (left col.) and with (right col.) DLG.
Each row
CIFARIO. In Figure 3 we make two important observations. First, DLG successfully acceler- ates all six integrators by non-trivial margin. In particular, an integrator without DLG al- ready performs well, the integrator combined with DLG outperforms other integrators com- bined with DI.G. For instance, compare the sults for KARI with those of other determinis- tic integrators. Second, DLG improves the per- formance lower bound for some deterministic integrators. While the performance of KARI and RK45 saturates at around 4 FID, KAR 1 and RK45 with DLG achieve around 2.4
DPM--abees22
5.26
6,03
(-4
(-0
(VE)
20189 (-11 NFE)
(6.09
16,41
(VP)
9.70
297
(VF)
14.12 NFF)
(-46 NFF)
4.1
(VE)
(-0. NEE)
(-45 NFLLL
NFB)
Comparison of fast samplers on CI- FAR 10 FID. Number in parenthesis indicates ex- tra or less NFE used. Best numbers are bolded.
Table 1:
CelebA-HQ-256. Figure 4 shows the results on CelebA CelebA-HQ-256, We observe that DLG improves computational efficiency and sample quality simultaneously. Indeed, in Figure 5 we observe re- markable improvements sample quality despite using fewer NFE. This demonstrates the scalabil- ity of DLG to generating high-resolution images. We also note that we did not perform exhaustive for HQ-2b, so fine-tuning could yield better results.
search of DLG hhyper-paramen
Achieving SOTA. DLG combined with KARI sets a new SOTA record for CIFARI0 O in the limited number NFE setting: 3.86 FID with 10.11 NFE and 2.63 FID with 20.11 NFE which beats the results of Zhang & Chen (222), 4.17 FID with 10 NFE and 2.86 FID with 20 NFE. DLG combined with KAR2 sets a new record on CelebA-HQ-25 HQ-256 among score- score-based models: 6.99 FID with 158.96 NFE which beats the current best result of Kim et al. (2022). 7.16 FID with 4000 NFE.
8

---

Under review as a conference paper at ICLR 2023
Fixed NFE 25
Fixed (- 0.1
Optimal (Den. NFE) / (NFE)
110
F10
F10
n=
0.01
4.5
NFE - 10
0.0.
=
0.1
==1
NTE - 50
3.5
4=
3.5
NFE
1.0
4 10.0
3-
2.5
2.5
2
2.0
0.2
0.4
0.6
0.8
0.1
0.2
0.3
0.4
055
0.6
0.7
0.0
0.9
10
20
30
40
50
(Den. /FFE/(NFE
NFE1
NFE
Figure 6: Ablation study of DLG with KARI. Dots indicate the points of lowest FID.
5.3 HYPER-PARAMETER ABLATION STUDY
Given integrator, DLG is determined by total NFE per sample n, NFE spent denoising samples Oddens and Langevin dynamics step size 1). We fix the integrator to be KAR1 and observe the effect of each component on CIFARIO image generation. We observed similar trends with other samplers.
VS. Rdden/t 'n. In the left panel of Figure 6 we fix NFE and vary v and Aden/n. Vlden governs individual sample quality, and nakip optimal FID is achieved when den n has intermediate values, not extreme values near 0 or 1. Also, lower den n is needed attain optimality for lower 7). This is because lower 17 means the MCMC chain travels closer to the image manifold at the cost of slower mixing.
dden governs sample diversity. Thus,
observe
NFE vs. Rdden/ / /n. the middle panel of Figure we fix 1) and vary NFE and ndden We observe two trends. First, in the small NFE regime, where 10 < NFE < 50, it is beneficial to decrease Wdeennn n as NFE increases. Second, in the large NFE regime, where NFE > 50, is beneficial to increase Tdden/ n as NFE increases. This is because if Rskip is sufficiently large, MCMC chain starts producing essentially independent samples, so increasing Tlkkip further provides no gain.
Also, as we increase NFE, the set of nnden/n which provides near-optimal performance becomes larger. Moreover, we see that most of the time, optimal FID is achieved when Tl > 0.5, i.e. when Vdden > Niskip- So, reasonable strategy for choosing Tdden given 7j and NFE budget is to find smallest Rakip which produces visually distinct samples, and then allocate Rdd = n Nskipp
NFE. In the right panel of Figure 6 choose optimal (in terms of FID) ndde/ /n for each combination of r) and NFE. We see choosing overly small or large 7) leads to performance degrada- tion. If n is within certain range, obtain similarly good performance. In the case of CIFARIO, we found it reasonable to set 1 E (0.05,
To choose for data of general dimension d, we define a value k := n v called displacement per dimension. If we see Eq. (14), Gaussian noise of zero mean and variance 7) added the sample at each update step. Gaussian annulus theorem tells us that a high-dimensional Gaussian noise with zero mean and variance has Euclidean norm approximately n/ d. So, the average displacement of the sample per dimension by the random noise is around // Vd. Since is dimension-indeeenndee independent value, given and d, we can set vdx. On CIFARIO 10, we have 1) E [0.05, 1.0). which translates to x E [0.0009, 0.018). This means, on CelebA-HQ-2556, we can choose E [0.4, 8.0). If the sampler was inefficient, chose smaller ε to off diversity for sample quality.
6 CONCLUSION
In this work, proposed DMCMC which combines MCMC with reverse-S/ODE integrators. This has led to improvements for both MCMC and diffusion models. For MCMC, DMCMC allows Markov chains visit disjoint modes. For diffusion models, DMCMC accelerates sampling by reducing the average integration interval length reverse-S/ODE. We developed a practical instance of DMCMC called DLG, and demonstrated the practicality and scalability of DLG through various experiments. In particular, DLG achieved state-of-the-art results on CIFAR 10 and CelebA-HQ-256 among score-based models. Overall, our work opens up an orthogonal approach accelerating score-based sampling. We leave exploration of other kinds of MCMC or diffusion process such as VP diffusion (see Appendix E. DMCMC as future work.

---

Under review as a conference paper at ICLR 2023
7 REPRODUCIBILITY STATEMENT
We provide an anonymous Google drive link to a zip file containing code and noise clas- sifier checkpoints for main experiments. https://drive..ooglle.. cttee/l//ee/eeeee//eeeeeeeeeee wtt.eeeeeeeeeiiiiooooooooooooo//
1ink We have also added pseudocode of DMCMC in Appendix C and hapee-parameters described Appendix
REFERENCES
Xiang Cheng and Peter Bartlett. Convergence of Langevin MCMC in KL-divergence. In Machine Learning Research, Volume 83: Algorithmic Learning Theory, 2018.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. In NewrlPS, 2022.
Adam D. Cobb, Atilm Güney Baydin, Andrew Markham, and Stephen J. Roberts. Introducing an explicit symplectic integration scheme for riemannian hamiltonian monte carlo. arxiv preprint 1510:669100223 2019.
Arnak S. Dalalyan. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society, 79(3):651-676, 2017.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In NeurlPS, 2021.
Raaz Dwivedi, Yuansi Chen, Martin J. Wainnwight, and Bin Yu. Log-concave sampling: Metropolis- Hastings algorithms are fast. Journal of Machine Learning Research, 20: 14,1142,4 2019.
Stuart Geman and Donald Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions Patter Analysis and Machine Intelligence, 6(6): 721-741, 1984.
Charles J. Geyer and Elizabeth A. Thompson. Annealing Markov chain Monte Carlo with applica- tions to ancestral inferences. Journal of the American Statistical Association, 90(431):909-999 920, 1995.
Wilfred K. Hastings. Monte Carlo sampling methods using Markov chains and their application.
Biometrika, 57:97-109, 1970.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to local Nash equilibrium. In NIPS, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.
Alexia Sircoeur-Martinnna Ke Li, Rémi Piché-Taillefer, Tal Kachman, and loannis Mitliagkas. Gotta go fast when generating data with score-based models. arxiv preprint arXiv:2105. 14080, 2021.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In ICLR, 2018.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. arxiv preprint Nii::2266000366 2022.
Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang. and Il-Chul Moon. Soft truncation: A universal truncation technique of score-based diffusion for high precision score estimation. In ICML, 2022.
Diedrik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. 2015.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
10

---

Under review as a conference paper at ICLR 2023
Paul Langevin. On the theory of Brownian motion. 1908.
Daniel Levy, Matthew D. Hoffman, and Jascha Sohl-Dickstein. Generalizing Hamiltonian Monte
Carlo with neural networks, In ICLR, 2018.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ODE solver for diffusion probabilistic sampling in around 10 steps. arxiv preprint arXiv:2206.0099 2022.
Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Ed- ward Teller. Equations of state calculations by fast computing machines. Chem. Phys., 21(6): 1087-1092, 1953.
Radford M. Neal. Annealed importance sampling. Statistics and Compuling, 11:125-139, 2001.
Radford M. Neal. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 2(1 ():2) 2011. 2011.
Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021.
Gareth O. Roberts and Jeffrey S. Rosenthal. Optimal scaling for various Metropolis-Hasting algo- rithms. Statistical Science, 16(4):351-367, 2001.
Robin San Roman, Eliya Nachmani, and Lior Wolf. Noise estimation for generative diffusion mod- cls. arxiv preprint arXiv:2104.02600 2021.
Jascha Sohi-Dickstein, Eric A. Weiss, Nicu Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learing using nonequilibrium thermodynamics. In ICML, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NeurlPS, 2019.
Yang Song and Diederik P. Kingma. How to train your energy-based models. arxiv preprint arX:v22101.032888 2021.
Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021b.
Josef Stoer and Roland Bulisch. Introduction to Numerical Analysis, volume 12. ence+Business Media New York, 2002.
Springer Sci-
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Compu- tation,
2011.
Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient Langevin dynamics. 2011.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. arxiv preprint arXiv:2204.13003 2022.
11

---

Under review as a conference paper at ICLR 2023
A DETAILED EXPERIMENT SETTINGS
Device. We use an RTX 2080 Ti or two Quadro RTX 6000 depending on the required VRAM.
Codes. For probability flow, RK45, reverse diffusion, and Euler-Maruyama integrators, modify the code provided by Song et al. (2021b) in the GitHub repository https: / / hub. com/ yang- sctssccoreeeeeeeee_eeeee pateee_ptrcchhhhh For KARI and KAR2, since Karras al. (2022) did release their implementation of the samplers, we used our implementation based on their paper. For evaluation, we use the FID implementation provided in the GitHub repository http:: /ighhb.. com ccmm/mseitzerr/yyeeeee
fid
Datasets. We use the CIFAR10 dataset (Krizhessky 2009) and the CelebA-HQ-256 dataset Karras et al. 2018).
Data processing. All data are normalized into the range [0, 1). Following Song (2021b), for all methods, a denoising step using Tweedie's denoising formula is applied at the end of the sampling process.
Noise predictor network Ts- The noise predictor network qe has four convolution layers followed by a fully connected layer. The convolution layers have channels 32, 64, 128, 256, and (amin max) discretized into 1k points miin (0max/mmmmiii for t spaced evenly on [0, 1]. On CIFARIO, qe is trained for 200 epochs, and on CelebA-HQ-2566, do trained for 100 epochs. We use Adam optimizer (Kingma & Ba 2015) with learning rate 0.001.
Mixture of Gaussians. MoG has 1k modes at randomly sampled CIFARIO training set images. For Langevin dynamics, dynamics chain to diverge. For DLG, we use step size = 1.0, Rakip = 1, Uden = 20, and the initialized single mode test
step size 7 0.0001. Using larger step size caused the Langevin
reverse diffusion integrator. For each method, all chains mixing capabilities in the worst-case scenario.
CIFARIO image generation. We pre-trained NCSN+ (cont.) score model provided Song et al. 2021b). For the baseline methods, we use the recommended settings. For DLG, the chain was
initialized by generating samples with the deterministic integrator Karras al. (2022) using 37
NFE, adding Gaussian noise of variance 0.25, and running 20 iterations of
Table
lists the paramm-rrsmeterss for DLG and the corresponding FID used to produce Figure
KARI
Probability Flow
RK45
Rdden
Dykip
FID
feden
Rgkip
FID
Rijon
P2kkip
FID
9
1
0.5
3.86
0.01
153.48
59.98
10
0.5
2.45
17
3
0.5
2.63
24
1
0.01
35.58
83.74
10
0.5
2.58
27
24
0.5
2.44
49
1
0.01
15.75
128.99
10
0.5
2.55
10
0.01
6.8
10
0.05
5.25
KAR2
Reverse Diffusion
Euler-Maruyama
Pidem
Dakkip
1)
FID
Aden
Raksp
?
FID
Relen
10akip
0.5
6.99
0.01
300.18
9
306.98
23
94.98
24
31
20
0.5
2.64
49
1
0.01
22.1
49
0.02
12.73
51
50
0.5
2.19
90
10
0.01
8.61
90
10
0.05
4.79
190
10
0.1
4.45
190
10
0.1
2.69
30
1.0
2.64
370
1.0
2.22
30
1.0
2.14
Table 2: DLG hyper-parameter and FID for integrators Figure
12

---

Under review as a conference paper at ICLR 2023
CelebA-HQ-256 image generation. We use pre-trained NCSN++ (cont.) score model provided by Song et al. (2021b). For the baseline methods, we use the recommended settings. For DLG, the chain initialiizd by generating samples with the stochastic integrator of Karras et al. (2022) using 37 NFE, adding Gaussian noise of variance 0.25, and running 70 iterations of Langevin- Gibbss. Table 3 lists the hyper-parametent for DLG and the corresponding FID used to produce Figure
KARI
Probability Flow
RK45
Tlden
Tlkkip
FID
Makip
FTD
Makipp
FID
17
0.8
14.11
140
5
0.25
28.37
125.95
0.8
15.29
KAR2
Reverse Diffusion
Euler-Maruyama
Tidee
Bakip
FID
Pakip
"
FID
Ridem.
Dakkip
9
FID
37
10
0.8
8.78
100
25
0.1
26.43
100
0.8
23.19
Table 3: DLG hyper-parameters and FID for deterministic samplers in Figure 4
Achieving SOTA. On CIFARIO, use KARI settings of Table
On CelcbA-HQ-256, we use
ddden 131, Rakip 27, 4.0, which achieves 6.99 FID.
B ADDITIONAL SAMPLES
B.1 DLG CHAIN VISUALIZATION
Figures 7 and each show a DLG chain on CIFARIO and CelebA-HQ-256, respectively. The chain progresses from left to right, from right end to left end of row below. On CIFARIO, we can see that the chain visits diverse classes. On CelebA-HQ-226, we can see that the chain transitions between diverse attributes such as gender, hair color, skin color, glasses, facial expression, posture, etc.
Figure Visualization of DLG chain on CIFARIO.
A
Figure 8: Visualization of a DLG chain CelebA-HQ-226.
B.2 ADDITIONAL UNCONDITIONAL SAMPLES
In Figures and 10 we show additional samples for CIFAR1O without and with DLG. Figures
and 12 show additional samples for CelehA-HQ-256 without and with DLG.
13

---

Under review as a conference paper at ICLR 2023
KAR1 / NFE=51/FID=41
KAR1+DLG NFE 11 FID-2:57
Probabiiity Flow / NFF-1000 / FID-9.51
Probabi ity Flow-DI G NFF-100.11 / FFDD-6.8
RK45 / 180.41 /ID--438
RK45+DLG NFE-66 39 FID-2445
Figure 9: Additional non-cherry-pice picked samples for deterministic integrators on CIFAR10 without (left col.) and with (right col.) DLG.
14

---

Under review as a conference paper at ICLR 2023
KAR2/ / NFE- 1001 / FID-3.04
KARZ+DLG / 1PTE-101.11 FIDD22119
Reverse Diffusion/ / NFE-200/ 14 35
Reverse Diffusion DLG / NNEE200011 11 F1D-4.45
Euler Maruyama/ / /NFE-100/ F1D-21.4
Euler Maruyama DLG NFE-1000 FID-4.79
Figure 10: Additional non-cherry-picker samples for stochastic integrators on CIFAR10 without (left col.) and with (right col.) DLG.
15

---

Under review as a conference paper at ICLR 2023
KAR1 / NNF-3399 FID
KARI+DLG / 21.97 / FI1111.11
Probability Flow NFE-400 / 427.33
Probabiity #lo NFFE-445.77 / FID328..3
RK45 / NNFE-378.49 FID-126.46
RK45+0LG / AFE-126.92 92.92 15.29
'N
Figure 11: Additional non-chery--iiee samples for deterministic integrators on CelcbA-HQ-256 without (left col.) and with (right col.) DLG.
16

---

Under review as a conference paper at ICLR 2023
KAR2/ / NFE-399 / FID-19.02
KAR2+ DLG / NFE-47 / F.D-8.78
Reverse D Diffusion/ NFE=400 / 7ID-240.79
Reverse Diffusion+DLG NFE-128 128 72 FID FID-26.43
Culer Meruyama NFC=400 / 32.732.73
Culer Maruyame+DLG / NTC=104.97 ND=23319
Figure 12: Additional non-cherry-pice picked samples for stochastic integrators on CelcbA-HQ-256
without (left col.) and with (right col.) DLG.
17

---

Under review as a conference paper at ICLR 2023
C PSEUDOCODES
Let us denote reverse-S/ODE, integrator as (800 æz, 0, n). Given point æ at noise level a, score function $8, and NFE budget n, returns the result of integrating reverse-S/ODE from to min starting from σ. That is, π returns a sample from (2))..
Algorithm MCMC Starting Point Generation
1: Input: Integration NFE budget 21, number of Langevin- Gibbs steps n2, Langevin step size
2: Sample 36 ((0,
3: to < T(89; 0o mmax, 71))
4: 2o f Zg + 0.5 where ~ N(O.I)
5: + 0..
6: for 1,2,
do
7: Zo <- 2g + 0.5.1 so σn) + /n where e~ (),
8: og 90(a mo)
9: end for
10: Return (Ro 00)
Algorithm Denoising Langevin Gibbs
1: Input: Rilen, Rakips MCMC starting point (æo, oo), Langevin step size v,, number of samples
needed N
2: (ăr +- (m 00)
3: for k= 1,2...... do
4: Initialize minimum noise level tracking variables (á, Ô) + (0, o)
5: for 1.2...... Riskip do
6: 2 - x 0.5 so σ) + /n- where e 1O.
7:
a ~ g0(ar ))
8:
o 6 then
9:
(#, + (#, 0)
10:
end if
11:
end for
12: Tt <- T (s0: &t &, Mede))
13:
end for
Return {
D MORE RELATED WORKS
D.1 COMPARISON OF DMCMC WITH THE PREDICTOR-COBE CORRECTOR SAMPLER
As both DMCMC and the predictor-corrector (PC) algorithm (Song al. 202 lb) rely on MCMC, one may ask whether DMCMC is a straightforward extension of PC, However, we assure the reader that DMCMC is not a trivial generalization of PC.
On a high level, the main message of our paper is that we can significantly improve diffusion sam- pling by choosing better initial points for the denoising process. This claim was verified through extensive experiments. On the other hand, the focus of the corrector step of PC is on improving the denoising process by refining the predictor integrator steps. So. DMCMC and PC contribute to orthogonal aspects of diffusion sampling. Indeed, the improvements offered by MCMC corrector step are rather marginal (see Table in (Song et al.) (22Tb)) compared to the improvements offered by DMCMC (see Figure in paper). On low level, distinguish DMCMC from PC three levels: theoretical, implementation- wise, and empirical.
Theoretically, MCMC in DMCMC and MCMC in PC reduce truncation error (error arising from numerically integrating reverse- reverse-S/ODE) in entirely distinct ways. We observe that diffusion model sampling can be broken down into two parts: (part 1) generating integrating the reverse-S/ODE starting from the initialization point to generate clean data. MCMC in PC aims improve part 2 whereas MCMC in DMCMC aims to improve part 1.
initialization point, and (part 2)
18

---

Under review as a conference paper at ICLR 2023
In PC, MCMC is used corrector. That is, MCMC corrects the distributions of intermediate points during integration of the reverse- SeorseeSOODEEE (part 2). That is why PC sampling proceeds by alternating between taking predictor step and running Langevin dynamics at fixed noise level.
In DMCMC, MCMC generates initialization points for reverse-S/ODE (part 1) that lie near the image manifold. This is possible because MCMC runs in the augmented space X' S by adaptively updating the noise level. This leads to reduced truncation error, or equivalently, acceleration, as is easier for reverse-S/ODE integrators generate clean data from points near the data manifold than from prior noise distribution samples (e.g.. Gaussian noise). For a more detailed explanation of the acceleration mechanism of DMCMC, we refer the reader to Appendix E.
DMCMC does not resemble PC even from the perspective of implementation. MCMC in PC runs during reverse-S/ODE integration, and MCMC in DMCMC runs before reverse-S/ODE integration:
PC (prior noise) - (integ. step) + (MCMC) + (integ. step) + (MCMC) -.... + (data)
DMCMC (MCMC to generate points near image manifold) + (integration) -> (data)
Moreover, as already mentioned, MCMC in PC runs in A' at fixed a whereas MCMC in DMCMC runs in X' x S by adaptively updating a.
Finally, DMCMC can also be used to accelcrate PC algorithms as well, as shown in Figure 3 This
means MCMC in DMCMC and PC play orthogonal roles in generating clean data. If DMCMC were
a straightforward extension of PC, we would not see such acceleration.
Predictor Corrector (log scale)
F1D
500
PC
DL
200
100
20
10
NFE=0
100
200
300
400
Figure 13: PC acceleration with DLG on CIFAR10.
D.2 ON THE NOISE PREDICTOR NETWORK IN DMCMC
Nichol & Dhariwal ) and Roman et al. (2021) have also used noise predictor networks to im- prove diffusion sampling. Although both works employed neural net predict the noise level / diffusion time for a given data input, we emphasize that the works do not overlap with the contri- butions of DMCMC. The main contribution of our paper is not the noise prediction network itself. The novelty of DMCMC arises from how we use the noise prediction network.
On a high level, the main message of our paper is that we can significantly improve diffusion sam- pling by choosing better initial points for denoising. A noise predictor network was used in the process of finding initial points. On the other hand, the focus of the mentioned works is on improv- ing the denoising process with a noise predictor network. So, DMCMC and the mentioned works contribute to orthogonal aspects of diffusion sampling.
Let us elaborate. We first note that diffusion model sampling can be broken down into two parts: (part 1) generating an initialization point, and (part 2) integrating the reverse-S/ODE to generate data from the initialization point.
Nichol & Dhariwal 202I) and Roman et al. 1021 use the noise prediction network to accelerate
integration of the reverse reverse-S/ODe S/ODE (improve part 2). This is achieved by learning the covariance of
the reverse distribution (Nichol & Dhariwal 2021) or by adjusting the noise schedule with a neural
net (Roman al. 2021)
In DMCMC, MCMC generates initialization points for EvverseSSODEE that lie near the image man- ifold (improve part 1). This is possible because MCMC runs in X' x S by adaptively updating the
19

---

Under review as a conference paper at ICLR 2023
noise level with a noise prediction network. This leads to acceleration, it is easier for reverse- S/ODE integrators to generate clean data from points near the data manifold than from prior noise distribution samples (e.g., Gaussian noise). Hence. our paper proposes acceleration approach
entirely different from those of and Nichol & Dhariwal (202T) and Roman et al. (202T).
In fact, DMCMC does not resemble Nichol & Dhariwal (2021) and Roman et al. (2021) even from the perspective of implementation. Noise predictors in Nichol & Dhariwal (2021) and Roman et al.
2021 are used during reverse-S/ODE integration, and the noise predictor in DMCMC is used before
reverse-S/ODE integration as a sub-step of MCMC.
E
MORE DISCUSSIONS
E.
MORE EXPLANATION ON How DMCMC ACCELERATES SAMPLING
The discussion at the end of Section 3.2 describes why DMCMC combined with reverse-S/ODE integrator is faster than using a reverse-S/ODE integrator alone. Here, we provide a more detailed explanation of the acceleration phenomenon. To begin, we note that DMCMC consists of two steps that are executed sequentially: (a) MCMC on ()z,,), and (b) denoising MCMC samples by reverse- S/ODE. Since DMCMC without (a) just standard diffusion, we see the acceleration behavior comes from using j(xz, a) samples as initial points for reverse-S/ODE. Thus, we need to show how using f(z., rr) samples as initial points for reverse-S/ODE accelerates image generation.
This proceeds in two steps. We first explain how MCMC produces samples (æn, an) from p(xr, o) such that or is significantly smaller than max- Then, we explain how integrating over (Amin: TT)) is faster than integrating over (amin max) as in standard diffusion.
To begin, we first explain how MCMC produces samples (z)) from f(æ, a) such that on is significantly smaller than maxx We observe that f Po (ar ())((² dı becomes flatter and wider as increases. This is because / (=(l))))(( Jdă means we are applying Gaussian smoothing to p(æz) with a Gaussian kernel of variance 0². For instance, if p(z) is normal distribution with variance I ( ((( dd is a normal distribution with variance 0² + 7 It follows that high density values of p(ær, 0r) occur when æ is near the data manifold and is small. Since MCMC traverses high-probability regions, we can expect {xr,) will be close to the data manifold and {a,} will be smaller than max- Indeed, in Appendix F.T we see that actual On values are significantly smaller than max = 50 in CIFAR1O.
Standard diffusion needs to integrate the reverse-S/ODE over the large interval (amin mmax) ) to produce clean images. On the other hand, in DMCMC, MCMC produces samples (In, ox) from p(x, such that or is significantly smaller than o max- So, in the DMCMC framework, generate clean images, we only need to integrate the reverse- means the cost of integrating over (an, maxx) vanishes for DMCMC, leading to accelerated image generation. (To be precise, the cost of integrating over (an, mmax) is replaced by the cost of running MCMC sample from p(x, o). but we observe Section 5.1 that DLG mixes rapidly, so this cost is negligible.)
over the small interval (amin T,). This
More rigorously, given the same computation budget and the same integration method, integrating over (amin, xx) has less truncation error than integrating over (amin: maax). Computation budget roughly corresponds to the number of discretization points of the integration interval we can use to approximate the reverse- reverse-S/ODE Thus, a shorter interval of integration means we can use smaller step size h during integration, which implies smaller error. For instance, Fuler's method has ((1))
local error. A more rigorous exposition is given in Chapter 7 of Stoer & Bulisch (2002).
This justifies how DMCMC can generate better samples than standard diffusion under a fixed com- putation budget. In other words, DMCMC can use less computation budget than standard diffusion to achieve similar sample quality as standard diffusion, i.e., DMCMC can accelerate sampling.
E.2 TRADE-OFF BETWEEN THE CONVERGENCE SPEED OF MCMC AND SHARPNESS OF Ú(a)
Let us assume that log p( σ) is strongly convex and has L Lipschitz continuous gradients. We also assume - log p(or) is strongly convex and has M Lipschitz continuous gradients. We use such assumptions, because in the setting where either log p(ær a) or log ((a) is nonconvex, it is
20

---

Under review as a conference paper at ICLR 2023
difficult to say anything theoretically meaningful about convergence of MCMC on the joint p(æ; o)). We also assume the MCMC of choice is Langevin dynamics for ease of analysis.
The sharpness of the prior ((o) can then be characterized by M. Intuitively, if p((r) is more peaked around zero, - log p(a) will have a gradient which changes more rapidly, and so M will be larger. We then note that since log p(x: a)) convex and - log p(x σ) has L + M Lipschitz continuous gradients.
log p( の) log p(o), log p(x, o)) is also strongly
We now resort to Theorem 3 in (Cheng & Bartlet 2018), which shows that the convergence time of Langevin dynamics on log p(2, .) in terms of KL. divergence grows in the order of O(L++ )) (ignoring log terms). So, we indeed see there is trade-off between the mixing time of MCMC and the choice of the prior. Using prior that is sharper around zero will increase M and thus increase convergence time quadratically. We speculate that similar analysis will hold for Langevin Gibbs as well, but a rigorous analysis of Langevin Gibbs is worthy of a paper of its own.
However, in practice, we do note that Langevin Gibbs with the prior used in our paper converges quite fast, as shown in Section 5.1 In the rightmost panel of Figure 2 we observe that the autocor- relation of image labels vanishes after only few iterations. space, image labels would have high autocorrelation. For visualization of DLG chains, we refer the readers to Appendix B.1 So, we can say Langevin Gibbs reliably produces samples from p(ar, o) even with small number of steps.
the sampler mixed poorly in the x
E.3 VE DIFFUSION VS. VP DIFFUSION IN DMCMC
A natural question is whether we can join MCMC with Severse-SODDE using the VP diffusion frame- work as well. It is true that the VP setting could be more stable than the VE setting, and that is why some recent solvers work in the VP setting. For instance, DPM-Solver (Lu et al.) 2022) only pro- vides experiment results in the VP setting, despite the fact that DPM-Solver can be applied to the VE setting as well. In the case of DEIS (Zhang & Chen 2022). there is large performance gap in the VE and VP settings. Specifically, the performance of DEIS deteriorates significantly in the VE setting.
We used VE diffusion because, from the perspective of generating better initialization points via MCMC, VE setting was better than VP setting. We believe the benefits outweigh the downsides, since, as shown in Table [ DLG beats all recent fast solvers regardless of whether that fast solver works in the VP setting or the VE setting. We conjecture that stability does not have a large influence on DMCMC because the reverse-S/ODE initialization points generated by MCMC are close to the image manifold, such that the variance of initialization points is small compared to the variance of prior noise. This is possibly why DMCMC improves the performance lower bound for some deterministic integrators (Section 5.2).
We also give detailed explanation of why DMCMC is more compatible with VE. Concretely, we observed two differences between VE diffusion and VP diffusion that made MCMC with VP diffusion difficult.
We first establish some notations. For a E (0,1), 1). we define the perturbation kernel Pa. ()) = M( | Voa, (1 - T) and the a-conditional distribution p(ı a) == f Pa | x)p(2) dā. If O, we get the prior noise distribution, which is the standard normal distribution. If a 1. we recover the data distribution. VP diffusion proceeds by decreasing a from 20216). We can generate clean data from Gaussian noise by using a numerical integrator to solve the reverse-S/OO -SOrDe for VP diffusion from a = to a 1. With prior p(a) on A == [0, 1), we then have joint distribution
(Song et al.
M a:) = p(x aa) p(a)
(17)
on the product space X x A. Then, analogous to DMCMC with VE diffusion, we may use MCMC to produces samples (( ) from p(æ, a:) with the help of an a-classifier ys and then integrate the Eiverse-SODDE from an to to generate clean data. However, we observed two differences between VE diffusion and VP diffusion that made MCMC with VP diffusion difficult.
21

---

Under review as a conference paper at ICLR 2023
In VE diffusion, the suppor (a Z)p(ā)dd grows wider increases. Thus, wherever the current MCMC æ iterate is, there always is some noise level a such that æ is in the support of (2 d)p(ā)d dæ so the score function provides meaningful (i.e., non-zero) direction. The noise classification network do predicts this optimal noise level a for æ as shown in Appendix F.1
On the other hand, in the VP diffusion framework, the prior noise distribution (standard normal distribution) has finite support. To be precise, the standard normal distribution has non-zero density everywhere, but numerically, density becomes zero points sufficiently far from the origin. As the data distribution also has finite support, all intermediate distributions p( ot) of VP diffusion have finite support as well. Since .CMC takes random steps, is possible for an MCMC iterate to land at point where no intermediate distribution p( a) of VP diffusion provides meaningful density or gradient. Even worse, the mean of the corruption process of VP diffusion shifts in the process of changing data samples to zero mean standard normal samples. So, high-density paths between prior noise samples and data samples can become very narrow in nigh-dimensional scenarios.
Due to these pathologies of VP diffusion, what we observed happening in practice was that Langevin Gibbs would repeat the following behavior: try to approach the image manifold in the æ update step so increases, step off the high-density path leading to the data manifold in the update step, and get absorbed into the prior noise distribution so a decays to O. Indeed, in Figure [4] we observe values oscillating around zero. This implies Hn essentially prior noise samples, so using {( (n.) as initial points for the reverse-S/ODE has no practical benefit.
This type of problem with narrow, high-density regions often plagues MCMC. For a toy example, see Figure 3 Cfb et al. (2019). We postulate that a better MCMC sampling scheme could enable us run DMCMC under the VP diffusion framework. For instance, one could use MCMC with rejection steps that would reject points that step off high-density regions, or incorporate Riemannian manifold structure into the sampling scheme. However, we believe this topic for future work.
0.05
0.04
de -elalxe)
0.03
0.01
0.00
0
200
400
B0D
800
1000
Iteration
Figure 14: an trajectories for DLG with VP diffusion in the MoG setting.
F MORE EXPERIMENTS
F. EVOLUTION OF a DURING LANGEVIN GIBBS ON X' x S
In Figure 15
we have visualized Langevin Gibbs trajectories of a in the MoG setting with 1k modes CIFARTO images (this setting decouples sampler behavior from score function approximation error) and the score network setting. We indeed observe that a moves up and down, allowing æ to travel between disjoint modes of the distribution. Moreover, in Figure 2 of Section 5.T æ samples cover all 1/ modes of the MoG. This experimentally proves sequence of DLG is exploring the entire image distribution.
In the MoG setting, let us define ón as the distance of æn to the closest mode in the MoG. In Figure 16 we see a., go(a a..) is almost identical to 8,/ Vd, where d is the dimension of a,, This
'hhen we mention the support of distribution, we mean the set of points where the numerical value (value represented on a computer) of the density is nonzero. This is because, even when distribution theoretically has nonzero density everywhere, it is possible that the numerical value of the density vanishes outside some hounded region.
22

---

Under review as a conference paper at ICLR 2023
reasonable, the Gaussian amnulus theorem tells us samples from high-dimensional Gaussian distribution of mean i and variance a² come from shell of radius a centered μa.
Specifically, due to the Gaussian annulus theorem, the samples of f Pa d)p(ā)dē are most likely to come from a shell of radius a centered around the image manifold. (See Figure 2 (a) Chung al, (022 Then, given certain æ which distance 5 from the image manifold, we can intuitively argue that optimal a for æ, i.e., of highest likelihood under p(a æ), can be determined by equating 5 = ov d such that 8 vd In the MoG case, this ö is approximated by the distance of æ to the closest mode in the MoG. Since the ar values predicted by the noise classifier agree with approximated 5 \/vd, we can see that the noise classifier is a good approximation of p(a æ)).
Or Trajectory (MoG)
0r Trajectory (Score Network)
6.5
6.5
6.0
6.0
5.5
5.5
MCMC ineration
MCMC Iteration
Figure 15: Visualization on trajectories in MoG and score network scttings.
5.4
5.75
5.2
5.50
5.25
3.5
5.5
5.00
8x 1
5.0
4.75
5.0
250
750
1000
250
500
750
1000
230
500
1000
250
500
750
1000
MCMC berasion
MCMC Iteration in
MCMC heration
MCNCC Reraaion
5.75
6.0
5.50
5.0
18105
5.4
5.25
5.5
5.00
5.0
5.0
4.75
1000
250
500 MCMC Ireaatonn
500
teratton
MCMC Itenation
MENC Iteration
5.0
5.6
5.6
150.6
5.0
5.5
5.4
5.2
5.2
5.0
5.0
3.0
terazion
Iicration
Iterationn
Figure 16: do prediction and (approximate) distance to image manifold in the MoG setting.
23

---

Under review as a conference paper at ICLR 2023
F.2 DENOISING STEP ABLATION
One may ask whether the denoising step in DMCMC is necessary component to generate clean samples. To answer this question, compare DLG with and without the denoising step. We the reverse diffusion integrator as the reverse-SDE solver in the denoising step. Figure 17 shows DLG without the denoising step completely fails to generate valid clean image samples, regardless of how large Rakip we use.
HD
500
200
100
50
Denoising Step
With Denoising (Rev. Diff.)
20
10
200
500
600
Figure 17: Ablation of the denoising step DMCMC.
24


Reviews:
[{"review_id": "mJCt3Ql6-rZ", "reviewer": "Reviewer_uZQw", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper the authors propose to extend the state space of diffusion models to answer the following question: how much should we sample at a given noise level? The way they proceed is as follows. First, they put some prior on the space of noise level $p(\\sigma)$ (in practice this distribution is chosen so that $p(\\sigma) \\propto 1/\\sigma$ to put more emphasis on the distribution with small noise. This is because we ultimately want to sample from the original distribution  (without noise). Then we can easily write the joint distribution on the space of image and noise levels by using $p(x|\\sigma)p(\\sigma)$. Then we sample using a Gibbs procedure which can be described as follows:\n* Given a noise level we can approximately sample from $p(x|\\sigma)$ leveraging diffusion models. In particular, we use an Unadjusted Langevin Dynamics targetting $p(x|\\sigma)$ where the gradient of the logarithm of this density is given by the learned score of the diffusion model. This part is really similar to a diffusion model\n* Second we need to sample from $p(\\sigma|x)$. To do so the authors train a neural network to predict the noise level. In fact even though we need to learn a probability distribution the authors found that just outputing one value of $\\sigma$ corresponding to the mode of the distribution worked well.\nThe authors then alternate between the two steps. They illustrate their method on CelebA-HQ 256 and CIFAR 10.", "summary_of_the_review": "I think this work is an interesting contribution to the field.\nHowever, as of now I think that there are a few elements missing:\n* a discussion of the literature to assess the novelty of the idea\n* a discussion of some methodological choices that seem to be quite non-classical (VE-SDE, corrector only step)\n* verify claim on SOTA for Celeba256.\n* most importantly, a study of the evolution of $\\sigma$ to assess if we are really sampling from a Markov chain.\nFor these reasons, I think that as of now reasons to reject the paper outweight reasons to accept it.", "strength_and_weaknesses": "STRENGTHS:\n* I think the ideas of the paper are clearly explained. I like the idea of extending the state space to also sample from the parameter s$\\sigma$. I think this is an elegant idea that could lead to further developments.\n* The experimental results seem to confirm that using the proposed method Denoising Langevin Gibbs (DLG) brings an advantage to existing sampling techniques.\n\nWEAKNESSES:\n* At the core level I think there is something weird with the strategy. Classical denoising diffusion models aim at denoising the image. They start with a Gaussian random variable and then go backward in time to reach the initial distribution. In that case the $\\sigma$ decrease towards $0$ which makes sense as we are trying to go back to the data distribution. But here this is quite different as we are also sampling from $p(\\sigma|x)$. At the end of the day (if things were perfect) we should sample from $p(x, \\sigma)$. Hence, the noise should travel to the whole interval $[\\sigma_{\\mathrm{min}}, \\sigma_{\\mathrm{max}}]$ and not be stuck at $\\sigma_{\\mathrm{min}}$. Being stuck near $\\sigma_{\\mathrm{min}}$ has the advantage that then the samples are perceptually good but then we do not explore the whole distribution. It would be incredibly useful and I think key to the paper if the authors could provide the evolution of the sequence of $\\sigma$. If this sequence is decreasing only then we are not exploring the distribution and the claim that we are doing a Markov chain in the augmented space is misleading. Maybe this is due to the fact that we use a classifier to learn the noise level $\\sigma$. In any case, I think that the sampling on $\\sigma$ which is the real methodological contribution of the paper is not investigated enough by the authors.\n* Why do the authors focus on the Variance Exploding SDE? In the literature it has been shown that usually VP-SDE (which can be connected to DDPM) are more stable and yield better results than VE-SDE. This is because VE-SDE is just a Brownian motion whereas VP-SDE is a Ornstein-Uhlenbeck process (for which the variance does not explode and converge to the dimension). I am pretty sure that the analysis still holds in that case.\n* I think that the authors missed a few references trying to predict the noise level in the sampling. For instance [1] also learned the covariance in DDPM models. In [2] the authors learn the noise level. The method seems very similar to what is introduced and is not discussed in the paper.\n* I think that the claims on SOTA for FID for Celeba256x256 as some methods seem to achieve 3.25 see https://paperswithcode.com/sota/image-generation-on-celeba-256x256\n\nOTHER QUESTIONS/COMMENTS:\n* I don't really understand the discussion at the end of Section 3.2.\n* In Equation (15) the authors seem to be using the Langevin dynamics targetting (the approximate) noised target distribution. This corresponds to the corrector step in [3]. Empirical analysis in [3] shows that using only the corrector step and not the predictor-corrector (or only predictor) step was very detrimental to the quality of the sampling. However it does not seem to affect the results of the present work. Could the authors give more details about this?\n\n[1] Improved Denoising Diffusion Probabilistic Models - Nichol, Dhariwal\n[2] Noise Estimation for Generative Diffusion Models - San Roman, Nachmani, Wolf\n[3] Score-Based Generative Modeling through Stochastic Differential Equations - Song, Sohl-Dickstein, Kingma, Kumar, Ermon, Poole", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Gi9Cu1_Zqf", "reviewer": "Reviewer_DAST", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a new way of acceleration diffusion based generative models that combines MCMC on a augmented space with reverse-S/ODE integrators. More specifically, a Gibbs sampler procedure is introduced to traverse the augmented space based on a pre-trained classifier. Numerical experiments demonstate the effectiveness of the proposed method.", "summary_of_the_review": "The idea of using MCMC to accelerate reverse integration of diffusion model is interesting and experiments show positive evidence. ", "strength_and_weaknesses": "Stength:\n\n1. The paper is well written and organized.\n2. The idea of using MCMC on the augment space for accelerating diffusion models is interesting and new.\n\n\nWeaknesses:\n\n1. The intialization of $x_0$ for DLG is not clearly stated. The author claimed that it needs to be set close to the image manifold. However, they also say $x_0$ is generated by sampling from a noise distribution, adding Gaussian noise of variance $0.25$, and runing Gibbs sampling a few iterations. I am confused with how $x_0$ is generated in practice. If it is generated by adding Gaussian noise to samples from real image data, the same trick can be applied to general diffusion models for acceleration which is not included in the ablation study.\n\n2. I think the comparison to LD in section 5.1 is unfair. It would be better to compare with ALD with appropriate annealing schedules.\n\n3. The biaseness towards high-density regions of $\\mathcal{X}$ is controlled by the prior $\\hat{p}(\\sigma)$. Without appropriate choice of $\\hat{p}(\\sigma)$, the claim in section 3.2 needs not to hold.\n\n4. Usually, MCMC takes some iterations for convergence. In practice, we want to use prior $\\hat{p}(\\sigma)$ the encourages small noise. There seems to be a trade-off between the mixing time of MCMC and the choice of $\\hat{p}(\\sigma)$ which is not discussed in this paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "88aMiRlpajt", "reviewer": "Reviewer_7H8V", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims at speeding up the reverse simulation process of score based denoising diffusion models through SDE. The main framework consists of two parts, where the first part is an MCMC based sampler that samples (t, x_t ~ p_t) jointly, where p_t is the marginal distribution of the forward process till time t. The second part is the standard reverse ODE simulation, where the simulation starts from this sampled time t and corresponding location x_t, and ends at t=0 to obtain the final sample. The main contribution comes from the first part where MCMC is implemented via Gibbs and stochastic gradient langevin with learned score function as the approximation. Experiments show that it can save the computation, while also improving several ODE-solver based approaches. \n", "summary_of_the_review": "Overall this paper proposed a practical approach for speeding up the reverse process sampling of denoising diffusion models. The main concern comes from the novelty and technical correctness of the assumption/approximation. The experiment section can potentially be improved with more baseline comparison.\n\n====\nupdated after the authors' response and more experimental details.", "strength_and_weaknesses": "Strength:\n\n- Improving the speed of the reverse process in denoising diffusion models is an important topic. \n- The proposed method is able to improve the efficiency, through the experimental justification.\n\nWeakness:\n\n- The novelty is somewhat limited, given the existing predictor-corrector algorithm. Using MCMC to obtain samples from p_t(x) is already done in Song et.al, 2021b. This paper directly uses MCMC to sample from p_t(x), which is a surrogate that has a trade-off between quality and the speed. But overall the technical contribution is limited as it is a relative straightforward extension of existing technique.\n\n- There might be a potential issue for the first stage (i.e., using MCMC for p_t). The score function required for MCMC is approximated using the learned score function. It would be fine if the sample x is roughly around the high density region of p_t(x), however the initial samples fed to the score function might be quite far away. These would be the out-of-distribution samples for the neural parameterized score function, where the neural network may not be able to extrapolate that well. \n\n- Paper [1] is included but is not directly mentioned or compared. The authors are encouraged to provide more experimental comparison with [1]. \n\nReference:\n\n[1] DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps, Lu et.al.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1YOOop0Yy97", "reviewer": "Reviewer_vzaX", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a faster algorithm, DMCMC, for integrating the reverse SDE/ODE associated with diffusion models. Unlike many existing works, DMCMC samples in the joint space of (x, sigma). Gibbs sampling is adopted for sampling in the joint space. Sampling x | sigma is just ordinary Langevin dynamics, and sampling sigma | x requires additional training of a time prediction network. After MCMC sampling, the algorithm will traverse to a time close to 0. Then, differential equation solvers can be adopted to generate the final sample. DMCMC can be combined with existing solvers, and it can generate high-quality samples in 10~20 NFE. ", "summary_of_the_review": "The paper proposes a novel way to accelerate the sampling of diffusion models. The proposed method can be combined with other approaches, which is an advantage. However, there are still some unclear aspects about the spectrum of the applicability / mechanism of the acceleration behavior.", "strength_and_weaknesses": "Strength:\n- Sampling from diffusion models is an important problem;\n- Sampling in the joint space is somewhat novel, and I am somewhat surprised that the authors are able to make that work.\n- The proposed sampler is efficient.\n\nWeaknesses:\n- How does the proposed method work with more recent fast solvers, such as DPM-Solver and DEIS? I wonder whether the proposed approach can still help if the solver itself is fast enough. \n- The noise predictor network is a trained distribution rather than the true conditional p(sigma | x). With such an approximate conditional, MCMC can no longer guarantee to converge to the true stationary distribution (so rigorously it is not a Gibbs sampling algorithm). I wonder to what extent does this approximation affect the convergence / sample quality.\n- It is not that intuitive to me how sampling in the joint space can be much faster than annealed MCMC. Doesn't the sampler still require many steps in the x space to mix? It would be better if the authors can better explain the mechanism behind the acceleration behavior. I understand that sampling from annealed distributions is better than directly sampling from p(x_0). But I don't quite understand why one should sample sigma rather than just using a deterministic schedule like other SDE solvers. Does it correspond to some adaptive solvers?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject