Original: Convergence of score-based generative modeling for general data distributions
Revised: Convergence of score-based generative modeling for general data distributions

Original: Anonymous Author(s)

Original: AffiliationAddress

Original: Abstract
Revised: Abstract

Original: We give polynomial convergence guarantees for denoising diffusion models that donot rely on the data distribution satisfying functional inequalities or strong smoothness assumptions.
Revised: We give polynomial convergence guarantees for denoising diffusion models for general data distributions, with no assumptions related to functional inequalities or smoothness.

Original: Assuminga L 2 -accurate score estimate, we obtain Wassersteindistance guarantees for any distributions of bounded support or sufficiently decaying tails, as well as TV guarantees for distributions with further smoothnessassumptions.
Revised: Assuming L 2 -accurate score estimates, we obtain Wasserstein distance guarantees for any distribution of bounded support or sufficiently decaying tails, as well as TV guarantees for distributions with further smoothness assumptions.

Original:  Introduction
Revised:  Introduction

Original: Diffusion models have gained huge popularity in recent years in machine learning, as a method tolearn and generate new samples from a data distribution.
Revised: Diffusion models have gained huge popularity in recent years in machine learning, as a method to learn and generate new samples from a data distribution.

Original: Score-based generative modeling (SGM),as a particular kind of diffusion model, usesa learned score function to transform white noise tothe data distribution by following a stochatic differential equation.
Revised: Score-based generative modeling (SGM), as a particular kind of diffusion model, uses learned score functions (gradients of the log-pdf) to transform white noise to the data distribution through following a stochatic differential equation.

Original: While SGM’s have achievedstate-of-the-art performance for artificial image and audio generation [SE19; Dat+19; Gra+19; SE20; Son+20; Men+21; Son+21b; Son+21a;Ram+22; Jin+22],  our theoretical understanding of thesemodels is still under development.
Revised: While SGM has achieved state-of-the-art performance for artificial image and audio generation [SE19; Dat+19; Gra+19; SE20; Son+20; Men+21; Son+21b; Son+21a; Jin+22], including being a key component of text-to-image systems [Ram+22], our theoretical understanding of these models is still nascent.

Revised: In particular, basic questions on the convergence of the generated distribution to the data distribution remain unanswered.

Original: There have been some recent progress in theoretical guarantees of SGM [LLT22; De +21]; however,these works either suffer from exponential dependence on parameters or rely on strong assumptionson the data distribution such assmoothness or functional inequalities, which are rarely satisfied inpractical situations.
Revised: Recent theoretical work on SGM has attempted to answer these questions [De +21; LLT22; De 22], but they either suffer from exponential dependence on parameters or rely on strong assumptions on the data distribution such as functional inequalities or smoothness, which are rarely satisfied in practical situations.

Original: Indeed, one of the most successful applications of diffusion model is conditionalgeneration of  images based on given text, for which the data distribution is multimodal and hencedoes not fit into the existing theory.
Revised: Indeed, data distributions—such as those arising in the hallmark application of generating images from text—are often multimodal and hence do not satisfy functional inequalities with reasonable constants, and supported on lower-dimensional manifolds and hence not smooth.

Original: In this work, we aim to develop theoreticalguarantees (withpolynomial complexity) for SGM for a more general class of distributions. 1.1 Prior work on convergence guarantees We highlight two recent works. [LLT22] is the first to give polynomial convergence guarantees in
Revised: In this work, we aim to develop theoretical convergence guarantees with polynomial complexity for

Original: TV distance under L 2 -accurate score error for a reasonable family of distributions.

Original: They introducea framework to reduce the analysis under L 2 -accurate score error to L ∞ -score error. However,they rely on the data distribution satisfying smoothness conditions and a log-Sobolev inequality—astrong assumption which essentially limits the guarantees to unimodal distributions. [De 22] givesconvergence in Wasserstein distance for distributions with bounded support M . In particular, thiscovers the case of distributions supported on lower-dimensional manifolds, where guarantees in TVdistance are unattainable.
Revised: SGM under minimal data assumptions.

Original: However, their guarantees have dependence exp( O (diam( M ) 2 /ε )) forgeneral distributions, and an improved, but still exponential dependence exp( (cid:101) O (Γ)) for distributionswith Hessian bounded by Γ /σ 2 t . We note that other works also analyze the generalization error of a learned score estimate [BMR20; De 22], andwithout further assumptions, learning an L 2 -accurate score estimate requires a number of samples exponential in the dimension. This is beyond the scope of our paper, as we assume that
Revised: We refer to Appendix A for a thorough discussion about prior works and our contributions.

Original:  an L 2 -accurate score estimate is obtainable. Upon finishing our paper, we learned of a concurrent and independent work [Che+22] which obtainedtheoretical guarantees for score-based generative modeling under similarly general assumptions onthe data distribution. 1.2 Our contributions
Revised:  Problem setting

Original: In this work, we analyze convergence in the most general setting of distributions of bounded support,as in [De 22]. We give Wasserstein bounds for any distribution of bounded support (or sufficientlydecaying tails), and TV bounds for distributions under smoothness assumptions, that are polynomialin all parameters, and do not rely on the data distribution satisfying any functional inequality. We streamline the analysis of [LLT22] and make a key improvement to their χ 2 -based analysis bybounding a certain KL divergence without relying on a global functional inequality. We use a keylemma that distriutions which are close in χ 2 -divergence have score functions that are close in L 2 ,which may be of independent interest.
Revised: Given samples from a data distribution P data , the problem of generative modeling is to learn the distribution in a way that allows generation of new samples.

Original: Problem setting and main results

Original: A general framework for many score-based generative models is where noise is injected intoa datadistribution P data via a forward SDE [Son+20]
Revised: A general framework for many scorebased generative models is where noise is injected into P data via a forward SDE [Son+20]

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) x 0 ∼ (cid:101) P 0 := P data .
Revised: where (cid:101) x 0 ∼ (cid:101) P 0 := P data .

Original: Let (cid:101) p t denote the density of (cid:101) x t .
Revised: Let (cid:101) p t denote the density of (cid:101) x t .

Original: Remarkably, (cid:101) x t also satisfies a reverse-time
Revised: Remarkably, (cid:101) x t also satisfies a reverse-time

Original: SDE,
Revised: SDE,

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) w t is a backward Brownian motion [And82].
Revised: where (cid:101) w t is a backward Brownian motion [And82].

Revised: Because the forward process transforms the data distribution to noise, the hope is to use the backwards process to transform noise into samples.

Original: Of course, in practice the score function ∇ ln (cid:101) p t is not available, and  has to be learned from empirical samples (cid:101) x t from the forward SDE starting from the data samples.
Revised: Of course, in practice the score function ∇ ln (cid:101) p t is not available, and and has to be learned from empirical samples (cid:101) x t from the forward SDE starting from the data samples.

Original: Denoising auto-encodersare often used for the score function estimate, which will result in a guarantee that the L 2 error
Revised: Denoising auto-encoders are often used for the score function estimate, which will result in a guarantee that the L 2 error

Original: E (cid:101) p t [ ∥∇ ln (cid:101) p t ( x ) − s ( x, t ) ∥ 2 ] is small [Vin11].
Revised: E (cid:101) p t [ ∥∇ ln (cid:101) p t ( x ) − s ( x, t ) ∥ 2 ] is small [Vin11].

Original: The natural question is  the following:
Revised: The natural question is then the following:

Original: Given L 2 -error bounds of the score function, how close is the distribution generatedby (2)  to the data distribution P data ?
Revised: Given L 2 -error bounds of the score function, how close is the distribution generated by (2) (with score estimate s ( x, t ) in place of ∇ ln (cid:101) p t , and appropriate discretization) to the data distribution P data ?

Original: We note it is more realistic to consider L 2 rather than L ∞ error, and this makes the analysis morechallenging.
Revised: We note it is more realistic to consider L 2 rather than L ∞ -error, and this makes the analysis more challenging.

Original: In addition, wewould hope to get a result with as few structural assumptions as possibleon P data , so that it can be useful in realistic scenarios where SGM is applied.
Revised: In addition, we hope to obtain a result with as few structural assumptions as possible on P data , so that it can be useful in realistic scenarios where SGM is applied.

Revised:  Main results

Original: To state the problem more precisely, we will consider specifically denoising diffusion probabilistic modeling (DDPM)
Revised: To state our results, we will consider a specific type of SGM called denoising diffusion probabilistic modeling (DDPM)

Original: [Son+20], where in the forward SDE, f ( x, t )
Revised: [HJA20], where in the forward SDE (1), f ( x, t )

Original: = − 12 g ( t ) 2 x for some nondecreasing function g to be chosen.
Revised: = − 12 g ( t ) 2 x for some nondecreasing function g to be chosen.

Original: The forward process is an Ornstein-Uhlenbeck process with timerescaling: (cid:101) x t has the same distribution asm t (cid:101) x 0 + σ t z, where
Revised: The forward process is an Ornstein-Uhlenbeck process with time rescaling: (cid:101) x t has the same distribution as

Original: [Equation]
Revised: [Equation]

Original: , and z ∼ N (0 , I ) .
Revised: , where

Original: [Equation]
Revised: [Equation]

Original: Given an estimate score function s ( x, t ) approximating ∇ ln (cid:101) p t ( x ) , we can simulate the reverseprocess (reparameterizing t ← (cid:91) T − t and denoting p t := (cid:101) p T − t )
Revised: Given an estimate score function s ( x, t ) approximating ∇ ln (cid:101) p t ( x ) , we can simulate the reverse process (reparameterizing t ← (cid:91) T − t and denoting p t := (cid:101) p T − t )

Original: [Equation]
Revised: [Equation]

Original:  with the exponential integrator discretization [ZC22], where h k = t k +1 − t k and η k +1 ∼ N (0 , I d ) :
Revised: [Equation] with the exponential integrator discretization [ZC22], where h k = t k +1 − t k and η k +1 ∼ N (0 , I d ) :

Original: z t k +1 =
Revised: z t k +1 =

Original: z 
Revised: z t k

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 5) where γ 1 ,k = exp

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] [Equation] = 
Revised: where γ 1 ,k = exp

Original: [Equation]
Revised: [Equation]

Original: G t 
Revised: t k

Original: [Equation]
Revised: [Equation]

Original: ,t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: 1 , and Gt
Revised: , and G

Original: [Equation]
Revised: [Equation]

Original: ,t

Original: [Equation]

Original: [Equation]

Original: We initialize z 0 with a prior distribution that approximates p 

Revised: We initialize z 0 with a prior distribution that approximates p

Original: = (cid:101) p T for sufficiently large T :
Revised: = (cid:101) p T for sufficiently large T :

Original: [Equation]
Revised: [Equation]

Original: While we focus on DDPM, we note that the continuous process underlying DDPM is equivalent tothat of score-matching Langevin diffusion (SMLD) under reparameterization in time and space (see[LLT22, §C.2]).
Revised: While we focus on DDPM, we note that the continuous process underlying DDPM is equivalent to that of score-matching Langevin diffusion (SMLD) under reparameterization in time and space (see [LLT22, §C.2]).

Original: We will further take g ≡ 1 for convenience in stating our results.
Revised: We will further take g ≡ 1 for convenience in stating our results.

Original: Our goal is to obtain a quantitative guarantee for the distance between the distribution q t K for z t K(for appropriate t K ≈ T ) and P data , under a L 2 -score error guarantee.
Revised: Our goal is to obtain a quantitative guarantee for the distance between the distribution q t K for z t K (for appropriate t K ≈ T ) and P data , under a L 2 -score error guarantee.

Original: In the following, we assume asequence of discretization points 

Revised: In the following, we assume a sequence of discretization points

Original: = t 0 < t 1 < · · · < t K ≤ T has been chosen.
Revised: = t 0 < t 1 < · · · < t K ≤ T has been chosen.

Original: Assumption 1 ( L 2 score error) .
Revised: Assumption 1 ( L 2 score error) .

Original: For any t ∈ { T − t 0 , . . .
Revised: For any t ∈ { T − t 0 , . . .

Original: , T − t K } , the error in the score estimateis bounded in L 2 ( (cid:101) p t ) :
Revised: , T − t K } , the error in the score estimate is bounded in L 2 ( (cid:101) p t ) :

Original: [Equation]
Revised: [Equation]

Original: We note that the gradient ∇ ln (cid:101) p t grows as 1 σ 2 t as t → 0 , so this is a reasonable assumption, andquantitatively weaker than a uniform bound over t .
Revised: We note that the gradient ∇ ln (cid:101) p t grows as 1 σ 2 t as t → 0 , so this is a reasonable assumption, and quantitatively weaker than a uniform bound over t .

Original: Assumption 2 (Bounded support) .
Revised: Assumption 2 (Bounded support) .

Original: P data is supported on B R (0) :
Revised: P data is supported on B R (0) :

Original: = (cid:8) x ∈ R d : ∥ x ∥ ≤ R (cid:9) .
Revised: = (cid:8) x ∈ R d : ∥ x ∥ ≤ R (cid:9) .

Original: For simplicity, we assume bounded support in the main text, but note that our results generalize todistributions with sufficiently fast power decay.
Revised: For simplicity, we assume bounded support when stating our main theorems, but note that our results generalize to distributions with sufficiently fast power decay.

Original: In the application of image generation, pixel valuesare bounded, so Assumption 2 is satisfied with R typically on the order of √ d .
Revised: In the application of image generation, pixel values are bounded, so Assumption 2 is satisfied with R typically on the order of √ d .

Original: These are the only assumptions we need to obtain a polynomial complexity guarantee.
Revised: These are the only assumptions we need to obtain a polynomial complexity guarantee.

Original: We alsoconsider the following stronger smoothness assumption, which is Assumption A.6 in [De 22] andwill give better dependencies.
Revised: We also consider the following stronger smoothness assumption, which is Assumption A.6 in [De 22] and will give better dependencies.

Original: Note that [De 22, Theorem I.8] shows a (nonuniform) version of
Revised: Note that [De 22, Theorem I.8] shows a (nonuniform) version of

Original: Assumption 3 holds when p 0 is a smooth density on a convex submanifold.
Revised: Assumption 3 holds when p 0 is a smooth density on a convex submanifold.

Original: Assumption 3.
Revised: Assumption 3.

Original: The following bound of the Hessian of the log-pdf holds for any t > 0 and x :
Revised: The following bound of the Hessian of the log-pdf holds for any t > 0 and x :

Original: [Equation]
Revised: [Equation]

Original: for some constant C > 0 .
Revised: for some constant C > 0 .

Original: Finally, the following smoothness assumption on (cid:101) p 0 will allow us to obtain TV guarantees.
Revised: Finally, the following smoothness assumption on (cid:101) p 0 will allow us to obtain TV guarantees.

Original: Assumption 4.
Revised: Assumption 4.

Original: P data admits a density (cid:101) p 

Revised: P data admits a density (cid:101) p

Original: ∝ e − V ( x ) where V ( x ) is L -smooth.
Revised: ∝ e − V ( x ) where V ( x ) is L -smooth.

Original:  Algorithm 
Revised: We are now ready to state our main theorems. Algorithm 1 DDPM with exponential integrator [Son+20; ZC22]

Original: INPUT:
Revised: INPUT:

Original: Time T ; discretization points 

Revised: Time T ; discretization points

Original: = t 0 < t 1 < · · · < t N ≤ T ; score estimates s ( · , T − t k ) ; radius R ; function g (default: g ≡ 1 )
Revised: = t 0 < t 1 < · · · < t N ≤ T ; score estimates s ( · , T − t k ) ; radius R ; function g (default: g ≡ 1 )

Original: Draw z 0 ∼ p prior from the prior distribution p prior given by (7).
Revised: Draw z 0 ∼ p prior from the prior distribution p prior given by (7).

Original: for k from 1 to N do
Revised: for k from 1 to N do

Original: Compute z t k from z t k − 1 using (5).
Revised: Compute z t k from z t k − 1 using (5).

Original: end for
Revised: end for

Original: Let (cid:98)
Revised: Let (cid:98)

Original: z
Revised: z

Original: t N =  z t N if z t N ∈ B R (0) ; otherwise, let (cid:98)
Revised: t N = m − 1 T − t N z t N if z t N ∈ B R (0) ; otherwise, let (cid:98)

Original: z t N = 0 .
Revised: z t N = 0 .

Original: Theorem 2.1 (Wasserstein+TV error for distributions with bounded support) .
Revised: Theorem 3.1 (Wasserstein+TV error for distributions with bounded support) .

Original: Suppose that Assumption 1 and 2 hold with R ≥ √ d .
Revised: Suppose that Assumption 1 and 2 hold with R ≥ √ d .

Original: Then there is a sequence of discretization points0 ={+points 
+}
Revised: Then there is a sequence of discretization {+points 
+}

Original:  t 0 < t 1 < · · · < t N < T with N =
Revised: = t 0 < t 1 < · · · < t N < T with N =

Original: O (poly( d, R, 1 /ε  W ))
Revised: O (poly( d, R, 1 /ε TV , 1 /ε W ))

Original: such that if ε σ = (cid:101) o (cid:16) ε6 .
Revised: such that if ε σ = (cid:101) O (cid:16) ε

Original: 5TV ε 5 W R 9 d 2 .
Revised: [Table] 5 TV [Table]

Original: 25 (cid:17) ,then the distributionq t N of the  output  z t N of DDPM is ε TV -close in TV distance to a distributionthat is ε W in W 2 distance from P data .
Revised: [Table] 75 (cid:17) , then the distribution of the scaled output m − 1 T − t N z t N of DDPM is ε TV -close in TV distance to a distribution that is ε W in W 2 -distance from P data .

Original: If in addition Assumption 3 holds with C ≥ R 2 , it suffices forε σ = (cid:101) o (cid:16) ε 4TV C 2 d (cid:17) (note that the (cid:101) o ( · ) hides logarithmic dependence on ε W ).
Revised: If in addition Assumption 3 holds with C ≥ R 2 , it suffices for ε σ = (cid:101) O (cid:16) ε 4TV C 2 d (cid:17) (note that the (cid:101) O ( · ) hides logarithmic dependence on ε W ).

Original: This result is perhaps surprising at first glance, as it is well known that for sampling algorithms suchas Langevin Monte Carlo, structural assumptions on the target distribution—such as a log-Sobolevinequality—are required to obtain similar theoretical guarantees, even with the knowledge of the exact score function.
Revised: This result is perhaps surprising at first glance, as it is well known that for sampling algorithms such as Langevin Monte Carlo, structural assumptions on the target distribution—such as a log-Sobolev inequality—are required to obtain similar theoretical guarantees, even with the knowledge of the exact score function.

Original: The key reason that we can do better is that we utilize a sequence of scorefunctions s t along the reverse SDE, which is not available in standard sampling settings.
Revised: The key reason that we can do better is that we utilize a sequence of score functions s t along the reverse SDE, which is not available in standard sampling settings.

Revised: Moreover, we choose T large enough so that q 0 =

Revised: p prior is close to p 0 , and it suffices to track the evolution of the true process (2), that is, maintain rather than decrease the error.

Original: To someextent, this result shows the power of DDPM and other reverse SDE-based methods compared withgenerative modeling based on standard Langevin Monte Carlo.
Revised: To some extent, this result shows the power of DDPM and other reverse SDE-based methods compared with generative modeling based on standard Langevin Monte Carlo.

Original: A statement with more precise dependencies, and which works for unbounded distributions withsufficiently decaying tails, can be found as Theorem D.2.
Revised: A statement with more precise dependencies, and which works for unbounded distributions with sufficiently decaying tails, can be found as Theorem E.2.

Original: We note that under the Hessian bound(Assumption 3), up to logarithmic factors, the same score error bound suffices to obtain a fixed TVdistance to a distribution arbitrarily close in W 2 distance.
Revised: We note that under the Hessian bound (Assumption 3), up to logarithmic factors, the same score error bound suffices to obtain a fixed TV distance to a distribution arbitrarily close in W 2 distance.

Original: By truncating the resulting distribution, wecan also obtain purely Wasserstein error bounds.
Revised: By truncating the resulting distribution, we can also obtain purely Wasserstein error bounds.

Original: Theorem 2.2 (Wasserstein error for distributions with bounded support) .
Revised: Theorem 3.2 (Wasserstein error for distributions with bounded support) .

Original: In the same setting as
Revised: In the same setting as

Original: Theorem 2.1, consider the distribution (cid:98) q t N of the  truncated output (cid:98) x t N of DDPM.
Revised: Theorem 3.1, consider the distribution (cid:98) q t N of the scaled and truncated output (cid:98) x t N of DDPM.

Original: If Assumptions 1and 2 hold with R ≥ √ d and ε σ = (cid:101) o (cid:16)
Revised: If Assumptions 1 and 2 hold with R ≥ √ d and ε σ = (cid:101) O (cid:16)

Original: [Figure]
Revised: [Equation]

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: (cid:17) , then with appropriate (polynomial) choiceof parameters, W 2 ( (cid:98) q [Equation] K , P data ) ≤ ε W .
Revised: , then with appropriate (polynomial) choice of parameters, W 2 ( (cid:98) q t K , P data ) ≤ ε W .

Original: If in addition Assumption 3 holds with C ≥ R 2 , then 
Revised: If in addition Assumption 3 holds with C ≥ R 2 , then ε σ = (cid:101) O (cid:16)

Original: [Equation]
Revised: [Equation]

Original: suffices.
Revised: suffices.

Original: With an extra assumption on the smoothness of P data , we can also obtain purely TV error bounds:
Revised: With an extra assumption on the smoothness of P data , we can also obtain purely TV error bounds:

Original: Theorem 2.3 (TV error for distributions under smoothness assumption) .
Revised: Theorem 3.3 (TV error for distributions under smoothness assumption) .

Original: Suppose that Assumptions 1and 4 hold, P data is subexponential (with a fixed constant), and denote R = max (cid:110) √ d, E P data ∥ X ∥ (cid:111) .
Revised: Suppose that Assumptionsand 4 hold, P data is subexponential (with a fixed constant), and denote R = max (cid:110) √ d, E P data ∥ X ∥ (cid:111) .

Original: Then there is a sequence of discretization points 

Revised: Then there is a sequence of discretization points

Original: = t 0 < t 1 < · · · < t N < T with N =
Revised: = t 0 < t 1 < · · · < t N < T with N =

Original: O (poly( d, R, 1 /ε TV ))
Revised: O (poly( d, R, 1 /ε TV ))

Original: such that if ε σ = (cid:101) o (cid:16) ε 11 .
Revised: such that if ε σ = (cid:101) O (cid:16) [Table]

Original: 5TV R 14 d 2 . 25 L 5 (cid:17) , then TV ( q t N , P data ) ≤ ε TV .
Revised: 5 (cid:17) , then the distribution q t N of the output z t N of DDPM satisfies TV( q t N , P data ) ≤ ε TV .

Original: If in addition Assumption 3 holds with C ≥ R 2 , then ε σ = (cid:101) o (cid:16) ε 4TV C 2 d (cid:17) suffices.
Revised: If in addition Assumption 3 holds with C ≥ R 2 , then ε σ = (cid:101) O (cid:16) ε 4TV C 2 d (cid:17) suffices.

Original: A more precise statement can be found as Theorem D.3, which also works more generally withsufficient tail decay.
Revised: A more precise statement can be found as Theorem E.3, which also works more generally with sufficient tail decay.

Original: We note that this result can be derived directly by combining Theorem D.2 and a
Revised: We note that this result can be derived directly by combining Theorem E.2 and a

Original: TV error bound between P data and p t N (Lemma C.4) depending on the smoothness of P data .
Revised: TV error bound between P data and p t N (Lemma D.4) depending on the smoothness of P data .

Original:  Proof overview
Revised:  Proof overview

Original: Our proof uses the framework by [LLT22] to convert guarantees under L ∞ -accurate score function tounder L 2 -accurate score function.
Revised: Our proof uses the framework by [LLT22] to convert guarantees under L ∞ -accurate score function to under L 2 -accurate score function.

Original: For the analysis under L ∞ -accurate score function, we interpolatethe discrete process with estimated score, z t ∼ q t , and derive a differential inequality
Revised: For the analysis under L ∞ -accurate score function, we interpolate the discrete process with estimated score, z t ∼ q t , and derive a differential inequality

Original: [Equation]
Revised: [Equation]

Original: We bound resulting error terms, making ample use of the Donsker-Varadhan variational principle toconvert expectations to be under p t .
Revised: We bound resulting error terms, making ample use of the Donsker-Varadhan variational principle to convert expectations to be under p t .

Original: Under small enough step sizes, this shows that χ 2 ( q t || p t ) growsslowly (Theorem A.10), which suffices as χ 2 -divergence decays exponentially in the forward process.
Revised: Under small enough step sizes, this shows that χ 2 ( q t || p t ) grows slowly (Theorem B.10), which suffices as χ 2 -divergence decays exponentially in the forward process.

Original: Our main innovation over the analysis of [LLT22] is bounding a KL divergence term KL( q t ψ t || p t )without a global log-Sobolev inequality for p t .
Revised: The most challenging error term to deal with is the KL divergence term KL( q t ψ t || p t ) . Our main innovation over the analysis of [LLT22] is bounding this term without a global log-Sobolev inequality for p t .

Original: We note that it suffices for p t to be a mixture ofdistributions each satisfying a log-Sobolev inequality, with the logarithm of the minimum mixtureweight bounded below, and in Lemma B.2, we show that we can decompose any distributed ofbounded support in this manner if we move a small amount of its mass.
Revised: We note that it suffices for p t to be a mixture of distributions each satisfying a log-Sobolev inequality, with the logarithm of the minimum mixture weight bounded below, and in Lemma C.2, we show that we can decompose any distribution of bounded support in this manner if we move a small amount of its mass.

Original: In Section C, we show thatthis does not significantly affect the estimate of the score estimate, by interpreting the score functionas solving a Bayesian inference problem.
Revised: In Section D, we show that this does not significantly affect the estimate of the score function, by interpreting the score function as solving a Bayesian inference problem: that of de-noising a noised data point.

Original: We note the key Lemma C.5 may be of independent interest.
Revised: More precisely, we show in Lemma D.5 that the difference between the score functions of two different distributions can be bounded in L 2 in terms of their χ 2 -divergence, which may be of independent interest.

Original: Finally, we reduce from the L 2 to L ∞ setting by bounding the probabilities of hitting a bad set wherethe score error is large, and carefully choose parameters (Section D).
Revised: Finally, we reduce from the L 2 to L ∞ setting by bounding the probabilities of hitting a bad set where the score error is large, and carefully choose parameters (Section E).

Original: This gives a TV error bound to(cid:101)
Revised: This gives a TV error bound to (cid:101) p

Original: p δ —the forward distribution at small positive time.
Revised: δ —the forward distribution at small positive time.

Original: Finally, we can bound the Wasserstein distanceof (cid:101) p δ to (cid:101) P 0 (in the general case) or the TV distance (under additional smoothness of (cid:101) P 0 .)
Revised: Finally, we can bound the Wasserstein distance of (cid:101) p δ to (cid:101) P 0 (in the general case) or the TV distance (under additional smoothness of (cid:101) P 0 .)

Original: In Section E we show that the Hessian is always bounded by O (cid:16) dσ 2 t (cid:17) with high probability (cf.
Revised: In Section F we show that the Hessian is always bounded by O (cid:16) dσ 2 t (cid:17) with high probability (cf.

Original: Assumption 3).
Revised: Assumption 3).

Original: We speculate that a high-probability rather than uniform bound on the Hessian (as in
Revised: We speculate that a high-probability rather than uniform bound on the Hessian (as in

Original: Lemma A.13) can be used to obtain better dependencies, and leave this as an open problem.
Revised: Lemma B.13) can be used to obtain better dependencies, and leave this as an open problem.

Original: Notation and definitions
Revised: Notation and definitions

Original: We let (cid:101) p t denote the density of (cid:101) x t under the forward process (1).
Revised: We let (cid:101) p t denote the density of (cid:101) x t under the forward process (1).

Original: Note that x 0 ∼ (cid:101) P 0 may not admit adensity, but (cid:101) 
Revised: Note that x 0 ∼ (cid:101) P 0 may not admit a density, but (cid:101) x

Original: x t will for t > 0 .
Revised: t will for t > 0 .

Original: For the reverse process, we use the notation p t = (cid:101) p T − t , x t = (cid:101) x
Revised: For the reverse process, we use the notation p t = (cid:101) p T − t , x t = (cid:101) x

Original: T − t .
Revised: T − t .

Original: We defined m t and σ t in (3), 
Revised: We defined m t and σ t in (3), m

Revised: [Equation]

Revised: = exp

Original: [Equation]
Revised: [Equation]

Original: and note that (cid:101) p t
Revised: and note that (cid:101) p t

Original: = ( M m t ♯ (cid:101) P 0 ) ∗ φ σ 2 t , where M m ( x ) = mx denotes multiplication by m , F ♯ P denotesthe pushforward of the measure P by F , and φ σ 2 is the density of N (0 , σ 2 I d ) .
Revised: = ( M m t ♯ (cid:101) P 0 ) ∗ φ σ 2 t , where M m ( x ) = mx denotes multiplication by m , F ♯ P denotes the pushforward of the measure P by F , and φ σ 2 is the density of N (0 , σ 2 I d ) .

Original: When g ≡ 1 , we notethe bound σ 2 t ≤ min { 1 , t } and σ 2 t = Θ(min { 1 , t } ) .
Revised: When g ≡ 1 , we note the bound σ 2 t ≤ min { 1 , t } and σ 2 t = Θ(min { 1 , t } ) .

Original: We will let z t denote the (interpolated) discrete process (see (12)) and let q t be the density of z t .
Revised: We will let z t denote the (interpolated) discrete process (see (12)) and let q t be the density of z t .

Original: Wedefine
Revised: We define

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: and note that q t ψ t is a probability density.
Revised: and note that q t ψ t is a probability density.

Original: We defined G t ′ ,t = (cid:82) t t ′ g
Revised: We defined G t ′ ,t = (cid:82) t t ′ g

Original: ( T − s ) 2 ds in (6).
Revised: ( T − s ) 2 ds in (6).

Original: We denote the estimated score function by either s ( x, t ) and s t ( x ) interchangeably.
Revised: We denote the estimated score function by either s ( x, t ) and s t ( x ) interchangeably.

Original: A random variable X is subgaussian with constant C if
Revised: A random variable X is subgaussian with constant C if

Original: [Equation]
Revised: [Equation]

Original: A R d -valued random variable X is subgaussian with constant C if for all v ∈ S
Revised: A R d -valued random variable X is subgaussian with constant C if for all v ∈ S

Original: d − 1 , ⟨ X, v ⟩ issubgaussian.
Revised: d − 1 , ⟨ X, v ⟩ is subgaussian.

Original: We also define
Revised: We also define

Original: [Equation]
Revised: [Equation]

Original: Given a probability measure P on R d with density p , the associated Dirichlet form is E p ( f, g ) :
Revised: Given a probability measure P on R d with density p , the associated Dirichlet form is E p ( f, g ) :

Original: =222 (cid:82)
Revised: = (cid:82)

Original: Rd
Revised: R

Original: [Equation]
Revised: [Equation]

Original: , ∇ g ⟩ P

Original: [Equation]
Revised: [Equation]

Original: dx
Revised: ⟩ p

Original: [Equation]
Revised: [Equation]

Original: = [Equation] R d ⟨∇ f, ∇ g ⟩ p ( x ) dx ; denote E p ( f ) =
Revised: x ) dx ; denote E p ( f ) =

Original: E p ( f, f ) .
Revised: E p ( f, f ) .

Original: we say that a log-Sobolevinequality (LSI) holds with constant C LS if for any probability density q ,
Revised: we say that a log-Sobolev inequality (LSI) holds with constant C LS if for any probability density q ,

Original: [Equation]
Revised: [Equation]

Revised: Note (cid:82) R d (cid:13)(cid:13)(cid:13)

Revised: [Equation]

Revised: x

Original: Note (cid:82) R d (cid:13)(cid:13) (cid:13) ∇ ln q ( x ) p ( x ) (cid:13)(cid:13) (cid:13) 2 q ( x ) dx is also known as the Fisher information of q with respect to p .
Revised: [Equation] x ) [Equation] 2 q ( x ) dx is also known as the Fisher information of q with respect to p .

Original: Alternatively, defining the entropy by Ent p ( f ) =
Revised: Alternatively, defining the entropy by Ent p ( f ) =

Original: E p f ( x ) ln f ( x ) − E p f
Revised: E p f ( x ) ln f ( x ) − E p f

Original: ( x ) ln E p f ( x ) , for any f ≥ 0 ,
Revised: ( x ) ln E p f ( x ) , for any f ≥ 0 ,

Original: [Equation]
Revised: [Equation]

Revised: p

Revised: [Equation]

Revised: [Equation]

Revised: A Discussion A.1 Prior work on convergence guarantees

Revised: We highlight two recent works which make progress on this problem.

Revised: [LLT22] are the first to give polynomial convergence guarantees in TV distance under L 2 -accurate score for a reasonable family of distributions.

Revised: They introduce a framework to reduce the analysis under L 2 -accurate score to

Revised: L ∞ -accurate score.

Revised: However, they rely on the data distribution satisfying smoothness conditions and a log-Sobolev inequality—a strong assumption which essentially limits the guarantees to unimodal distributions.

Revised: [De 22] instead make minimal data assumptions, giving convergence in Wasserstein distance for distributions with bounded support M .

Revised: However, their guarantees have dependence exp( O (diam( M ) 2 /ε )) for general distributions, and an improved, but still exponential dependence exp( (cid:101) O (Γ)) for distributions with Hessian bounded by Γ /σ 2 t .

Revised: We note that other works also analyze the generalization error of a learned score estimate [BMR20;

Revised: De 22].

Revised: This is an important question because without further assumptions, learning an L 2 -accurate score estimate requires a number of samples exponential in the dimension.

Revised: As this is beyond the scope of our paper, we assume that an L 2 -accurate score estimate is obtainable.

Revised: A.2 Our contributions

Revised: In this work, we analyze convergence in the most general setting of distributions of bounded support, as in [De 22].

Revised: We give Wasserstein bounds for any distribution of bounded support (or sufficiently decaying tails), and TV bounds for distributions under smoothness assumptions, that are polynomial in all parameters, and do not rely on the data distribution satisfying any functional inequality.

Revised: We streamline the χ 2 -based analysis of [LLT22], with significant changes as to completely remove the use of functional inequalities.

Revised: In particular, the biggest challenge—and our key improvement—is to bound a certain KL-divergence without reliance on a global functional inequality.

Revised: For this, we prove a key lemma that distributions which are close in χ 2 -divergence have score functions that are close in L 2 , which may be of independent interest.

Revised: Upon finishing our paper, we learned of a concurrent and independent work [Che+22] which obtained theoretical guarantees for score-based generative modeling under similarly general assumptions on the data distribution.

Revised: We note that although our bounds are obtained under similar assumptions (with our assumption of the score estimate accuracy slightly weaker than theirs), our proof techniques are quite different.

Revised: Following the “bad set” idea from [LLT22], we derived a change-of-measure inequality with Theorem E.1, while the analysis in [Che+22] is based on the Girsanov approach.

Original: A DDPM with L ∞ -accurate score estimate
Revised: B DDPM with L ∞ -accurate score estimate

Original: We consider the error between the exact backwards SDE (4) and the exponential integrator withestimated score (5).
Revised: We consider the error between the exact backwards SDE (4) and the exponential integrator with estimated score (5).

Original: In this section, we bound the error assuming that the score estimate s is accuratein L ∞ .
Revised: In this section, we bound the error assuming that the score estimate s is accurate in L ∞ .

Original: Assumption 5 ( L ∞ score error) .
Revised: Assumption 5 ( L ∞ score error) .

Original: For any t ∈ { T − t 0 , . . .
Revised: For any t ∈ { T − t 0 , . . .

Original: , T − t K } , the error in the score estimateis bounded:
Revised: , T − t K } , the error in the score estimate is bounded:

Original: [Equation]
Revised: [Equation]

Original: (11) for some non-decreasing function ε 2 ∞ ,t .
Revised: for some non-decreasing function ε 2 ∞ ,t .

Original: In Section D, we will relax this condition to score function being accurate in L 2 .
Revised: In Section E, we will relax this condition to score function being accurate in L 2 .

Original: First, we construct the following continuous-time process which interpolates the discrete-timeprocess (5), for t ∈ [ t k , t k +1 ] :
Revised: First, we construct the following continuous-time process which interpolates the discrete-time process (5), for t ∈ [ t k , t k +1 ] :

Original: [Equation]
Revised: [Equation]

Original: Integration gives that
Revised: Integration gives that

Original: [Equation]
Revised: [Equation]

Original: where G t ′ ,t is defined in (6).
Revised: where G t ′ ,t is defined in (6).

Original: Letting q t be the distribution of z t and p t be the distribution of x t , we have by [LLT22, Lemma A.2]that [Equation]
Revised: Letting q t be the distribution of z t and p t be the distribution of x t , we have by [LLT22, Lemma A.2] that

Original: [Equation]
Revised: [Equation]

Original: (Note that in our case, (cid:98) f also depends on z t rather than just z t k , but this does not change thecalculation.)
Revised: (Note that in our case, (cid:98) f also depends on z t rather than just z t k , but this does not change the calculation.)

Original: Define ϕ t , ψ t as in (8): ϕ t ( x) = q t ( x ) p [Equation] x
Revised: Define ϕ t , ψ t as in (8): ϕ t ( x

Original: [Equation]
Revised: [Equation]

Original: , ψ

Original: [Equation] ( x [Equation] = 
Revised: = q t

Original: [Equation]
Revised: [Equation]

Original: x
Revised: ψ

Original: [Equation]
Revised: [Equation]

Original: pt ϕ 2 t .

Original: To bound (14), we use the following lemma.
Revised: To bound (14), we use the following lemma.

Original: Lemma A.1 (cf.
Revised: Lemma B.1 (cf.

Original: [EHZ21, Lemma 1], [LLT22, Lemma A.3]) .
Revised: [EHZ21, Lemma 1], [LLT22, Lemma A.3]) .

Original: For any C > 0 and any R d -valuedrandom variable u , we have
Revised: For any C > 0 and any R d -valued random variable u , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Young’s inequality,
Revised: By Young’s inequality,

Original: [Equation]
Revised: [Equation]

Original: Lemma A.2.
Revised: Lemma B.2.

Original: Suppose that (11) holds for t =
Revised: Suppose that (11) holds for t =

Original: T − t k , ∇ ln p t k ( x ) is L T − t k -Lipschitz, g is nondecreasing, and that h k ≤
Revised: T − t k , ∇ ln p t k ( x ) is L T − t k -Lipschitz, g is nondecreasing

Original: [Equation]
Revised: [Equation]

Original: tk g
Revised: and that h k ≤

Original: [Equation]
Revised: [Equation]

Original: T − t k ) 2 .

Original: Then we have for t ∈ [ t k , t k +1 ] that
Revised: Then we have for t ∈ [ t k , t k +1 ] that

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We bound the second term on the RHS of (14).
Revised: We bound the second term on the RHS of (14).

Original: By Lemma A.1,
Revised: By Lemma B.1,

Original: [Equation]
Revised: [Equation]

Original: Now
Revised: Now

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: by definition of ε ∞ ,t , so by Lemma A.3,
Revised: by definition of ε ∞ ,t , so by Lemma B.3,

Original: [Equation]
Revised: [Equation]

Original: The condition on h k and the fact that g is non-decreasing implies 192 G 2 t k ,t L 2 T − t k ≤ 12 .
Revised: The condition on h k and the fact that g is non-decreasing implies 192 G [Equation] k [Equation]

Original: Rearranginggives
Revised: t L 2 T − [Equation] k ≤ 12 . Rearranging gives

Original: [Equation]
Revised: [Equation]

Original: Substituting into (15) and that inequality into (14) give the conclusion.
Revised: Substituting into (15) and that inequality into (14) give the conclusion

Revised: [Equation]

Original: Lemma A.3.
Revised: Lemma B

Original:  Suppose that h k ≤ 1 2 g( T − t k ) 2 .
Revised: [Equation] Suppose that h k ≤ [Equation] g

Revised: [Equation]

Revised: T − t k ) 2 .

Original: Then for t ∈ [ t k +1 , t k ] ,
Revised: Then for t ∈ [ t k +1 , t k ] ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Consider (13).
Revised: Consider (13).

Original: The assumption on h k implies G t k ,t ≤ G t k ,t k +1 ≤ 12 , so exp (cid:0) 12 G t k ,t (cid:1) − 1 ≤257 G t k ,t .
Revised: The assumption on h k implies G t k ,t ≤ G t k ,t k +1 ≤ 12 , so exp (cid:0) 12 G t k [Equation] t (cid:1) − 1 ≤ G t k ,t .

Original: Let Y denote the last term of (13).
Revised: Let Y denote the last term of (13).

Original: Then∥
Revised: Then

Original: [Equation]
Revised: [Equation]

Revised: ∥ Y ∥ .

Original: Again using G t k ,t ≤ 12 , rearranging gives
Revised: Again using G t k ,t ≤ 12 , rearranging gives

Original: [Equation]
Revised: [Equation]

Original:  and
Revised: , and

Original: [Equation]
Revised: [Equation]

Original: By Lemma A.4,
Revised: By Lemma B.4,

Original: E [ ψ t ( z

Original: [Equation]
Revised: [Equation]

Original: ∥
Revised: The lemma follows.

Original: Y ∥ 2 ] ≤ 4 G [Equation] 8 KL( ψ
Revised: Lemma B.4. For t ∈ [ t k , t k +1 ] ,

Original: [Equation]
Revised: [Equation]

Original: q t || p t
Revised: exp

Original: [Equation]
Revised: [Equation]

Original: + 2 d

Original: + 16 ln2) . The lemma follows. Lemma A.4. For t ∈ [ t k , t k +1 ] ,
Revised: ln

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Note that Y :
Revised: Note that Y :

Original: = (cid:82) t t k exp (cid:16) 12 (cid:82) t ′ t k g
Revised: = (cid:82) t t k exp [Equation]

Original: ( T − t ′′ ) 2 dt ′′ (cid:17) g ( t ′ ) dw t ′ is a Gaussian random vector withvariance(cid:90) t t k exp (cid:32)(cid:90) t ′ t k g
Revised: [Equation]

Original: ( T − t ′′
Revised: t [Equation] −

Original: [Equation]
Revised: [Equation]

Original:  2 dt ′′ (cid:33)
Revised: ) 2 dt [Equation]

Revised: g ( t ′ ) dw t ′ is a Gaussian random vector with variance

Original: [Equation]
Revised: [Equation]

Original: (Note that this calculation shows that the continuous-time process (12) does agree with the discretetime process (5) at t = t k +1 .)
Revised: (Note that this calculation shows that the continuous-time process (12) does agree with the discretetime process (5) at t = t k +1 .)

Original: Using the Donsker-Varadhan variational principle, for any randomvariable X ,
Revised: Using the Donsker-Varadhan variational principle, for any random variable X ,

Original: [Equation]
Revised: [Equation]

Original: Applying this to X = c ( ∥ Y ∥ − E ∥
Revised: Applying this to X = c ( ∥ Y ∥ − E ∥

Original: Y ∥ ) 2 for a constant c > 0 to be chosen later, and (cid:101) P such thatd
Revised: Y ∥ ) 2 for a constant c > 0 to be chosen later, and (cid:101) P such that d

Original: [Equation]
Revised: [Equation]

Original: P d P ( z t ) =
Revised: P d P ( z t ) =

Original: ψ t ( z t ) , we can bound
Revised: ψ t ( z t ) , we can bound

Original: [Equation]
Revised: [Equation]

Original: Now following [Che+21, Theorem 4], we set c =1 8(exp( G tk,t ) − 1) , so that
Revised: Now following [Che+21, Theorem 4] [Equation] we set c =

Original: [Equation]
Revised: [Equation]

Original: 1) (cid:21) ≤ 2 .
Revised: , so that [Equation]

Original: Next, we have
Revised: Next, we have

Original: [Equation]
Revised: [Equation]

Original: Noting that E p t ϕ 2 t = χ 2 ( q t || p t ) + 1 ≥ 1 , we have that
Revised: Noting that E p t ϕ 2 t = χ 2 ( q t || p t ) + 1 ≥ 1 , we have that

Original: [Equation]
Revised: [Equation]

Original: Substituting everything into (17) gives the desired inequality.
Revised: Substituting everything into (17) gives the desired inequality.

Original: Let
Revised: Let

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: In order to bound the RHS in Lemma A.2, we need to bound all four of these quantities, which we doin Lemma A.5, A.6, A.8, and Section B, respectively.
Revised: In order to bound the RHS in Lemma B.2, we need to bound all four of these quantities, which we do in Lemma B.5, B.6, B.8, and Section C, respectively.

Original: The main innovation in our analysis comparedto [LLT22] is a new way to bound K , which we present in a separate section.
Revised: The main innovation in our analysis compared to [LLT22] is a new way to bound K , which we present in a separate section.

Original: First we bound K z .
Revised: First we bound K z .

Original: Recall the norm
Revised: Recall the norm

Original: [Equation]
Revised: [Equation]

Original: (In other words, this is the usual Orlicz norm applied to ∥ X ∥ 2 .)
Revised: (In other words, this is the usual Orlicz norm applied to ∥ X ∥ 2 .)

Original: Lemma A.5.
Revised: Lemma B.5.

Original: For t ∈ [ t k , t k +1 ] ,
Revised: For t ∈ [ t k , t k +1 ] ,

Original: [Equation]
Revised: [Equation]

Original: p

Original: [Equation]

Original: +

Original: [Equation]

Original: [Equation]

Original: Proof.
Revised: Proof.

Original: By the Donsker-Varadhan variational principle,
Revised: By the Donsker-Varadhan variational principle,

Original: [Equation]
Revised: [Equation]

Original: for any s > 0 .
Revised: for any s > 0 .

Original: Choosing s = 2 ∥ x t ∥ − 2 2 ,ψ 2 , we have E p t (cid:104)
Revised: Choosing s = 2 ∥ x t ∥ − 2 2 ,ψ 2 , we have E p t (cid:104)

Original: e s 2 ∥ x ∥ 2 (cid:105) ≤ 2 , which gives the desiredinequality.
Revised: e s 2 ∥ x ∥ 2 (cid:105) ≤ 2 , which gives the desired inequality.

Original: The following bounds K V ; note that the proof does not depend on the definition of q t , only that it is aprobability density.
Revised: The following bounds K V ; note that the proof does not depend on the definition of q t , only that it is a probability density.

Original: Lemma A.6 ([LLT22, Corollary C.7], [Che+21, Lemma 16]) .
Revised: Lemma B.6 ([LLT22, Corollary C.7], [Che+21, Lemma 16]) .

Original: [Equation]
Revised: [Equation]

Original: We use the following lemma to bound K ∆ V in Lemma A.8.
Revised: We use the following lemma to bound K ∆ V in Lemma B.8.

Original: Lemma A.7 ([LLT22, Lemma C.12]) .
Revised: Lemma B.7 ([LLT22, Lemma C.12]) .

Original: Suppose that p ( x ) ∝ e − V ( x ) is a probability density on
Revised: Suppose that p ( x ) ∝ e − V ( x ) is a probability density on

Original: R d , where V ( x ) is L -smooth.
Revised: R d , where V ( x ) is L -smooth.

Original: Let p α ( x ) =
Revised: Let p α ( x ) =

Original: α d p ( αx ) and φ σ 2 ( x ) denote the density function of
Revised: α d p ( αx ) and φ σ 2 ( x ) denote the density function of

Original: N (0 , σ 2 I d ) .
Revised: N (0 , σ 2 I d ) .

Original: Then for σ 2 ≤ [Equation]
Revised: Then for σ 2 ≤ 1 2 α 2 L , (cid:13)(cid:13)(cid:13)(cid:13) ∇

Original: [Equation]
Revised: [Equation]

Original: Lemma A.8.
Revised: Lemma B.8.

Original: Suppose that h k ≤ 1 4 Lg ( T − t k ) 2 where ∇ ln p t is L T − t -smooth ( L T − t ≥ 1 ) and
Revised: Suppose that h k ≤ 1 4 Lg ( T − t k ) 2 where ∇ ln p t is L T − t -smooth ( L T − t ≥ 1 ) and

Original: L = max t ∈ [ t k ,t k +1 ]
Revised: L = max t ∈ [ t k ,t k +1 ]

Original: L T − t .
Revised: L T − t .

Original: For t ∈ [ t k , t k +1 ] ,
Revised: For t ∈ [ t k , t k +1 ] ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We have the following relationship for t ∈ [ t k , t k +1 ] : 
Revised: We have the following relationship for t ∈ [ t k , t k +1 ] : p [Equation] p

Original: [Equation]
Revised: [Equation]

Original: where p α ( x ) = α d p ( αx ) , α = e 12 (cid:82) ttk g ( T − s ) 2 ds and σ 2 = 1 − e − (cid:82)
Revised: where p α ( x ) = α d p ( αx ) , α = e 12 (cid:82) ttk g ( T − s ) 2 ds and σ 2 = 1 − e − (cid:82)

Original: ttk g ( T − s [Equation] 2 ds .
Revised: ttk g ( T − s ) 2 ds .

Original: Observe that sinceh
Revised: Observe that since h k ≤ 1 4 g

Original: [Equation]
Revised: [Equation]

Revised: T − t k ) 2 ,

Original: [Equation]
Revised: [Equation]

Original: We note that
Revised: We note that

Original: [Equation]
Revised: [Equation]

Original: so the hypothesis of Lemma A.7 is satisfied.
Revised: so the hypothesis of Lemma B.7 is satisfied.

Original: Using Lemma A.
Revised: Using Lemma B

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: we obtain
Revised: we obtain

Revised: E

Original: [Equation]
Revised: [Equation]

Original: Now we put everything together.
Revised: Now we put everything together.

Original: Write G t =
Revised: Write G t =

Original: G t k ,t for short.
Revised: G t k ,t for short.

Original: Suppose L t is non-increasing.
Revised: Suppose L t is non-increasing.

Original: By
Revised: By

Original: Lemma A.2,
Revised: Lemma B.2,

Original: [Equation]
Revised: [Equation]

Original: where E ≤ 16 G 2 t L 2 T − t k
Revised: T − [Equation] q [Equation]

Original: ( K z + 16 K V ) + 64 G t L 2 T − t k (8 K + 2 d
Revised: p [Equation] + [Equation]

Original:  + 16 ln 2) + ε 2 ∞ ,T − t k + K ∆ V .
Revised: E where E ≤ 16 G 2 t L 2 T − t k ( K z + 16 K V ) + 64 G t L 2 T − t k (8 K + 2 d + 16 ln 2) + ε 2 ∞ ,T − t k + K ∆ V .

Original: By Lemma A.8, K ∆ V ≤ 25 L 2 T − t (8 G t d + G 2 t K z ) + 100 L 2 T − t G 2 t K V , so 
Revised: By Lemma B.8, K ∆ V ≤ 25 L 2 T − t (8 G t d + G 2 t K z ) + 100 L [Equation] − t G 2 t K V , so E ≤ 41 G

Revised: [Equation]

Original: E
Revised: L

Original: [Equation]
Revised: [Equation]

Original: k .
Revised: K [Equation] [Equation] K

Revised: [Equation]

Original: By Lemma A.5, K z ≤ ∥ x t ∥ 22 ,ψ 2 ( K + ln 2) , and by Corollary A.6, K V ≤
Revised: + 64 G [Equation] L [Equation]

Revised: [Equation]

Revised: +

Revised: [Equation]

Original: {++ 
+}
Revised: {++ 
+}

Revised: ln

Original: [Equation]
Revised: [Equation]

Original:  2 dL , so
Revised: ) + ε [Equation] By Lemma B.5, K z ≤ ∥ x t ∥ 22 ,ψ 2 ( K + ln 2) , and by Corollary B.6, K V ≤

Original: [Equation]
Revised: [Equation]

Original: + 2 dL (cid:19) + 64 G t L 2 T − t (8 K + 6 d + 16 ln 2) + ε 2 ∞ ,T − t k . Now, if h k ≤ ε ′ hk 20 g ( T − t k ) 2 L T − tk +1 , then
Revised: E

Original: E ≤ ε ′ h k(cid:20)
Revised: [Equation] (cid:16)

Original: ∥ x t ∥ 22 ,ψ 2 (

Original: [Equation]
Revised: [Equation]

Original: + ln
Revised: + 2 dL , so E

Original: [Equation]
Revised: [Equation]

Original: ) + (cid:18)

Original: [Equation]
Revised: [Equation]

Original: · E

Original: [Equation]
Revised: [Equation]

Original: t (cid:18)q
Revised: (cid:18)

Original: [Equation]
Revised: [Equation]

Original: p
Revised: dL

Original: [Equation]
Revised: [Equation]

Original: (cid:19) + 2 dL T − t (cid:19)(cid:21) + 4 ε ′ h k L T − t (8 K +2 d
Revised: + 64 G [Equation] (8 K +

Original:  + 16 ln 2) + ε 2 ∞ ,T − t k . Let M T − t :=
Revised: [Equation] + 16 ln 2) + ε [Equation]

Original: ∥ x t ∥ 22 ,ψ 2 . Assume that K ≤ 
Revised: Now, if h k ≤ [Equation] , then

Original: A T − t χ 2 ( q t || p t )+1 + B T − t . Then we obtain12 g ( T − t ) [Equation] χ
Revised: E

Original: [Equation]
Revised: [Equation]

Original: ( q t [Equation] p t ) + 1)
Revised: Let M T − t :=

Original: · E [Equation] [Equation] p
Revised: ∥ x t ∥ 22 ,ψ 2 .

Original: [Equation] ) + ( χ [Equation] [Equation]
Revised: Assume that K ≤

Original:  t || pt ) + [Equation] )( ε ′ h k 2 · ((
Revised: A T − t χ 2 ( q t || p

Original: [Equation]
Revised: [Equation]

Original:  T [Equation] t + ln
Revised: )+1 + B T − t .

Original: [Equation] [Equation] T − t +
Revised: Then we obtain

Original: [Equation]
Revised: [Equation]

Original: dL )

Original: + ε ′

Original: h k · L T − t (8 B T − t + 6 d + 16 ln 2)) + ε 2 ∞ ,T − t k (cid:105) . If
Revised: If ε ′ h k ≤ min

Original: [Equation]
Revised: [Equation]

Original: , then
Revised: , then

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised: If ε ′ h k ≤ min

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: we get 
Revised: we get d

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: Integration gives
Revised: Integration gives

Original: [Equation]
Revised: [Equation]

Original: ε 2 T − t g ( T − t ) 2 dt.

Original: Taking ε ′ =
Revised: Taking ε ′ =

Original: ε ln (cid:16) T T − tN (cid:17) then gives the following Theorem A.10.
Revised: ε ln (cid:16) T T − tN (cid:17) then gives the following Theorem B.10.

Original: We first introduce a technicalassumption.
Revised: We first introduce a technical assumption.

Original: Definition A.9.
Revised: Definition B.9.

Original: Let f : R > 

Revised: Let f : R >

Original: → R > 0 .
Revised: → R > 0 .

Original: We say that f has at most power growth and decay (with someconstant c > 0 ) 
Revised: We say that f has at most power growth and decay (with some constant c > 0 ) if max

Original: if max u ∈ [ t 2 ,t ] f ( u ) ∈ (cid:104) f ( t ) c , cf ( t ) (cid:105) .
Revised: [Equation]

Original: Theorem A.10.
Revised: Theorem B.10.

Original: Suppose that the following hold.1.
Revised: Suppose that the following hold.

Original: Assumption 5 holds for ε ∞ ,t . 
Revised: Assumption 5 holds for ε ∞ ,t . 2.

Original: [Equation]
Revised: ∥ (cid:101) x t ∥ 22 ,ψ 2 ≤ M t .

Original: The KL bound KL( ψ t q t || p t ) ≤
Revised: The KL bound KL( ψ t q t || p t ) ≤

Original: [Equation]
Revised: [Equation]

Original: B 
Revised: B T

Original: [Equation]
Revised: [Equation]

Original:  holds for any density q t and t < t N ,where ψ t ( x ) =
Revised: t holds for any density q t and t < t N , where ψ t ( x ) =

Original: q t ( x ) /p t ( x ) χ 2 ( q t || p t )+1 .4.
Revised: q t ( x ) /p t ( x ) χ 2 ( q t [Equation] p t ) +1 .

Original: g ( t ) , A t , B t , L t , M t have at most polynomial growth and decay (with some constant c ).
Revised: g ( t ) , A t , B t , L t , M t have at most polynomial growth and decay (with some constant c ).

Original: Then there is some constant c ′ (depending on c ) such that if the step sizes satisfy
Revised: Then there is some constant c ′ (depending on c ) such that if the step sizes satisfy

Original: [Equation]
Revised: [Equation]

Original: k [Equation] min
Revised: ≤ min

Original: [Equation]
Revised: [Equation]

Original: t k
Revised: −

Original: [Equation]
Revised: [Equation]

Original: where ε ′ h
Revised: where ε ′ h

Original: k = min
Revised: k = min

Original: [Equation]

Original: g

Original: [Equation]
Revised: [Equation]

Original: then for 0 ≤ k ≤ N ,
Revised: then for 0 ≤ k ≤ N ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: This follows from the above calculations and the observation that if we replace F ( T − t )by F ( T − t k ) , for some F satisfying the power growth and decay assumption, then we change thebound by at most a constant factor, because the step size satisfies h k
Revised: This follows from the above calculations and the observation that if we replace F ( T − t ) by F ( T − t k ) , for some F satisfying the power growth and decay assumption, then we change the bound by at most a constant factor, because the step size satisfies h k

Original: = t
Revised: = t

Original: k +1 − t k ≤ T − t k 2 .
Revised: k +1 − t k ≤ T − t k 2 .

Original: We specialize this theorem in the case of distributions with bounded support.
Revised: We specialize this theorem in the case of distributions with bounded support.

Original: Note that although notevery initial distribution (cid:101) p t may satisfy a KL inequality as required by condition 3 of Theorem 30,
Revised: Note that although not every initial distribution (cid:101) p t may satisfy a KL inequality as required by condition 3 of Theorem 30,

Original: Lemma B.2 will give the existence of a distribution that does, and is close in TV-error.
Revised: Lemma C.2 will give the existence of a distribution that does, and is close in TV-error.

Original: Later in
Revised: Later in

Original: Section C, we show that this will have a small effect on the score function, and hence allow us toprove our main theorems.
Revised: Section D, we show that this will have a small effect on the score function, and hence allow us to prove our main theorems.

Original: Corollary A.11.
Revised: Corollary B.11.

Original: Suppose that Assumptions 5 and 2 hold, R 2 ≥ d , g ≡ 1 , and that (cid:101) P 0is such that the KL inequality (30) holds.
Revised: Suppose that Assumptions 5 and 2 hold, R 2 ≥ d , g ≡ 1 , and that (cid:101) Pis such that the KL inequality (30) holds.

Original: Let δ = T − t N .
Revised: Let δ = T − t N .

Original: If 0 < δ, ε < 12 , h k 
Revised: If 0 < δ, ε < 12 , h k =

Original: [Equation]
Revised: O

Original: [Equation]
Revised: [Equation]

Original: , then for any 0 ≤ k ≤ N
Revised: , then for any 0 ≤ k ≤ N

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: For g ≡ 1 , note that σ 2 T − t =
Revised: For g ≡ 1 , note that σ 2 T − t =

Original: Θ(min { T − t, 1 } ) .
Revised: Θ(min { T − t, 1 } ) .

Original: From Lemma A.13, we can choose
Revised: From Lemma B.13, we can choose

Original: [Equation]
Revised: [Equation]

Revised: (cid:18)

Revised: [Equation]

Revised: { ( T

Revised: [Equation]

Revised: [Equation]

Original: From Lemma A.15, we can choose
Revised: From Lemma B.15, we can choose

Original: [Equation]
Revised: [Equation]

Original: The KL inequality (30) gives us
Revised: The KL inequality (30) gives us

Original: [Equation]
Revised: [Equation]

Original: We now check the requirements on h k .
Revised: We now check the requirements on h k .

Original: We needε ′
Revised: We need

Original: h k

Original: = O (cid:32) (cid:112) A T − t k M T − t k +(cid:33) ⇐ = ε ′ h
Revised: [Equation]

Original: k

Original: = O(cid:18) max { R, √ d } (cid:19) (22) ε ′ h k = O (cid:18)
Revised: O

Original: L T − t k A T − t k (cid:19)

Original: ⇐ = ε ′ h

Original: k

Original: = O (cid:18) T − t k R(cid:19) (23) ε ′ h k
Revised: [Equation]

Original: = O   (cid:113) ε/ ln (cid:0)

Original: Tδ (cid:1) (cid:112) ( T − t k )(( B T − t k + 1) M T − t k + dL T − t k ) 

Original:  .

Original: For T − t k ≤ 1 , (24) is implied byε
Revised: For T − t k ≤ 1 , (24) is implied by

Original: [Equation]
Revised: [Equation]

Original: O
Revised:  

Original: [Equation]
Revised: [Equation]

Original:  (cid:113) [Equation] [Equation] (cid:0) Tδ (cid:1) (cid:114)
Revised:  (cid:118)(cid:117)(cid:117)(cid:116)

Original: ( T − t k ) (cid:16) ln (cid:16) 1 ε K (cid:17)

Original: + d ln (cid:0) Rδ (cid:1)(cid:17) max { R 2 , d } + dR 2 T − t k  

Original: ⇐ = ε ′ h

Original: k

Original: = O  (cid:118)(cid:117)(cid:117)(cid:116) ε ( T − t k ) d max { R 2 , d } ln (cid:0)

Original: Tδ (cid:1) ln (cid:16) Rδε K (cid:17)   , and for T − t k > 1 ,ε ′ h

Original: k

Original: [-= O   (cid:113) ε/ ln (cid:0) Tδ (cid:1) (cid:113) T (cid:0) ln (cid:0) 
 ε (cid:1) + d ln (cid:0) R δ (cid:1)(cid:1) max { R 2 , d } + dR  ⇐ = ε ′ h-]
Revised: [-= O   (cid:113) ε/ ln (cid:0) Tδ (cid:1) (cid:113) T (cid:0) ln (cid:0) 
 ε (cid:1) + d ln (cid:0) R δ (cid:1)(cid:1) max { R 2 , d } + dR  ⇐ = ε ′ h-][Equation]

Original: k = O  (cid:115) ε Td
Revised: (cid:1)

Original: [Equation]
Revised: [Equation]

Original: Rδε

Original:  [Equation]  .
Revised: (cid:16) [Equation]

Revised:  , and for T − t k > 1 , ε

Revised: [Equation]

Revised: h k

Revised: [Equation]

Revised: O

Revised: [Equation]

Original: Finally, the last requirement is
Revised: Finally, the last requirement is

Original: [Equation]
Revised: [Equation]

Original: As long as R 2 = Ω( d ) and ε < 1 , the last equation implies all the others.
Revised: As long as R 2 = Ω( d ) and ε < 1 , the last equation implies all the others.

Original: Plugging this into
Revised: Plugging this into

Original: Theorem A.10 gives the result.
Revised: Theorem B.10 gives the result.

Original: Above, we use the Hessian bound (cid:13)(cid:13) ∇ 2 ln p t ( x ) (cid:13)(cid:13)
Revised: Above, we use the Hessian bound (cid:13)(cid:13) ∇ 2 ln p t ( x ) (cid:13)(cid:13)

Original: ≤ R 2 σ 4 t given in Lemma A.13.
Revised: ≤ R 2 σ 4 t given in Lemma B.13.

Original: Under the strongersmoothness assumption given by Assumption 3, we can take the step sizes to be larger.
Revised: Under the stronger smoothness assumption given by Assumption 3, we can take the step sizes to be larger.

Original: Corollary A.12.
Revised: Corollary B.12.

Original: Suppose that Assumptions 5, 2, 3 hold, C ≥ R 2 ≥ d , g ≡ 1 , and that (cid:101) P 0 issuch that the KL inequality (30) holds.
Revised: Suppose that Assumptions 5, 2, 3 hold, C ≥ R 2 ≥ d , g ≡ 1 , and that (cid:101) P 0 is such that the KL inequality (30) holds.

Original: Let δ = T − t N .
Revised: Let δ = T − t N .

Original: If 0 < δ, ep < 12 and ε < 1 / √ T ,h k
Revised: If 0 < δ, ep < 1 2 and ε < 1 / √ T , h k

Original: = O (cid:18)
Revised: = O (cid:18)

Revised: ε max

Original: [Equation]
Revised: [Equation]

Original:  , then for any 0 ≤ k ≤ N ,
Revised: (cid:19) , then for any 0 ≤ k ≤ N ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We instead have the bound L t =
Revised: We instead have the bound L t =

Original: Cσ 2 t .
Revised: Cσ 2 t .

Original: The requirement (22) stays the same, while (23) isimplied by ε′
Revised: The requirement (22) stays the same, while (23) is implied by ε

Original:  h k
Revised: ′ h k

Original: = O (1 /C ) .
Revised: = O (1 /C ) .

Original: Inequality (24), for T − t k ≤ 1 , is implied by
Revised: Inequality (24), for T − t k ≤ 1 , is implied by

Original: [Equation]

Original: O  (cid:118)(cid:117)(cid:117)(cid:116)

Original: [Equation]
Revised: [Equation]

Original:  

Original: [Equation]

Original: and for T − t k > 1 ,
Revised: and for T − t k > 1 ,

Original: [Equation]
Revised: [Equation]

Original: Finally, the last requirement is implied by
Revised: Finally, the last requirement is implied by

Original: [Equation]
Revised: [Equation]

Original: and for C ≥ R 2 , ε ≤ 1 / √ T , implies all the others.
Revised: and for C ≥ R 2 , ε ≤ 1 / √ T , implies all the others.

Original: A.1 Auxiliary bounds
Revised: B.1 Auxiliary bounds

Original: In this section we give bounds on the Hessian ( L t , Lemma A.13), initial χ 2 divergence χ 2 ( q 0 || p 0 )
Revised: In this section we give bounds on the Hessian ( L t , Lemma B.13), initial χ 2 divergence χ 2 ( q 0 || p 0 )

Original: (Lemma A.14), and Orlicz norm ( M t , Lemma A.15).
Revised: (Lemma B.14), and Orlicz norm ( M t , Lemma B.15).

Original: Lemma A.13 (Hessian bound) .
Revised: Lemma B.13 (Hessian bound) .

Original: Suppose that µ is a probability measure supported on a bounded set
Revised: Suppose that µ is a probability measure supported on a bounded set

Original: M ⊂ R d with radius R .
Revised: M ⊂ R d with radius R .

Original: Then letting φ σ 2 denote the density of N (0 , σ 2 I d ) ,
Revised: Then letting φ σ 2 denote the density of N (0 , σ 2 I d ) ,

Original: [Equation]
Revised: [Equation]

Original: Therefore, for (cid:101) P 0 supported on B R (0) ,  we have
Revised: Therefore, for (cid:101) P 0 supported on B R (0) , R ≥ 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let µ x,σ 2 denote the density µ ( du ) weighted with the gaussian φ σ 2 ( u − x ) , that is, 
Revised: Let µ x,σ 2 denote the density µ ( du ) weighted with the gaussian φ σ 2 ( u − x ) , that is, µ x,σ 2 ( du ) = e −∥ x − u ∥ 2 2 σ 2 µ

Original: [Equation]
Revised: [Equation]

Original:  µ [Equation]
Revised: σ 2 µ ( du ) .

Original: We note the following calculations:
Revised: We note the following calculations:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The covariance of a distribution supported on a set of radius R is bounded by R 2 in operator norm.
Revised: The covariance of a distribution supported on a set of radius R is bounded by R 2 in operator norm.

Original: Inequality (25) then follows from (28).
Revised: Inequality (25) then follows from (28).

Original: For (26), note that (cid:101) p t =
Revised: For (26), note that (cid:101) p t =

Original: M m t
Revised: M m t

Original: ♯ (cid:101) P 0 ∗ φ σ 2 t , where m t is given by (3) and M m denotes multiplication bym .
Revised: ♯ (cid:101) P 0 ∗ φ σ 2 t , where m t is given by (3) and M m denotes multiplication by m .

Original: Since M m t ♯ (cid:101) P 0 is supported on B m t R (0) ⊂ B R (0)  , the result follows.
Revised: Since M m t ♯ (cid:101) P 0 is supported on B m t R (0) ⊂ B R (0) and σ t ≤ 1 , the result follows.

Original: Lemma A.14 (Bound on initial χ 2 -divergence) .
Revised: Lemma B.14 (Bound on initial χ 2 -divergence) .

Original: Suppose that (cid:101) P 0 is supported on B R (0) .
Revised: Suppose that (cid:101) P 0 is supported on B R (0) .

Original: Letp prior =
Revised: Let p prior =

Original: N (0 , (1 − e G 0 ,t )
Revised: N (0 , (1 − e G 0 ,t )

Original: I d ) .
Revised: I d ) .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Revised: exp

Original: ,T )
Revised: [Equation] − G [Equation] [Equation]

Revised: 1 − exp

Original: [Equation]
Revised: [Equation]

Original: and for 0 < ε < 12 and G 0 ,T≥ ln (cid:16) 4 R 2 ε 2 (cid:17) ∨ 1 , we have χ 2 ( p prior || (cid:101) p T ) ≤ ε 2 .
Revised: − [Equation] 0 ,T ) (cid:21)

Original: Proof.
Revised: Proof.

Original: We have for x 0 ∼ (cid:101) P 0 that
Revised: We have for x 0 ∼ (cid:101) P 0 that

Original: [Equation]
Revised: [Equation]

Original: Using convexity of χ 2 -divergence then gives the result.
Revised: Using convexity of χ 2 -divergence then gives the result.

Original: For G 0 ,T ≥ ln (cid:16) 4 R 2 ε 2 (cid:17) ∨ 1 , we haveexp
Revised: For G 0 ,T ≥ ln (cid:16) 4 R 2 ε 2 (cid:17) ∨ 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Lemma A.15 (Subgaussian bound) .
Revised: Lemma B.15 (Subgaussian bound) .

Original: Suppose (cid:101) P 0 is supported on B R (0) .
Revised: Suppose (cid:101) P 0 is supported on B R (0) .

Original: Then for X ∼ (cid:101) p t , 
Revised: Then for X ∼ (cid:101) p t , ∥ X ∥ 2 ,ψ 2 ≤ (cid:114) e [Equation] max { [Equation]

Revised: √

Original: [Equation]
Revised: [Equation]

Original:  where m t , σ t are as in (3) and C 1 is an absolute constant.
Revised: , where m t , σ t are as in (3) and C 1 is an absolute constant.

Original: Proof.
Revised: Proof.

Original: Let Y ∼ (cid:101) P 0 s.t.
Revised: Let Y ∼ (cid:101) P 0 s.t.

Original: X
Revised: X

Original: = m t Y + σ t ξ for some ξ ∼ N (0 , I d ) independent of Y .
Revised: = m t Y + σ t ξ for some ξ ∼ N (0 , I d ) independent of Y .

Original: Define
Revised: Define

Revised: U =

Revised: ∥

Revised: X ∥ 2 := (cid:16)(cid:80) di

Original: {+=
+}
Revised: {+=
+}

Revised: X 2 i (cid:17) 1 / 2 , then for p ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: where Γ is the commonly used gamma function and C 1 is an absolute constant.
Revised: where Γ is the commonly used gamma function and C 1 is an absolute constant.

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: where K = 2 m t R + 3 C 1 σ t √ d .
Revised: where K = 2 m t R + 3 C 1 σ t √ d .

Original: Now consider V
Revised: Now consider V

Original: = U/K , then for some λ > 0 small enough, by
Revised: = U/K , then for some λ > 0 small enough, by

Original: Taylor expansion,
Revised: Taylor expansion,

Original: [Equation]
Revised: [Equation]

Original: Note that E (cid:2) V 2 p (cid:3) ≤ (2 p ) p , while Stirling’s approximation yields p ! ≥ ( p/e ) p .
Revised: Note that E (cid:2) V 2 p (cid:3) ≤ (2 p ) p , while Stirling’s approximation yields p ! ≥ ( p/e ) p .

Original: Substituting thesetwo bounds, we get
Revised: Substituting these two bounds, we get

Original: [Equation]
Revised: [Equation]

Original: provided that 2 eλ 2 < 1 , in which case the geometric series above converges.
Revised: provided that 2 eλ 2 < 1 , in which case the geometric series above converges.

Original: To bound this quantityfurther, we can use the numeric inequality 1 / (1 − x ) ≤ e 2 x which is valid for x ∈ [0 , 1 / 2] .
Revised: To bound this quantity further [Equation] we can use the numeric inequality 1 / (1 − x ) ≤ e 2 x which is valid for x ∈ [0 , 1 / 2] .

Original: It followsthat
Revised: It follows that

Original: [Equation]
Revised: [Equation]

Original:  all λ satisfying | [Equation] | [Equation]
Revised: 4 eλ 2 for all λ satisfying | λ | ≤ 1 / 2 √ e.

Original: Now set 4 eλ 2 =
Revised: Now set 4 eλ 2 =

Original: ln 2 , then
Revised: ln 2 , then

Original: [Equation]
Revised: [Equation]

Original: which implies that
Revised: which implies that

Original: [Equation]
Revised: [Equation]

Original: B Bounding the KL divergence
Revised: C Bounding the KL divergence

Original: In this section, we bound the quantity K = KL( ψ t q t || p t ) , where ψ t is as in (8).
Revised: In this section, we bound the quantity K = KL( ψ t q t || p t ) , where ψ t is as in (8).

Original: While p t is definedby the DDPM process, in this section we do not assume q t is the density of the discretized process;rather, it is any density for which E p t (cid:16) q t p t (cid:17) and χ 2 ( q t || p t ) are finite.
Revised: While p t is defined by the DDPM process, in this section we do not assume q t is the density of the discretized process; rather, it is any density for which E p t (cid:16) q t p t (cid:17) and χ 2 ( q t || p t ) are finite.

Original: Lemma B.1.
Revised: Lemma C.1.

Original: Suppose that (cid:101) P 0 is a probability measure on R d such that
Revised: Suppose that (cid:101) P 0 is a probability measure on R d such that

Original: [Equation]
Revised: [Equation]

Original: where w j > 0 , (cid:80) mj
Revised: where w j > 0 , (cid:80) mj

Original: =1 w j = 1 , and each (cid:101) P j, 0 is a probability measure.
Revised: =1 w j = 1 , and each (cid:101) P j, 0 is a probability measure.

Original: For t > 0 , let (cid:101) p t and (cid:101) p j,tbe the densities obtained by running the forward DDPM process (1) for time t , and p t = (cid:101) p T − t ,p j,t = (cid:101) p j,T − t .
Revised: For t > 0 , let (cid:101) p t and (cid:101) p j,t be the densities obtained by running the forward DDPM process (1) for time t , and p t = (cid:101) p T − t , p j,t = (cid:101) p j,T − t .

Original: Let w min =
Revised: Let w min =

Original: min 1 ≤ j ≤ m w j and suppose all the (cid:101) P j,t satisfy a log-Sobolev constant390 with constant C t .
Revised: min 1 ≤ j ≤ m w j and suppose all the (cid:101) P j,t satisfy a log-Sobolev constant with constant C t .

Original: Then for any q t , where ψ t is as in (8)
Revised: Then for any q t , where ψ t is as in (8)

Original: [Equation]
Revised: [Equation]

Original: While we need p t to satisfy a log-Sobolev inequality to get a bound of the form
Revised: While we need p t to satisfy a log-Sobolev inequality to get a bound of the form

Original: [Equation]
Revised: [Equation]

Original: ([LLT22, Lemma C.8]), we note that if we allow additive slack, it suffices for p t to be a mixture ofdistributions satisfying a log-Sobolev inequality, with the logarithm of the minimum mixture weightbounded below.
Revised: ([LLT22, Lemma C.8]), we note that if we allow additive slack, it suffices for p t to be a mixture of distributions satisfying a log-Sobolev inequality, with the logarithm of the minimum mixture weight bounded below.

Original: In Lemma B.2 we will see that we can almost decompose any distribution of boundedsupport in this manner, if we move a small amount of the mass.
Revised: In Lemma C.2 we will see that we can almost decompose any distribution of bounded support in this manner, if we move a small amount of the mass.

Original: Proof.
Revised: Proof.

Original: Let f t : [ m ] → R be the function
Revised: Let f t : [ m ] → R be the function

Original: [Equation]
Revised: [Equation]

Original: By decomposition of entropy and the fact that each P i,t satisfies LSI with constant C T − t ,
Revised: By decomposition of entropy and the fact that each P i,t satisfies LSI with constant C T − t ,

Original: [Equation]
Revised: [Equation]

Original: f

Original: [Equation]

Original: f

Original: [Equation]

Original: f

Original: [Equation]

Original: (cid:13)(cid:13)(cid:13)(cid:13)

Original: [Equation] ( x ) [Equation] ( x ) p
Revised: ψ t ( x ) q t ( x ) dx + Ent w ( f t ) = 2 C t (cid:90)

Original: [Equation]

Original:  x ) (cid:13)(cid:13)(cid:13)(cid:13) [Equation] ( x ) q t ( x ) dx [Equation]
Revised: R d (cid:13)(cid:13)(cid:13)(cid:13) ∇ q t ( x ) p t ( x ) (cid:13)(cid:13)(cid:13)(cid:13) 2 ψ t ( x ) p t ( x )q t ( x ) dx + Ent w ( f t ) =

Original: Ent

Original: [Equation]

Original: ( f t ) =

Original: [Equation]

Original:  (cid:90) R [Equation] (cid:13)(cid:13)(cid:13)(cid:13)
Revised: 2 C t χ 2 ( q t || p t ) + 1 · (cid:90) (cid:13)(cid:13)(cid:13)(cid:13) ∇ q t ( x ) p t ( x ) (cid:13)(cid:13)(cid:13)(cid:13)

Original: [Equation] ln q t ( x )  p [Equation] x ) (cid:13)(cid:13)(cid:13)(cid:13)
Revised: p t ( x ) dx + Ent w ( f t ) ≤ 2 C t χ 2 ( q t || p t ) + 1 · E p t (cid:18) q t p t (cid:19)

Original: [Equation]

Original: x )

Original: [Equation]

Original: ( x ) dx

Original: [Equation]

Original: Ent [Equation] ( f t ) = [Equation]
Revised: + ln

Original: (cid:90)

Original: R d (cid:13)(cid:13)(cid:13)(cid:13) ∇ q t ( x ) p

Original: [Equation]

Original: [-( x ) (cid:13) (cid:13) (cid:13)(cid:13) 
 ψ t ( x ) p t ( x )q t [Equation]-] where the last inequality follows from noting w j f t ( j ) is a probability mass function on [ m ] , so thatf t ( j ) ≤ 1 w j and
Revised: [-( x ) (cid:13) (cid:13) (cid:13)(cid:13) 
 ψ t ( x ) p t ( x )q t [Equation]-](cid:18)w min (cid:19) , where the last inequality follows from noting w j f t ( j ) is a probability mass function on [ m ] , so that f t ( j ) ≤ 1 w j and

Original: [Equation]
Revised: [Equation]

Original: Lemma B.2.
Revised: Lemma C.2.

Original: Suppose 0 < ε K < 12 , and that P 0 is a probability measure such that P 0 ( M ) ≥ 1 − ε K 8 .
Revised: Suppose 0 < ε K < 12 , and that P 0 is a probability measure such that P 0 ( M ) ≥ 1 − ε K 8 .

Original: Let N (cid:0) M , σ t 2 (cid:1) denote the covering number of M with balls of radius σ t .
Revised: Let N (cid:0) M , σ t 2 (cid:1) denote the covering number of M with balls of radius σ t .

Original: Given δ > 0 , there existsa distribution (cid:101) P 0 such that χ 2 ( (cid:101) P 0 || P 0 ) ≤ ε K and considering the DDPM process started with (cid:101) P 0 ,for all 0 ≤ t ≤ T − δ ,[Equation] p [Equation]
Revised: Given δ > 0 , there exists a distribution (cid:101) P 0 such that χ 2 ( (cid:101) P 0 || P 0 ) ≤ ε K and considering the DDPM process started with (cid:101) P 0 , for all 0 ≤ t ≤ T − δ ,

Original: ≤

Original: [Equation]
Revised: [Equation]

Original: In particular, for M = B R (0) in R d ,
Revised: In particular, for M = B R (0) in R d ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Partition M into disjoint subsets M j , 1 ≤ j ≤ N := N ( M , σ δ / 2) of diameter at most σ δ ,and decompose
Revised: Partition M into disjoint subsets M j , 1 ≤ j ≤ N := N ( M , σ δ / 2) of diameter at most σ δ , and decompose

Original: [Equation]
Revised: [Equation]

Original: where p j is supported on M j and P ∗
Revised: where p j is supported on M j and P ∗

Original: =
Revised: =

Original: P 0 ( ·|M c ) .
Revised: P 0 ( ·|M c ) .

Original: We will zero out the coefficients of all smallcomponents: let Z = (cid:80) j : w j ≥ εK 8 N w j and
Revised: We will zero out the coefficients of all small components: let Z = (cid:80) j : w j ≥ εK 8 N w j and

Original: [Equation]
Revised: [Equation]

Revised: 0 , otherwise, and define

Original: [Equation]
Revised: [Equation]

Original: and define
Revised: Note that Z ≥ 1 − ε K 8 −

Original: [Equation]
Revised: [Equation]

Original: Note that Z ≥ 1 − ε K 8 − (cid:80) j : w j ≤ εK 8 N ≥ 1 − ε K 4 .
Revised: j : w j ≤ εK 8 N ≥ 1 − ε K 4 .

Original: As probability distributions on [ m ] ∪ {∗} ,χ [Equation] w | [Equation]
Revised: As probability distributions on [ m ] ∪ {∗} ,

Original: ≤

Original: [Equation]
Revised: [Equation]

Original: and hence the same bound holds for χ 2 ( (cid:101) P 0 || P 0 ) .
Revised: and hence the same bound holds for χ 2 ( (cid:101) P 0 || P 0 ) .

Original: Note each M m t ♯ (cid:101) P j, 0 is supported on a set ofdiameter m t σ ≤ σ .
Revised: Note each M m t ♯ (cid:101) P j, 0 is supported on a set of diameter m t σ ≤ σ .

Original: By Theorem 1 of [CCN21], noting that
Revised: By Theorem 1 of [CCN21], noting that

Original: [Equation]
Revised: [Equation]

Original: when Σ = σ 2 I and ∥ µ 2 − µ 1 ∥ ≤ σ , (cid:101) P j,t =
Revised: when Σ = σ 2 I and ∥ µ 2 − µ 1 ∥ ≤ σ , (cid:101) P j,t =

Original: ( M m t ♯ (cid:101) P j, 0 ) ∗ φ σ 2 satisfies a log-Sobolev inequalitywith constant 6(1 + e ) σ 2 t .
Revised: ( M m t ♯ (cid:101) P j, 0 ) ∗ φ σ 2 satisfies a log-Sobolev inequality with constant 6(1 + e ) σ 2 t .

Original: The result then follows from Lemma B.1.
Revised: The result then follows from Lemma C.1.

Original: For M = B R (0) , we use thebound N ( B R (0) , σ δ / 2) ≤ (cid:16) 1 + 4 R σ δ (cid:17) d
Revised: For M = B R (0) , we use the bound N ( B R (0) , σ δ / 2) ≤ (cid:16) 1 + [Equation]

Original:  [Ver18, Corollary 4.2.13].
Revised: Rσ δ (cid:17) d [Ver18, Corollary 4.2.13].

Original: In the next section, we show that we can move a small amount of mass ε without significantlyaffecting the score function.
Revised: In the next section, we show that we can move a small amount of mass ε without significantly affecting the score function.

Original: This is necessary, as our guarantees on the score estimate are for theoriginal distribution and not the perturbed one in Lemma B.2.
Revised: This is necessary, as our guarantees on the score estimate are for the original distribution and not the perturbed one in Lemma C.2.

Original: C The effect of perturbing the data distribution on the score function
Revised: D The effect of perturbing the data distribution on the score function

Original: In this section we consider the effect of perturbing the data distribution on the score function.
Revised: In this section we consider the effect of perturbing the data distribution on the score function.

Original: Thekey observation is that the score function can be interpreted as the solution to an inference problem,that of recovering the original data point from a noisy sample, with data distribution as the givenprior distribution.
Revised: The key observation is that the score function can be interpreted as the solution to an inference problem, that of recovering the original data point from a noisy sample, with data distribution as the given prior distribution.

Original: We show through a coupling argument that we can bound the difference betweenthe score functions in terms of the distance between the two data distributions.
Revised: We show through a coupling argument that we can bound the difference between the score functions in terms of the distance between the two data distributions.

Original: This will allow us to“massage” the data distribution in order to optimally bound KL( ψ t q t || p t ) in Section B.
Revised: This will allow us to “massage” the data distribution in order to optimally bound KL( ψ t q t || p t ) in Section C.

Original: C.1 Perturbation under χ 2 error and truncation
Revised: D.1 Perturbation under χ 2 error and truncation

Original: We first give a general lemma on denoising error from a mismatched prior.
Revised: We first give a general lemma on denoising error from a mismatched prior.

Original: Lemma C.1.
Revised: Lemma D.1 (Denoising error from mismatched prior) .

Original: Let φ be a probability density on R d , and P 0 ,x , P 1 ,x be measures on R d .
Revised: Let φ be a probability density on R d , and P 0 ,x , P 1 ,x be measures on R d .

Original: For i = 0 , 1 ,let P i denote the joint distribution of x i ∼ P
Revised: For i = 0 , 1 , let P i denote the joint distribution of x i ∼ P

Original: i,x and y i
Revised: i,x and y i

Original: =
Revised: =

Original: x i + ξ i where ξ i ∼ φ , and let P i,y denotethe marginal distribution of y .
Revised: x i + ξ i where ξ i ∼ φ , and let P i,y denote the marginal distribution of y .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Revised: sup

Revised: [Equation]

Original: Let ε TV =
Revised: Let ε TV =

Original: TV( P 0 ,x , P 1 ,x ) and ε 2 χ = χ 2 ( P 0 ,x || P 1 ,x ) .
Revised: TV( P 0 ,x , P 1 ,x ) and ε 2 χ = χ 2 ( P 0 ,x || P 1 ,x ) .

Original: Then(cid:90)
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: For φ = φ σ 2 , the upper bound is O (cid:16) σ 2 ε χ (cid:16) d + ln (cid:16) 1 ε TV (cid:17)(cid:17)(cid:17) .
Revised: For φ = φ σ 2 , the upper bound is O (cid:16) [Equation]

Original: Note the tricky part of the proof is to deal with P 1 ( dx 1 | y 0 ) , which can be thought of as inferring xassuming the incorrect prior P 1 ,x , rather than the actual prior P 0 ,x .
Revised: Note the tricky part of the proof is to deal with P 1 ( dx 1 | y 0 ) , which can be thought of as inferring x assuming the incorrect prior P 1 ,x , rather than the actual prior P 0 ,x .

Original: Proof.
Revised: Proof.

Original: For notational clarity, we will denote draws from the conditional distribution as (cid:98) x 0 and (cid:98) x 1 ,for example P 0 ( d (cid:98) x 0 | y 0 ) .
Revised: For notational clarity, we will denote draws from the conditional distribution as (cid:98) x 0 and (cid:98) x 1 , for example P 0 ( d (cid:98) x 0 | y 0 ) .

Original: Let r i
Revised: Let r i

Original: ( y ) =
Revised: ( y ) =

Original: (cid:82) R d ( (cid:98) x
Revised: (cid:82) R d ( (cid:98) x

Original: i − y )
Revised: i − y )

Original: P i ( d (cid:98) x i | y ) .
Revised: P i ( d (cid:98) x i | y ) .

Original: Let P 0 , 1 be a coupling of ( x 0 , y 

Revised: Let P 0 , 1 be a coupling of ( x 0 , y

Original: = x 0 + ξ 0 , x 1 , y 

Revised: = x 0 + ξ 0 , x 1 , y

Original: = y 1 + ξ 1 ) such that x 0 = x 1 with probability 1 − ε TV and ξ 0 = ξ 1 with probability1 .
Revised: = y 1 + ξ 1 ) such that x 0 = x 1 with probability 1 − ε TV and ξ 0 = ξ 1 with probability 1 .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: Define a measure Q (not necessarily a probability measure) on R d by
Revised: Define a measure Q (not necessarily a probability measure) on R d by

Original: Q ( A ) :
Revised: [Equation]

Original: = P 0 , 1 ( y 0 ∈ A and [-y 
-]
Revised: and [-y 
-][Equation] [Equation]

Original: = y 1 ) .
Revised: [Equation]

Original: Note that
Revised: Note that

Original: [Equation]
Revised: [Equation]

Original: , so Q is absolutely continuous with respect to P 0 ,y and P 1 ,y , and by assumption on the coupling,
Revised: so Q is absolutely continuous with respect to P 0 ,y and P 1 ,y , and by assumption on the coupling,

Original: Q ( R d ) ≥ 1 − [Equation] TV .
Revised: Q ( R d ) ≥ 1 − ε TV .

Original: Under P 0 , 1 , when y 

Revised: Under P 0 , 1 , when y

Original: = y 1 , we can couple P 

Revised: = y 1 , we can couple P

Original: ( d (cid:98) x 0 | y 0 ) and P 1 ( d (cid:98) x 1 | y 0 )
Revised: ( d (cid:98) x 0 | y 0 ) and P 1 ( d (cid:98) x 1 | y 0 )

Original: so that x 0 = x 1 withprobability min (cid:110) dQdP 0 ,y , dQdP 1 ,y (cid:111) .
Revised: so that x 0 = x 1 with probability min (cid:110) dQdP 0 ,y , dQdP 1 ,y (cid:111) .

Original: Let (cid:98) P ( d (cid:98) x 0 , d (cid:98) x 1 | y 0 )
Revised: Let (cid:98) P ( d (cid:98) x 0 , d (cid:98) x 1 | y 0 )

Original: denote this coupled distribution.
Revised: denote this coupled distribution.

Original: Then as in
Revised: Then as in

Original: Lemma C.5,
Revised: Lemma D.5,

Original: [Equation]
Revised: [Equation]

Original: We bound this by first bounding
Revised: We bound this by first bounding

Original: [Equation]
Revised: [Equation]

Original: which follows from the two inequalities (using (31))
Revised: which follows from the two inequalities (using (31))

Original: [Equation]
Revised: [Equation]

Original: From (32), and the fact that the distribution of ( x i , y i ) is the same as ( (cid:98) x i , y i ) by Nishimori’s identity,we obtain 
Revised: From (32), and the fact that the distribution of ( x i , y i ) is the same as ( (cid:98) x i , y i ) by Nishimori’s identity, we obtain [Equation] I

Original: [Equation]
Revised: [Equation]

Original: Now for the second term (II),
Revised: Now for the second term (II),

Original: [Equation]
Revised: [Equation]

Original: The first term satisfies (cid:82) { y 0 ̸ = y 1 } P 0 , 1 ,y ( dy 0 , dy 1 )
Revised: The first term satisfies (cid:82) { y 0 ̸ = y 1 } P [Equation] 1 ,y ( dy 0 , dy 1 )

Original: ∥
Revised: ∥

Original: r 0 ( y0 )
Revised: r [Equation] ( y

Revised: [Equation]

Original: ∥ 2 ≤ m (2) ( ε TV ) .
Revised: ∥ 2 ≤ m (2) ( ε TV ) .

Original: For the second term, wenote that Cauchy-Schwarz gives for any measures P and Q that
Revised: For the second term, we note that Cauchy-Schwarz gives for any measures P and Q that

Original: [Equation]
Revised: [Equation]

Original: to switch from the measure P 0 ,y to P 1 ,y :
Revised: to switch from the measure P 0 ,y to P 1 ,y :

Original: [Equation]
Revised: [Equation]

Original: (Note that intentionally, the measure is P 1 ,y , though we use y 0 for the variable.)
Revised: (Note that intentionally, the measure is P 1 ,y , though we use y 0 for the variable.)

Original: Hence,
Revised: Hence,

Revised: [Equation]

Revised: so

Original: [Equation]
Revised: [Equation]

Original: where we used the data processing inequality.
Revised: where we used the data processing inequality.

Original: For φ = φ σ 2 , we obtain by Lemma C.6 that the bound is 
Revised: For φ = φ σ 2 , we obtain by Lemma D.6 that the bound is O

Original: [Equation]
Revised: [Equation]

Original: d
Revised: TV

Original: [Equation]
Revised: [Equation]

Original: (cid:18)
Revised: We use this lemma to obtain a bound on the L 2 score error under perturbation of the distribution, by interpreting the score as the solution to a de-noising problem.

Original: [Equation] (cid:19)(cid:19)(cid:19) . Lemma C.2.
Revised: Lemma D.2 ( L 2 score error under perturbation) .

Original: Let (cid:101) P (0) =
Revised: Let (cid:101) P (0) =

Original: (cid:101) P (0) and (cid:101) P (1) = (cid:101) P (1) be two probability distributions on R d such thatχ 2 ( (cid:101) P (1) || (cid:101) P (0) ) ≤ ε 2 χ ≤ 1 .1.
Revised: (cid:101) P (0) and (cid:101) P (1) = (cid:101) P (1) be two probability distributions on R d such that χ 2 ( (cid:101) P (1) || (cid:101) P (0) ) ≤ ε 2 χ ≤ 1 .

Original: For any σ > 0 ,
Revised: For any σ > 0 ,

Original: [Equation]
Revised: [Equation]

Original: Let (cid:101) p ( i ) t be the density resulting from running (1) starting from (cid:101) P ( i ) , and let σ t be as in (3) .
Revised: Let (cid:101) p ( i ) t be the density resulting from running (1) starting from (cid:101) P ( i ) , and let σ t be as in (3) .

Original: Then for any t > 0 ,
Revised: Then for any t > 0 ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: For part 1, note by (27) that
Revised: For part 1, note by (27) that

Original: [Equation]
Revised: [Equation]

Original: where (cid:101) P ( i ) y,σ 2 is the “tilted" probability distribution defined by
Revised: where (cid:101) P ( i ) y,σ 2 is the “tilted" probability distribution defined by

Original: [Equation]
Revised: [Equation]

Original: By Bayes’s rule, this can be viewed as the conditional probability that x 0 = x given x t = y , wherex 0 ∼ (cid:101) P ( i ) and y
Revised: By Bayes’s rule, this can be viewed as the conditional probability that x 0 = x given x t = y , where x 0 ∼ (cid:101) P ( i ) and y

Original: = x 0 + σξ , ξ ∼ N (0 , I d ) .
Revised: = x 0 + σξ , ξ ∼ N (0 , I d ) .

Original: Hence this fits in the framework of Lemma C.1 and
Revised: Hence this fits in the framework of Lemma D.1 and

Original: [Equation]
Revised: [Equation]

Original: , giving the result.
Revised: , giving the result.

Original: For part 2, note that (cid:101) p ( i ) t
Revised: For part 2, note that (cid:101) p ( i ) t

Original: = ( M m t ♯ (cid:101) P ( i ) ) ∗ φ σ 2 t .
Revised: = ( M m t ♯ (cid:101) P ( i ) ) ∗ φ σ 2 t .

Original: Applying part 1 with (cid:101) P ( i ) ← (
Revised: Applying part 1 with (cid:101) P ( i ) ← (

Original: cid:91) M m t
Revised: cid:91) M m t

Original: ♯ (cid:101) P ( i ) (whichpreserves χ 2 -divergence) and σ = σ t gives the result.
Revised: ♯ (cid:101) P ( i ) (which preserves χ 2 -divergence) and σ = σ t gives the result.

Original: Finally, we argue that a score estimate that is accurate with respect to (cid:101) p (1) t will still be accurate withrespect to (cid:101) p (0) t , with high probability.
Revised: Finally, we argue that a score estimate that is accurate with respect to (cid:101) p (1) t will still be accurate with respect to (cid:101) p (0) t , with high probability.

Original: When using this lemma, we will substitute in the bound from
Revised: When using this lemma, we will substitute in the bound from

Original: Lemma C.2.
Revised: Lemma D.2.

Original: Lemma C.3.
Revised: Lemma D.3.

Original: Let (cid:101) P (0) and (cid:101) P (1) be two probability distributions on R d with TV distance ε .
Revised: Let (cid:101) P (0) and (cid:101) P (1) be two probability distributions on R d with TV distance ε .

Original: Supposethe estimated score function s t ( x ) satisfies
Revised: Suppose the estimated score function s t ( x ) satisfies

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]

Original: [Table] [Equation] [Table] for t ∈ (0 , T ] , and ∇ ln (cid:101) p (0) t is L t -Lipschitz.
Revised: for t ∈ (0 , T ] , and ∇ ln (cid:101) p (0) t is L t -Lipschitz.

Original: Then for t ∈ (0 , T ] and any ε ∞ > 0 ,
Revised: Then for t ∈ (0 , T ] and any ε ∞ > 0 ,

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Revised: p

Original: [Equation] [Equation] 
Revised: [Equation] [Equation] [Equation]

Original: Proof.
Revised: Proof.

Original:  have[Table] The first term is bounded by TV( (cid:101) P (0) , (cid:101) P (1) ) ≤ ε .
Revised: We have

Original: For the second term, by Chebyshev’s Inequality,

Original: P

Original: [Table]
Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: The first

Revised: [Table]

Revised: is bounded by

Revised: [Table]

Revised: second term

Revised: [Table]

Revised: by Chebyshev’s

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: For the last term, again by Chebyshev’s Inequality,
Revised: For the last term, again by Chebyshev’s Inequality,

Original: [Equation]
Revised: [Equation]

Original: We conclude the proof by combining the these three inequalities.
Revised: We conclude the proof by combining the these three inequalities.

Original: We will need the following to obtain a TV error bound to (cid:101) p 0 .
Revised: Finally, we will need the following to obtain a TV error bound to (cid:101) p 0 in Theorem 3.3.

Original: Lemma C.4.
Revised: Lemma D.4.

Original: Suppose that (cid:101) p [-
-] ∝ e − V ( x ) is a probability density on R d with bounded first moment482 E (cid:101) p 

Revised: Suppose that (cid:101) p [-
-]0 ∝ e − V ( x ) is a probability density on R d with bounded first moment E (cid:101) p

Original: ∥ X ∥ , and V is L -smooth.
Revised: ∥ X ∥ , and V is L -smooth.

Original: Then for t > 0 such that α t σ t ≤ 12 L , we have
Revised: Then for t > 0 such that α t σ t ≤ 12 L , we have

Original: [Equation]
Revised: [Equation]

Original: Here α t = 1 /m t and σ t are defined in (3) .
Revised: Here α t = 1 /m t and σ t are defined in (3) .

Original: In particular, TV ( (cid:101) pδ ,
Revised: In particular, TV ( (cid:101) p

Original: [Equation]
Revised: [Equation]

Original: p 0 ) ≤ ε TV ifδ =
Revised: if

Original: O ( ε 2 TV R 2 L 2 ) and
Revised: [Equation] and

Original: [-R = max (cid:110) √ d, E (cid:101) p 
 ∥ X ∥ (cid:111) .-]
Revised: [-R = max (cid:110) √ d, E (cid:101) p 
 ∥ X ∥ (cid:111) .-][Equation]

Original: Proof.
Revised: Proof.

Original: Without loss of generality, we assume that (cid:101) p 0 ( x ) =
Revised: Without loss of generality, we assume that (cid:101) p 0 ( x ) =

Original: e − V ( x ) .
Revised: e − V ( x ) .

Original: Note that (cid:101) p t ( x ) =486 (cid:82) α dt (cid:101) p 0 ( α t y )
Revised: Note that (cid:101) p t ( x ) = (cid:82) α dt (cid:101) p [Equation]

Original:  φ σ 2 t ( x − y ) dy .
Revised: α t y ) φ σ 2 t ( x − y ) dy .

Original: Let (cid:101) q t ( x ) :
Revised: Let (cid:101) q t ( x ) :

Original: = α dt (cid:101) p 0 ( α t x ) , which is also a probability density on R d .
Revised: = α dt (cid:101) p 0 ( α t x ) , which is also a probability density on R d .

Original: Then by the triangle inequality,
Revised: Then by the triangle inequality,

Original: [Equation]
Revised: [Equation]

Original: For the second term,
Revised: For the second term,

Original: [Equation]
Revised: [Equation]

Original: [ L ∥ x ∥ ( α t − 1) + d ln α t ] , where in the second inequality, we use the fact that 1 − e x ≤ | x | for all x ≤ 0 .
Revised: , where in the second inequality [Equation] we use the fact that 1 − e x ≤ | x | for all x ≤ 0 .

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: Now for the first term,
Revised: Now for the first term,

Original: [Equation]
Revised: [Equation]

Original: where φ ( y ) is the density of the d -dimensional standard Gaussian distribution.
Revised: where φ ( y ) is the density of the d -dimensional standard Gaussian distribution.

Original: Apply Minkowski’sinequality for integrals:
Revised: Apply Minkowski’s inequality for integrals:

Original: [Equation]
Revised: [Equation]

Original: where in the third inequality, we use the elementary inequality e x ≤ x + e x 2 , which is valid forall x ∈ R , and in the fifth inequality, we use 1 1 − 2 x ≤ e 4 x , which holds for x ∈ [0 , 1 / 3] .
Revised: where in the third inequality, we use the elementary inequality e x ≤ x + e x 2 , which is valid for all x ∈ R , and in the fifth inequality, we use 1 1 − 2 x ≤ e 4 x , which holds for x ∈ [0 , 1 / 3] .

Original: Hence if
Revised: Hence if

Original: Lα t σ t ≤ 1 / 2 , we have
Revised: Lα t σ t ≤ 1 / 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Now we conclude the proof by combining the bounds for TV( (cid:101) p t , [Equation] q t ) and TV( (cid:101) p 0 , (cid:101) q t [Equation]
Revised: Now we conclude the proof by combining the bounds for TV( (cid:101) p t , (cid:101) q t ) and TV( (cid:101) p 0 , (cid:101) q t ) :

Original: [Equation]
Revised: [Equation]

Original: where we use the fact that ln x ≤ x − 1 for all x ≥ 1 .
Revised: where we use the fact that ln x ≤ x − 1 for all x ≥ 1 .

Original: Recall that α t = 1 /m t
Revised: Recall that α t = 1 /m t

Original: =
Revised: =

Original: e t/ 2 and σ 2 t = 1 − e − twhen g ≡ 1 .
Revised: e t/ 2 and σ 2 t = 1 − e − t when g ≡ 1 .

Original: It suffices for
Revised: It suffices for

Original: [Equation]
Revised: [Equation]

Original: which is implied by
Revised: which is implied by

Original: [Equation]
Revised: [Equation]

Original: for appropriate constants, as R ≥ max (cid:110) √ d, E [Equation] p 

Revised: for appropriate constants, as R ≥ max (cid:110) √ d, E (cid:101) p

Original: ∥ X∥
Revised: ∥ X

Original: [Equation]
Revised: [Equation]

Original: C.2 Perturbation under TV error
Revised: D.2 Perturbation under TV error

Original: Although we will not need it in our proof, we note that we can derive a similar perturbation resultunder TV error, which might be of independent interest.
Revised: Although we will not need it in our proof, we note that we can derive a similar perturbation result under TV error, which might be of independent interest.

Original: Lemma C.5.
Revised: Lemma D.5.

Original: Let K ( x, dy ) be a probability kernel on R d , let P 0 ,x , P 1 ,x be measures on R d .
Revised: Let K ( x, dy ) be a probability kernel on R d , let P 0 ,x , P 1 ,x be measures on R d .

Original: Let
Revised: Let

Original: P i denote the joint distribution of x i ∼ P i,x and y i ∼ K
Revised: P i denote the joint distribution of x i ∼ P i,x and y i ∼ K

Original: ( x i , · ) , and let P i
Revised: ( x i , · ) , and let P i

Original: ,y denote the marginaldistribution of y .
Revised: ,y denote the marginal distribution of y .

Original: Suppose there is a coupling P 0 , 1 of ( x 0 , y 0 ) ∼ P 0 and ( x 1 , y 1 ) ∼ P 1 such that
Revised: Suppose there is a coupling P 0 , 1 of ( x 0 , y 0 ) ∼ P 0 and ( x 1 , y 1 ) ∼ P 1 such that

Original: • x 0 = x 1 with probability 1 − ε , • x 0 = x 1 implies y 

Revised: • x 0 = x 1 with probability 1 − ε , • x 0 = x 1 implies y

Original: = y 1 , and• E [ ∥ y [-
-] − y 1 ∥ 2 ] ≤ ε 2 W .
Revised: = y 1 , and • E [ ∥ y [-
-]0 − y 1 ∥ 2 ] ≤ ε 2 W .

Original: Define the tail error by
Revised: Define the tail error by

Original: [Equation]
Revised: [Equation]

Original: Let r i
Revised: Let r i

Original: ( y ) =
Revised: ( y ) =

Original: (cid:82) R d x i P i ( dx i | y ) , and suppose that r 1 ( y ) =
Revised: (cid:82) R d x i P i ( dx i | y ) , and suppose that r 1 ( y ) =

Original: (cid:82) R d x 1 P 1 ( dx 1 | y ) is L 1 -Lipschitz.
Revised: (cid:82) R d x 1 P 1 ( dx 1 | y ) is L 1 -Lipschitz.

Original: Then(cid:90) [Equation] ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:90) [Equation]
Revised: Then

Original: ) −

Original: [Equation]

Original: R d

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: For notational clarity [Equation] we will denote draws from the conditional distribution as (cid:98) x 0 and (cid:98) x 1 ,for example P 0 ( d (cid:98) x 0 | y 0 ) .
Revised: For notational clarity, we will denote draws from the conditional distribution as (cid:98) x 0 and (cid:98) x 1 , for example P 0 ( d (cid:98) x 0 | y 0 ) .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: For the first term (I) [Equation] we split it as
Revised: For the first term (I), we split it as

Original: [Equation]
Revised: [Equation]

Original: ii ) .

Original: Define the measure Q on R d by
Revised: Define the measure Q on R d by

Original: [-Q ( A ) : = P 0 , 1 ( y 0 ∈ A and y 
 = y 1 ) .-]
Revised: [-Q ( A ) : = P 0 , 1 ( y 0 ∈ A and y 
 = y 1 ) .-][Equation]

Original: As in Lemma C.2, under P 0 , 1 , when y 

Revised: As in Lemma D.2, under P 0 , 1 , when y

Original: = y 1 , we can couple P 

Revised: = y 1 , we can couple P

Original: ( d (cid:98) x 0 | y 0 ) and P 1 ( d (cid:98) x 1 | y 0 )
Revised: ( d (cid:98) x 0 | y 0 ) and P 1 ( d (cid:98) x 1 | y 0 )

Original: so [-thatx 
-] = x 1 with probability min (cid:110) dQdP 0 ,y , dQdP 1 ,y (cid:111) .
Revised: so [-thatx 
-]that x 0 = x 1 with probability min (cid:110) dQdP 0 ,y , dQdP 1 ,y (cid:111) .

Original: Let (cid:98) P ( d (cid:98) x 0 , d (cid:98) x 1 | y 0 )
Revised: Let (cid:98) P ( d (cid:98) x 0 , d (cid:98) x 1 | y 0 )

Original: denote this coupled distribution.
Revised: denote this coupled distribution.

Original: Then
Revised: Then

Revised: [Equation]

Revised: ≤

Revised: [Equation]

Revised: m

Revised: [Equation]

Revised: + m

Original: [Equation]
Revised: [Equation]

Original: as in Lemma C.2.
Revised: as in Lemma D.2.

Original: Now
Revised: Now

Original: [Equation]
Revised: [Equation]

Original: Finally, for the second term (II), we use the fact that r 1 is L 1 Lipschitz and the coupling to conclude
Revised: Finally, for the second term (II), we use the fact that r 1 is L 1 Lipschitz and the coupling to conclude

Original: [Equation]
Revised: [Equation]

Original: We conclude the proof by combining the inequalities for (i), (ii), and (II).
Revised: We conclude the proof by combining the inequalities for (i), (ii), and (II).

Original: For the second upper bound, we note that
Revised: For the second upper bound, we note that

Original: [Equation]
Revised: [Equation]

Original: C.3 Gaussian tail calculation
Revised: D.3 Gaussian tail calculation

Revised: We use the following Gaussian tail calculation in the proof of Lemma D.2.

Original: Lemma C.6.
Revised: Lemma D.6.

Original: Let µ be the standard Gaussian measure on N (0 , I d ) .
Revised: Let µ be the standard Gaussian measure on N (0 , I d ) .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By the χ 2 tail bound in [LM00], for t ≥ 0 ,
Revised: By the χ 2 tail bound in [LM00], for t ≥ 0 ,

Original: [Equation]
Revised: [Equation]

Original: so ∥ X ∥ 2 is stochastically dominated by a random variable with cdf F ( y )
Revised: so ∥ X ∥ 2 is stochastically dominated by a random variable with cdf F ( y )

Original: = 1 − e − y − 2 d 3 .
Revised: = 1 − e − y − 2 d 3 .

Original: Then letting
Revised: Then letting

Original: P Y be the measure corresponding to F ,
Revised: P Y be the measure corresponding to F ,

Original: [Equation]
Revised: [Equation]

Original: A ∥ x ∥ 2 µ ( dx ) ≤ sup P Y ( A ) ≤ ε (cid:90)
Revised: ln (cid:18)

Original: A yP Y ( dy ) =
Revised: [Equation]

Original:  (cid:90) ∞
Revised: (cid:19)(cid:19) + (cid:90) ∞

Original: 2 d +3ln ( 1 ε ) ydF ( y ) =
Revised: [Equation] +3ln ( 1 ε ) e

Original: ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19) + (cid:90) ∞
Revised: [Equation] (cid:18) [Equation]

Original: 2 d +3ln ( 1 ε ) e − y − 2 d 3 dy = ε (cid:18) 2 d +3 ln (cid:18) 1 ε (cid:19)(cid:19) + 3 ε andsup µ ( A ) ≤ ε (cid:90) A ∥ x ∥ 4 µ ( dx ) ≤ sup P Y ( A ) ≤ ε (cid:90) A y 2 P Y ( dy ) = (cid:90) ∞
Revised: +

Original: 2 d +3ln ( 1 ε ) y 2 dF ( y ) =
Revised: [Equation]

Original: ε (cid:18) 2 d + 3ln (cid:18)ε (cid:19)(cid:19)+ (cid:90) ∞ 2 d +3ln ( 1 ε  ) 2 ye − y − 2 d 3 dy
Revised: ln (cid:18) 1 ε (cid:19)(cid:19) + 3 ε and sup µ ( A ) ≤ ε (cid:90)

Original: [-= ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19)− (cid:104) 3 ye − y − 2 d 3 (cid:105) (cid:12)(cid:12)(cid:12) ∞ 
 d +3ln ( 1 ε ) + (cid:90) ∞ 2 d +3ln ( 1 ε ) 3 e − y − 2 d 3 dy = ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19)+ 3 ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19) + 9 ε.-]
Revised: [-= ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19)− (cid:104) 3 ye − y − 2 d 3 (cid:105) (cid:12)(cid:12)(cid:12) ∞ 
 d +3ln ( 1 ε ) + (cid:90) ∞ 2 d +3ln ( 1 ε ) 3 e − y − 2 d 3 dy = ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19)+ 3 ε (cid:18) 2 d + 3 ln (cid:18)ε (cid:19)(cid:19) + 9 ε.-][Equation]

Original: D Guarantees under L 2 -accurate score estimate
Revised: E Guarantees under L 2 -accurate score estimate

Original: We will state our results under a more general tail bound assumption.
Revised: We will state our results under a more general tail bound assumption.

Original: Assumption 6 (Tail bound) .
Revised: Assumption 6 (Tail bound) .

Original: R : [0 , 1] → [0 , ∞ ) is a function such that P data ( B R ( ε ) (0)) ≥ 1 − ε .
Revised: R : [0 , 1] → [0 , ∞ ) is a function such that P data ( B R ( ε ) (0)) ≥ 1 − ε .

Original: Our result will require R ( ε ) to grow at most as a sufficiently small power of ε − 1 as ε → 0 ; inparticular, this holds for subexponential distributions.
Revised: Our result will require R ( ε ) to grow at most as a sufficiently small power of ε − 1 as ε → 0 ; in particular, this holds for subexponential distributions.

Original: By taking R to be a constant function, thiscontains the assumption of bounded support (Assumption 2) as a special case.
Revised: By taking R to be a constant function, this contains the assumption of bounded support (Assumption 2) as a special case.

Original: D.1 TV error guarantees
Revised: E.1 TV error guarantees

Original: We follow the framework of [LLT22] to convert guarantees under L ∞ -accurate score estimate, toguarantees under L 2 -accurate score estimate.
Revised: We follow the framework of [LLT22] to convert guarantees under L ∞ -accurate score estimate, to guarantees under L 2 -accurate score estimate.

Original: Theorem D.1 ([LLT22, Theorem 4.1]) .
Revised: Theorem E.1 ([LLT22, Theorem 4.1]) .

Original: Let (Ω , F , P ) be a probability space and {F n } be a filtrationof the sigma field F .
Revised: Let (Ω , F , P ) be a probability space and {F n } be a filtration of the sigma field F .

Original: Suppose X n ∼ p n , Z n ∼ q n , and Z n ∼ q n are F n -adapted random processestaking values in Ω , and B n ⊆
Revised: Suppose X n ∼ p n , Z n ∼ q n , and Z n ∼ q n are F n -adapted random processes taking values in Ω , and B n ⊆

Original: Ω are sets such that the following hold for every n ∈ N 0 .
Revised: Ω are sets such that the following hold for every n ∈ N 0 .

Original: If Z k ∈ B ck for all 0 ≤ k ≤ n − 1 , then Z n = Z n .2.
Revised: If Z k ∈ B ck for all 0 ≤ k ≤ n − 1 , then Z n = Z n .

Original: χ 2 ( q n || p n ) ≤ D 2 n .3.
Revised: χ 2 ( q n || p n ) ≤ D 2 n .

Original: P ( X n ∈ B n ) ≤ δ n .
Revised: P ( X n ∈ B n ) ≤ δ n .

Original: Then the following hold.
Revised: Then the following hold.

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Theorem D.2 (DDPM with L 2 -accurate score estimate) .
Revised: Theorem E.2 (DDPM with L 2 -accurate score estimate) .

Original: Let 0 < ε χ , ε TV , δ < 12 .
Revised: Let 0 < ε χ , ε TV , δ < 12 .

Original: Suppose that
Revised: Suppose that

Original: Assumption 6 for a sufficiently small value of c that R 0 is such that R (cid:16) cε 3TV δ 6 ε 12 χ R 190 d 5 (cid:17) ≤ R 0 , and
Revised: Assumption 6 for a sufficiently small value of c that R 0 is such that R (cid:16) cε 3TV δ 6 ε 12 χ R 190 d 5 (cid:17) ≤ R 0 , and

Original: R 20 ≥ d .
Revised: R 20 ≥ d .

Original: Suppose one of the following cases holds.1.
Revised: Suppose one of the following cases holds.

Original: Let P data , s ( · , t ) be such that Assumption 1 holds, with R 20 ≥ d .
Revised: Let P data , s ( · , t ) be such that Assumption 1 holds, with R 20 ≥ d .

Original: Suppose that
Revised: Suppose that

Original: [Equation]
Revised: [Equation]

Original: where B = R 40 d ln (cid:0) Tδ (cid:1) ln (cid:16) R 0 d δε TV ε χ (cid:17) , and we run (5) starting from p prior for time
Revised: where B = R 40 d ln (cid:0) Tδ (cid:1) ln (cid:16) R 0 d δε TV ε χ (cid:17) , and we run (5) starting from p prior for time

Original: [Equation]
Revised: [Equation]

Original:  step 
Revised: steps with step sizes satisfying h k

Original: [Equation]
Revised: [Equation]

Original: Let P data , s ( · , t ) be such that Assumptions 1 and 3 hold, with C ≥ R 20 .
Revised: Let P data , s ( · , t ) be such that Assumptions 1 and 3 hold, with C ≥ R 20 .

Original: Suppose
Revised: Suppose

Original: [Equation]
Revised: [Equation]

Original: where B = C 2 d ln (cid:0) Tδ (cid:1) ln
Revised: where B = C 2 d ln (cid:0) Tδ (cid:1) ln

Original: (cid:16) R[-
-]
Revised: (cid:16) R[-
-]

Revised: [Equation]

Original: d δε TV ε χ (cid:17) , and we run (5) starting from p prior for time
Revised: d δε TV ε χ (cid:17) , and we run (5) starting from p prior for time

Original: [Equation]
Revised: [Equation]

Original: steps with step sizes satisfying h 
Revised: steps with step sizes satisfying h k =

Original: [Equation]
Revised: [Equation]

Original: [Equation] Then the resulting distribution q t N is such that q t N is ε TV -far in TV distance from a distribution q t N ,where q t N satisfies χ 
Revised: Then the resulting distribution q t N is such that q t N is ε TV -far in TV distance from a distribution q t N , where q t N satisfies χ 2 ( q t N ||

Original: [Equation]

Original: q t N || p t N ) ≤ ε 2 χ .
Revised: p t N ) ≤ ε 2 χ .

Original: In particular, taking ε χ = ε TV , we have TV( q T , P data ) ≤ 
Revised: In particular, taking ε χ = ε TV , we have TV( q T , P data ) ≤ 2 ε TV .

Original: Note that the condition on R can be satisfied if R ( ε ) =
Revised: Note that the condition on R can be satisfied if R ( ε ) =

Original: o ( R − 1 / 19 ) (no effort has been made tooptimize the exponent).
Revised: o ( R − 1 / 19 ) (no effort has been made to optimize the exponent).

Original: Proof.
Revised: Proof.

Original: We invoke Lemma B.2 for a ε K to be chosen, to obtain a distribution (cid:101) P 0 on B R 0 (0) , where
Revised: We invoke Lemma C.2 for a ε K to be chosen, to obtain a distribution (cid:101) P 0 on B R 0 (0) , where

Original: R 0 ≥ R ( ε K / 8) .
Revised: R 0 ≥ R ( ε K / 8) .

Original: Let B =
Revised: Let B =

Original: R 

Revised: R

Original: d ln (cid:0) Tδ (cid:1) ln
Revised: d ln (cid:0) Tδ (cid:1) ln

Original: (cid:16) R 0 δε K (cid:17) and B = C 

Revised: (cid:16) R 0 δε K (cid:17) and B = C

Original: d ln (cid:0) Tδ (cid:1) ln
Revised: d ln (cid:0) Tδ (cid:1) ln

Original: (cid:16) R 0 δε K (cid:17) in case 1 and case2, respectively; our choice of ε K
Revised: (cid:16) R 0 δε K (cid:17) in case 1 and case 2, respectively; our choice of ε K

Original: = O (cid:16)
Revised: = O (cid:16)

Original: ε 2TV
Revised: ε 2TV

Original: δ 6 n 2 R 60 (cid:17) will give the definition of B in the theorem statement.
Revised: δ 6 n 2 R 60 (cid:17) will give the definition of B in the theorem statement.

Original: In the following, we define (cid:101) p t with (cid:101) P 0 , rather than P data , as the initial distribution.
Revised: In the following, we define (cid:101) p t with (cid:101) P 0 , rather than P data , as the initial distribution.

Original: Note that since
Revised: Note that since

Original: TV( P data , (cid:101) P 0 ) ≤ √ ε K
Revised: TV( P data , (cid:101) P 0 ) ≤ √ ε K

Original: = o ( ε TV ) (and the same holds for their evolutions under (1)), it suffices toconsider convergence to (cid:101) p δ .
Revised: = o ( ε TV ) (and the same holds for their evolutions under (1)), it suffices to consider convergence to (cid:101) p δ .

Original: We first define the bad sets where the error in the score estimate is large,
Revised: We first define the bad sets where the error in the score estimate is large,

Original: [Equation]
Revised: [Equation]

Original: (35) for some ε ∞ ,t to be chosen.
Revised: for some ε ∞ ,t to be chosen.

Original: Given t ≥ 0 , let t − =
Revised: Given t ≥ 0 , let t − =

Original: t k where k is such that t ∈ [ t k , t k +1 ) .
Revised: t k where k is such that t ∈ [ t k , t k +1 ) .

Original: Given bad sets B t , define the interpolatedprocess on [ t k , t k +1 ) by
Revised: Given bad sets B t , define the interpolated process on [ t k , t k +1 ) by

Original: [Equation]
Revised: [Equation]

Original: (36) where b ( z, t )
Revised: where b ( z, t )

Original: =(cid:26)
Revised: =

Original: [Equation]
Revised: [Equation]

Original: In other words, simulate the reverse SDE using the score estimate as long as the point is in the goodset at the previous discretization timepoint t k , and otherwise use the actual gradient ∇ ln p t .
Revised: In other words, simulate the reverse SDE using the score estimate as long as the point is in the good set at the previous discretization timepoint t k , and otherwise use the actual gradient ∇ ln p t .

Original: Letq t denote the distribution of z t when z 0 ∼ q 0 .
Revised: Let q t denote the distribution of z t when z 0 ∼ q 0 .

Original: Note that this process is defined only for purposesof analysis, as we do not have access to ∇ ln p t .
Revised: Note that this process is defined only for purposes of analysis, as we do not have access to ∇ ln p t .

Original: As before, we let denote q t the distribution of z tdefined by (12).
Revised: As before, we let denote q t the distribution of z t defined by (12).

Original: We can couple this process with the exponential integrator (5)using s so that as long as x t m ̸∈ B T − t m ,the processes agree, thus satisfying condition 1 of Theorem D.1.
Revised: We can couple this process with the exponential integrator (5) using s so that as long as x t m ̸∈ B T − t m , the processes agree, thus satisfying condition 1 of Theorem E.1.

Original: Then by Lemma C.3,
Revised: Then by Lemma D.3, substituting the bound of Lemma D.2 for the L 2 score error,

Original: [Equation]
Revised: [Equation]

Original: Then by choice of h k and either Corollary A.11 or A.12, when (cid:82) t n 0 ε 2 t dt = O (1) ,
Revised: Then by choice of h k and either Corollary B.11 or B.12, when (cid:82) t n 0 ε [Equation] dt = O (1) ,

Original: [Equation]
Revised: [Equation]

Original: where ε = ε 2 χ 4 .
Revised: where ε = ε 2 χ 4 .

Original: For χ 2 ( q t k || p t k ) to be bounded by ε 2 χ , it suffices for the terms in (37) to be boundedby ε 2 χ 2 , ε 2 χ 4 , ε 2 χ 4 ; this is implied by
Revised: For χ 2 ( q t k || p t k ) to be bounded by ε 2 χ , it suffices for the terms in (37) to be bounded by ε 2 χ 2 , ε 2 χ 4 , ε 2 χ 4 ; this is implied by

Original: [Equation]
Revised: [Equation]

Original: by Lemma A.
Revised: by Lemma B.

Original: (cid:90) t n ε 2 ∞ ,T − t dt = O ( ε 2 χ ) .
Revised: [Equation]

Original: By Theorem D.1,
Revised: By Theorem E.1,

Original: [Equation]
Revised: [Equation]

Original: For this to be bounded by ε TV , it suffices for
Revised: For this to be bounded by ε TV , it suffices for

Original: [Equation]
Revised: [Equation]

Original: We bound (42) crudely, as the dependence on ε K will be logarithmic.
Revised: We bound (42) crudely, as the dependence on ε K will be logarithmic.

Original: Using ε 2 t k = ε 2 σ /σ 4 t k , it sufficesthat
Revised: Using ε 2 t k = ε 2 σ /σ 4 t k , it suffices that

Original: [Equation]
Revised: [Equation]

Original: We will return to this after deriving a condition on ε σ .
Revised: We will return to this after deriving a condition on ε σ .

Original: It remains to bound (38) and (41).
Revised: It remains to bound (38) and (41).

Original: We breakup the timepoints depending on whether T − t > 1 .
Revised: We break up the timepoints depending on whether T − t > 1 .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: and u k = T − t ′ k , where t n coarse − 1 ≤ T − 1 ≤ t ′ 1 .
Revised: and u k = T − t ′ k , where t n coarse − 1 ≤ T − 1 ≤ t ′ 1 .

Original: Let h ′
Revised: Let h ′

Original: k
Revised: k

Original: = t ′ k +1 − t ′ k .
Revised: = t ′ k +1 − t ′ k .

Original: Note the “fine” timepointswill be closer together than the “coarse” timepoints.
Revised: Note the “fine” timepoints will be closer together than the “coarse” timepoints.

Original: We break up the integral (38) and the sum (41)into the parts involving the coarse and fine timepoints.
Revised: We break up the integral (38) and the sum (41) into the parts involving the coarse and fine timepoints.

Original: For (38), it suffices to have(38) , coarse:
Revised: For (38), it suffices to have (38) , coarse:

Original: [Equation]
Revised: [Equation]

Original: so it suffices to take ε 2 ∞ ,T − t k ≍ ε 2 χ T .
Revised: so it suffices to take ε 2 ∞ ,T − t k ≍ [Equation] 2 χ T .

Original: Let α = 3 in case 1 and α = 1 in case 2.
Revised: Let α = 3 in case 1 and α = 1 in case 2.

Original: For the fine part,recalling our choice of h ′ k , it suffices to have (note we can redefine ε t = ε t k when t ∈
Revised: For the fine part, recalling our choice of h ′ k , it suffices to have (note we can redefine ε t = ε t k when t ∈

Original: [ t k , t k +1 )without any harm)(38)
Revised: [ t k , t k +1 ) without any harm)

Original: [Equation]
Revised: [Equation]

Original: fine: [Equation] [Equation] For (41), it suffices to have(41) , coarse:
Revised: For (41), it suffices to have

Original: [Equation]
Revised: [Equation]

Original: (cid:19) (45) and(41) , fine:
Revised: 41) , coarse:

Original: [Equation]
Revised: [Equation]

Original: k
Revised: 45) and (41) , fine:

Original: [Equation]
Revised: [Equation]

Original: Note that in light of the required step sizes, we can take n coarse ≍ T 2 B ε 2 χ .
Revised: Note that in light of the required step sizes, we can take n coarse ≍ T 2 B ε 2 χ .

Original: Considering the equality caseof Hölder’s inequality on (44) 1 / 3 (46) 2 / 3 suggests that we take
Revised: Considering the equality case of Hölder’s inequality on (44) 1 / 3 (46) 2 / 3 suggests that we take

Original: [Equation]
Revised: [Equation]

Original: Note that the number of steps needed in the fine part is O (cid:16) Bε 
Revised: Note that the number of steps needed in the fine part is O (cid:16) Bε 2 χ

Original: [Equation]

Original: δ 2 (cid:17) in the first case and O (cid:16) Bε 2 χ (cid:17) ln (cid:0) 1 δ (cid:1)in the second case.
Revised: δ 2 (cid:17) in the first case and O (cid:16) Bε 2 χ (cid:17) ln (cid:0) 1 δ (cid:1) in the second case.

Original: We can check that (47) and (48) make (44) and (46) satisfied.
Revised: We can check that (47) and (48) make (44) and (46) satisfied.

Original: Finally, we calculate the denominator for ε σ .
Revised: Finally, we calculate the denominator for ε σ .

Original: In case 1, note that starting from T − t ′ 

Revised: In case 1, note that starting from T − t ′

Original: = O (1) andtaking steps of size h ′ k
Revised: = O (1) and taking steps of size h ′ k

Original: ≍

Original: [Equation] 3 , it takes n fine =Θ (cid:16) Bε [Equation]
Revised: ≍ ε 2 χ B ( T − t ′ k ) 3 , it takes n fine =

Original:  χ [Equation] 2 (cid:17) steps to reach T − t = δ .
Revised: Θ (cid:16) Bε 2 χ δ 2 (cid:17) steps to reach T − t = δ .

Original: [Equation]
Revised: [Equation]

Revised: 11 / 6δ 5 / 3 .

Original: Then we obtain
Revised: Then we obtain

Original: [Equation]
Revised: [Equation]

Original: In case 1, our requirement is
Revised: In case 1, our requirement is

Original: [Equation]
Revised: [Equation]

Original: but note that the first bound is more stringent.
Revised: but note that the first bound is more stringent.

Original: Now, returning to (43), we see that it suffices to takeε K
Revised: Now, returning to (43), we see that it suffices to take ε K

Original: = O
Revised: = O

Original: [Equation]
Revised: [Equation]

Original: 2+ β (cid:33) for any β > 0 (this will “solve” the log(1 /ε K ) appearing in B .)
Revised: for any β > 0 (this will “solve” the log(1 /ε K ) appearing in B .)

Original: In case 2, we have instead u k =exp (cid:16) − Θ (cid:16) ε
Revised: In case 2, we have instead u k =

Original: [Equation]
Revised: [Equation]

Original: so
Revised: B

Original: [Equation]
Revised: [Equation]

Original: B . Theorem D.3 (TV error for DDPM with L 2 -accurate score estimate and smoothness) .
Revised: Theorem E.3 (TV error for DDPM with L 2 -accurate score estimate and smoothness) .

Original: Let 0 <ε
Revised: Let 0 < ε

Original: TV < 12 .
Revised: TV < 12 .

Original: Suppose that Assumption 4 and 6 for a sufficiently small value of c that R 0 is such that
Revised: Suppose that Assumption 4 and 6 for a sufficiently small value of c that R 0 is such that

Original: R (cid:16) cε 15TV R
Revised: [Equation] L

Original: [Equation]
Revised: [Equation]

Original: d 5 L 12 (cid:17) ≤ R 0 , and R 20 ≥ max (cid:110) d, E P data (cid:104)
Revised: ≤ R 0 , and R 20 ≥ max (cid:110) d, E P data (cid:104)

Original: ∥
Revised: ∥

Original: X ∥ 2 (cid:105)(cid:111) , and one of the following cases holds.1.
Revised: X ∥ 2 (cid:105)(cid:111) , and one of the following cases holds.

Original: Let P data , s ( · , t ) be such that Assumption 1 holds.
Revised: Let P data , s ( · , t ) be such that Assumption 1 holds.

Original: Suppose that
Revised: Suppose that

Original: [Equation]
Revised: [Equation]

Original: where B = R[-4 
-]
Revised: where B = R[-4 
-]

Original: d ln (cid:16) TR

Original: [Equation]
Revised: [Equation]

Original: (cid:17) ln (cid:16) R

Original: [Equation] TV ε χ (cid:17) , and we run (5) starting from p prior for time[Equation] ε
Revised: , and we run (5) starting from p prior for time

Original: [Equation]
Revised: [Equation]

Original:   steps with step sizes satisfying h k =
Revised:  steps with step sizes satisfying h k =

Original: O (cid:16) ε 2 χ

Original: [Equation]
Revised: [Equation]

Original: max { T − t k , ( T − t k ) − 3 } (cid:17) .

Original: Let P data , s ( · , t ) be such that Assumptions 1 and 3 hold, with C ≥ R 20 .
Revised: Let P data , s ( · , t ) be such that Assumptions 1 and 3 hold, with C ≥ R 20 .

Original: Suppose
Revised: Suppose

Original: [Equation]
Revised: [Equation]

Original: where B = C 2 d ln (cid:16) TR 20 L 2 ε 2TV (cid:17) ln (cid:16) R 30 dL 2 ε 4TV (cid:17) , and we run (5) starting from p prior for time
Revised: where B = C 2 d ln (cid:16) TR 20 L 2 ε 2TV (cid:17) ln (cid:16) R 30 dL 2 ε 4TV (cid:17) , and we run (5) starting from p prior for time

Original: [Equation]
Revised: [Equation]

Original: steps with stepsizes satisfying h k =
Revised: step

Original: [Equation]
Revised: [Equation]

Original: Then the resulting distribution q t N is such that q t N is ε TV -far in TV distance from the data distribution
Revised: Then the resulting distribution q t N is such that q t N is ε TV -far in TV distance from the data distribution

Original: P data .
Revised: P data .

Original: Proof.
Revised: Proof.

Original: With the result of Theorem D.2, we see that TV( q t N , p t N ) ≤ 2 ε TV .
Revised: With the result of Theorem E.2, we see that TV( q t N , p t N ) ≤ 2 ε TV .

Original: Now by Lemma C.4, ifwe further assume
Revised: Now by Lemma D.4, if we further assume

Original: [Equation]
Revised: [Equation]

Original: then TV( p t N , P data ) ≤ ε TV .
Revised: then TV( p t N , P data ) ≤ ε TV .

Original: We conclude the proof by triangle inequality and replacing the δdependence with O ( ε 2TV R 20 L 2 ) in the previous theorem.
Revised: We conclude the proof by triangle inequality and replacing the δdependence with O ( ε 2TV R 20 L 2 ) in the previous theorem.

Original: Proof of Theorem 2.3.
Revised: Proof of Theorem 3.3.

Original: If P data is subexponential with a fixed constant, note that Assumption 6 holdswith R ( ε ) =
Revised: If P data is subexponential with a fixed constant, note that Assumption 6 holds with R ( ε ) =

Original: O (cid:0) ln (cid:0) 1 ε (cid:1)(cid:1) and hence R 0 is logarithmic in all parameters.
Revised: O (cid:0) ln (cid:0) 1 ε (cid:1)(cid:1) and hence R 0 is logarithmic in all parameters.

Original: D.2 Wasserstein error guarantees
Revised: E.2 Wasserstein error guarantees

Original: Proof of Theorem 2.1.
Revised: Proof of Theorem 3.1.

Original: If T − t N = δ  , then W 2 ( (cid:101) p 0 ,  (cid:101) p δ ) ≤  σ δ  ≤ √  δ .
Revised: Suppose T − t N = δ . Note by (3) that M m − 1 δ # (cid:101) p δ has the same distribution as (cid:101) x 0 + m − 1 δ σ δ z , where (cid:101) x 0 ∼ (cid:101) P 0 and z ∼ N (0 , I d ) . Then W 2 ( (cid:101) p 0 , M m − 1 δ ♯ (cid:101) p δ ) ≤ m − 1 δ σ δ √ d ≤ √ e δ − 1 ≤ √ 2 δd (for δ ≤ 1 ).

Original: Choosing δ = ε 2 W  , we see by Theorem D.2 it suffices to take
Revised: Choosing δ = ε 2 W 2 d , we see by Theorem E.2 it suffices to take

Original: [Equation]
Revised: [Equation]

Original: Simplifying gives ε σ = (cid:101) o [Equation] ε 6 .
Revised: Simplifying gives ε σ = (cid:101) o (cid:16) ε 6 .

Original: 5TV ε 5 W R 9 d 2 .
Revised: 5TV ε 5 W R 9 d 4 .

Original: 25 (cid:17) .
Revised: 75 (cid:17) .

Original: If Assumption 3 also holds, then it suffices to take
Revised: If Assumption 3 also holds, then it suffices to take

Original: [Equation]
Revised: [Equation]

Original: Simplifying gives ε σ [Equation]
Revised: Simplifying gives ε σ = (cid:101) o (cid:16) ε 4TV C 2 d (cid:17) .

Original: Proof of Theorem 2.2.
Revised: Proof of Theorem 3.2.

Original: To obtain purely Wasserstein error guarantees, we include an extra step ofreplacing any sample z t N ∼ q t N falling outside B R (0) by 0 .
Revised: To obtain purely Wasserstein error guarantees, we include an extra step of replacing any sample z t N ∼ q t N falling outside B R (0) by 0 .

Original: Suppose T − t N = δ .
Revised: Suppose T − t N = δ .

Original: Let (cid:98) q t
Revised: Let (cid:98) q t

Original: N be theresulting distribution.
Revised: N be the resulting distribution.

Original: Then 
Revised: Then as above,

Original: [Equation]
Revised: [Equation]

Original: We choose δ = ε 2 W 4 so the first term is ≤ ε W 2 .
Revised: We choose δ = ε 2 W 8 d so the first term is ≤ ε W 2 .

Original: It suffices to bound the second term W 2 ( (cid:101) p δ , (cid:98) q t N ) alsoby ε W 2 .
Revised: It suffices to bound the second term W 2 ( (cid:101) p δ , (cid:98) q t N ) also by ε W 2 .

Original: We bound it in terms of TV( (cid:101) p
Revised: We bound it in terms of TV( (cid:101) p

Original: δ , (cid:98) q t N ) using the fact that (cid:98) q t N is supported on B R (0) and usinga Gaussian tail calculation for (cid:101) p δ .
Revised: δ , (cid:98) q t N ) using the fact that (cid:98) q t N is supported on B R (0) and using a Gaussian tail calculation for (cid:101) p δ .

Original: Consider a coupling of x t N = (cid:101) x δ ∼ (cid:101) p δ and (cid:98)
Revised: Consider a coupling of x t N = (cid:101) x δ ∼ (cid:101) p δ and (cid:98)

Original: z t N ∼ (cid:98) q t N such that637 x δ ̸ = (cid:98) z t N with probability ε TV .
Revised: z t N ∼ (cid:98) q t N such that x δ ̸ = (cid:98) z t N with probability ε TV .

Original: Express (cid:101) x
Revised: Express (cid:101) x

Original: δ = m δ (cid:101) x 0 + σ δ ξ where (cid:101) x 0 ∼ (cid:101) p 0 .
Revised: δ = m δ (cid:101) x 0 + σ δ ξ where (cid:101) x 0 ∼ (cid:101) p 0 .

Original: Now
Revised: Now

Revised: E

Original: [Equation]
Revised: [Equation]

Original: where the bound on the second term uses Lemma C.6.
Revised: where the bound on the second term uses Lemma D.6.

Original: Using R 2 ≥ d , we see that it suffices to chooseε TV =
Revised: Using R 2 ≥ d , we see that it suffices to choose ε TV =

Original: O (cid:16) ε 2 W R 2 (cid:17) for appropriate choice of constants.
Revised: O (cid:16) ε 2 W R 2 (cid:17) for appropriate choice of constants.

Original: By Theorem D.2, it suffices to take
Revised: By Theorem E.2, it suffices to take

Original: [Equation]
Revised: [Equation]

Original: Simplifying gives (cid:101) o(cid:16)
Revised: Simplifying gives (cid:101) o

Original: [Equation]
Revised: [Equation]

Original: In case 2, it suffices to take
Revised: In case 2, it suffices to take

Original: [Equation]
Revised: [Equation]

Original: Simplifying gives ε σ = (cid:101) o (cid:16) ε 8 W C 2 R 8 d (cid:17) .
Revised: Simplifying gives ε [Equation]

Original: E High-probability bound on the Hessian
Revised: F High-probability bound on the Hessian

Original: In this section we obtain a high-probability bound on the Hessian of ln (cid:101) p t , i.e., the Jacobian of thescore function.
Revised: In this section we obtain a high-probability bound on the Hessian of ln (cid:101) p t , i.e., the Jacobian of the score function.

Original: To see why we expect Hessian to usually be smaller than the worst-case bound given by Lemma A.13,note that we can express (27) and (28) as[Equation] [Equation]
Revised: To see why we expect Hessian to usually be smaller than the worst-case bound given by Lemma B.13, note that we can express (27) and (28) as

Original: [Equation]
Revised: [Equation]

Original: where X ∼ µ and Y = X + σξ , ξ ∼ N (0 , I d )
Revised: where X ∼ µ and Y = X + σξ , ξ ∼ N (0 , I d )

Original: We expect that the random variable Y − X isdistributed as N (0 , σ 2 I d ) , which suggests that the covariance (50) may be bounded by 1 σ 2 ratherthan 1 σ with high probability.
Revised: We expect that the random variable Y − X is distributed as N (0 , σ 2 I d ) , which suggests that the covariance (50) may be bounded by 1 σ 2 rather than 1 σ with high probability.

Original: Indeed, we can easily construct an example where the worst case of
Revised: Indeed, we can easily construct an example where the worst case of

Original: Lemma A.13 is attained—for example, µ = 12 ( δ − v + δ v ) for ∥ v ∥ 2 = R , at x = 0 —but this point hasexponentially small probability density under µ ∗ φ σ 2 .
Revised: Lemma B.13 is attained—for example, µ = 12 ( δ − v + δ v ) for ∥ v ∥ 2 = R , at x = 0 —but this point has exponentially small probability density under µ ∗ φ σ 2 .

Original: The following lemma uses a ε -net argument to bound the operator norm of the variance of a conditionaldistribution, with high probability.
Revised: The following lemma uses a ε -net argument to bound the operator norm of the variance of a conditional distribution, with high probability.

Original: Lemma E.1.
Revised: Lemma F.1.

Original: Suppose X is a R d -valued random variable over the probability space (Ω , G , P ) , and
Revised: Suppose X is a R d -valued random variable over the probability space (Ω , G , P ) , and

Original: F ⊆ G is a σ -subalgebra.
Revised: F ⊆ G is a σ -subalgebra.

Original: If X is subgaussian, then
Revised: If X is subgaussian, then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Jensen’s inequality and Markov’s inequality, for any v ∈ S d − 1 ,
Revised: By Jensen’s inequality and Markov’s inequality, for any v ∈ S d − 1 ,

Original: [Equation]
Revised: [Equation]

Original: where the last inequality follows from taking c =
Revised: where the last inequality follows from taking c =

Original: ∥
Revised: ∥

Original: X ∥ ψ 2 .
Revised: X ∥ ψ 2 .

Original: Now take a 12 -net N of S d − 1 of size ≤ 5 d[Ver18, Cor.4.2.13].
Revised: Now take a 12 -net N of S d − 1 of size ≤ 5 d [Ver18, Cor.

Original: By a union bound,
Revised: By a union bound,

Original: [Equation]
Revised: [Equation]

Original: when we take λ = ∥ X ∥ ψ(cid:114) ln(cid:16) 2 · 5 d ε (cid:17) .
Revised: when we take λ = ∥ X ∥ ψ [Equation] ln

Revised: [Equation]

Revised: [Equation]

Revised: d ε (cid:17) .

Original: By [Ver18, Lemma 4.4.1], the operator norm can be boundedby the norm on an ε -net,
Revised: By [Ver18, Lemma 4.4.1], the operator norm can be bounded by the norm on an ε -net,

Original: [Equation]
Revised: [Equation]

Original: where the second inequality holds when A is symmetric.
Revised: where the second inequality holds when A is symmetric.

Original: The result follows from applying this to663 E [ v ⊤ XX ⊤ v |F ] .
Revised: The result follows from applying this to E [ v ⊤ XX ⊤ v |F ] .

Original: From this we obtain the desired high-probability bound.
Revised: From this we obtain the desired high-probability bound.

Original: Lemma E.2.
Revised: Lemma F.2.

Original: There is a universal constant C such that the following holds.
Revised: There is a universal constant C such that the following holds.

Original: For any startingdistribution (cid:101) P 0 , letting (cid:101) P t be the law of the DDPM process (1) at time t , we have
Revised: For any starting distribution (cid:101) P 0 , letting (cid:101) P t be the law of the DDPM process (1) at time t , we have

Original: [Equation]
Revised: [Equation]

Original: Note that there is no dependence on the radius.
Revised: Note that there is no dependence on the radius.

Original: Proof.
Revised: Proof.

Original: Apply (50) with µ = M m t
Revised: Apply (50) with µ = M m t

Original: ♯ (cid:101) P 0 to obtain ∇ 2 ln (cid:101) p t .
Revised: ♯ (cid:101) P 0 to obtain ∇ 2 ln (cid:101) p t .

Original: Noting that Y − X ∼ N (0 , σ 2 I d ) issubgaussian with ∥ Y − X ∥
Revised: Noting that Y − X ∼ N (0 , σ 2 I d ) is subgaussian with ∥ Y − X ∥

Original: ψ 2 ≤ C 2 σ for some universal constant C 2 , the result follows from
Revised: ψ 2 ≤ C 2 σ for some universal constant C 2 , the result follows from

Original: Lemma E.1.
Revised: Lemma F.1.