Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers
Authors: Yuchen Liang
Peizhong Ju
Yingbin Liang
Ness Shroff
Venue: ICLR 2025 Poster (2025)
Keywords: generative models
denoising diffusion probabilistic model (DDPM)
convergence analysis
zero-shot conditional sampling
model mismatch
Primary area: learning theory

Abstract: The denoising diffusion model has recently emerged as a powerful generative technique, capable of transforming noise into meaningful data. While theoretical convergence guarantees for diffusion models are well established when the target distribution aligns with the training distribution, practical scenarios often present mismatches. One common case is in the zero-shot conditional diffusion sampling, where the target conditional distribution is different from the (unconditional) training distribution. These score-mismatched diffusion models remain largely unexplored from a theoretical perspective. In this paper, we present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers, focusing on target distributions with finite second moments. We show that score mismatches result in an asymptotic distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise. Interestingly, the derived convergence upper bound offers useful guidance for designing a novel bias-optimal zero-shot sampler in linear conditional models that minimizes the asymptotic bias. For such bias-optimal samplers, we further establish convergence guarantees with explicit dependencies on dimension and conditioning, applied to several interesting target distributions, including those with bounded support and Gaussian mixtures. Our findings are supported by numerical studies.

Published as a conference paper at ICLR 2025
THEORY ON SCORE-MISMATCHED DIFFUSION MODELS AND ZERO-SHOT CONDITIONAL SAMPLERS
Yuchen Liang', Peizhong Jul,
Yingbin Liang!, Ness Shroff
The Ohio State University Universitty of Kentucky
ABSTRACT
The denoising diffusion model has recently emerged as powerful generative technique, capable of transforming noise into meaningful data. While theoreti- cal convergence guarantees for diffusion models are well established when the target distribution aligns with the training distribution, practical scenarios often present mismatches. One common case is in the zero-shot conditional diffusion sampling, where the target conditional distribution is different from the (uncon- ditional) training distribution. These score-mismatched diffusion models remain largely unexplored from a theoretical perspective. In this paper, we present the first performance guarantee with explicit dimensional dependencies for general score- mismatched diffusion samplers, focusing on target distributions with finite second moments. We show that score mismatches result in asymptotic distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise. Interestingly, the derived convergence upper bound offers useful guidance for designing novel bias bias-optimal zero-shot sampler in linear conditional models that minimizes the asymptotic bias. For such bias-optimal samplers, we further establish convergence guarantees with explicit dependencies on dimension and conditioning, applied to several interesting target distributions, including those with bounded support and Gaussian mixtures. Our findings are supported by numerical studies.
INTRODUCTION
Generative modeling stands as a cornerstone in deep learning, with the goal of producing samples whose distribution emulates that of the training data. Traditional approaches encompass variational autoencoders (VAE) (Kingma& & Welling 2022), generative adversarial networks (GANs) (Goodfellow [ct al. 2014), normalizing flows (Rezende Mohamed, 2015), and others. Recently, diffusion models, especially the denoising diffusion probabilistic models (DDPMs) (Sohl-Dickstein al. 2015 Ho
jet al.] 2020). have emerged as particularly compelling generative models, gaining widespread acclaim for their stable and cutting-edge performance across various tasks, such image and video generation (Ramesh et al. 2022 Rombach et al. (022).
In ideal situations, the training and target distributions of generative models match each other. However, this often does not hold in practice, where distributional mismatch between the training and target distributions can occur due to various reasons such as possible privacy constraints, need for computational efficiency, and knowledge gap between training and sampling processes. Specifically for diffusion models, such mismatches exhibit between the scores obtained from the training data and the scores of the target distribution from which we want to generate samples. One common scenario that existing studies primarily focus on is conditional diffusion models in image generation tasks (see Croitoru et al. (2023); [Li et al. (2023) y Moser et al. (2024) for surveys of diffusion models in computer vision). Different from unconditional image generation, conditional image samplers aim to generate images that are consistent with the given information, either a text-prompt (as in text-to-image synthesis) or sub-image (as in image super-reso super-resolution). For example, in image super-resolution, given the input of low-resolld www-esolutionn image, the goal is not to generate some arbitrary high-resolution image but the whose corresponding low-resolution part matches the given input. Here the diffusion models are well-trained on the unconditional distribution of bigh-resolution images, whereas the target distribution is the conditional distribution given the low-resolution input. If one

---

Published as a conference paper at ICLR 2025
uses these well-trained unconditional scores to generate conditional samples, there will be mismatch at each step of the sampling process.
One class of methods to tackle the conditional sampling problem is to include extra-guided training, where a modified score function trained with the extra knowledge of the conditioning information
(Dhariwal & Nichol, 2021] Ho & Salimans 2022). On the theory side, several recent works (Yuan
(et al. [023] Wu et al. 2024, Fu et al. 2024) provided performance guarantees for such conditional diffusion samplers, where score guidance is obtained through extra training based on the conditional information. However, the additional guided training in these samplers requires extra computations and needs to be conducted for every image conditioning, which may not be efficient in practice.
Alternatively, zero-shof conditional image samplers arise prevalent approach (e.g.. Choi et al. (202I); Chung et al. (2022b at 2023); Wang et al. (2023): Song et al. (2023a); Fei et al. (2023) for training-free conditional generation given well-traind unconditional scores. For each conditioned image, zero-shot samplers require no additional training to modify the scores. Instead, they adjust the scores during sampling by calculating rectified scores based on conditional information to mitigate the mismatch between the oracle conditional scores and the approximated ones _ Despite their empirical promise, theoretical guarantee on these zero-shot samplers is largely unexplored. In Gupta et al.) (2024), the authors provided a super-polynomiie lower bound for zero-shot sampling result. In Xu & Chi (2024), the authors proposed and analyzed a plug-and and-play conditional sampler. Their amalysis relies the properties of the Markov transition kernel specific to their plug-am blug-and-yn and--laa play model, which does not appear to be applicable to several widely used zero-shot samplers, such as Come-Closed Miiter-Dffuue-FFee Faster (CCDF) (Chung et al., 2022b) and Denoising Diffusion Null-space Model (DDNM) [Wang et al. 2023). Therefore, there is a need to provide the performance guarantee for those popular zero-shot conditional samplers.
converse
In this paper, we address two key theoretical research gaps in zero-shot score-mismatched diffusion models: (i) We provide performance guarantees for general core-mismatched diffusion models, extending their applicability beyond the primary focus of existing theoretical studies on the special case of conditional image generation. (ii) We analyze zero-shot conditional diffusion models, which are generally applicable to existing zero-shot samplers such as CCDF (Chung et al.] (0226) and DDNM (Wang al. 2023) which the analysis in Xu & Chi 2024) is not applicable (as discussed above).
1.1 OUR CONTRIBUTIONS
Technically, the main challenge due to mismatched scores is to analyze the expected tilting factor (Liang et al.) 2024) under a mean-perturbed Gaussian, providing an upper bound of the asymptotic orders of all Gaussian non-centralized moments. Our detailed contributions are as follows.
We provide the first asy-psympt con- vergence bound on the KL divergence between the target and generated distributions when there is mismatch between the sampling and target scores in DDPM samplers, for general target distributions having finite second moments. We show that the score mismatch at each diffusion step introduces an asymptotic distributional bias that is proportional to the accumulated mismatch. We also provide the first explicit dimensional dependency when the sixth moment of the target distribution exists. Our result is applicable to general forms of mismatch between the target and training scores, which greatly extend the focus of the existing theoretical research on conditional score mismmmhisttch diffusion models.
Convergence of General Score-Mismatched DDPM:
We then apply our results to zero-shot conditional DDPM samplers, as long the conditioning involves certain deterministic or random transformations of the data. This provides the first theoretical guarantees for several existing shot samplers, such as CCDF (Chung et al. (022b) and DDNM (Wang et al. 2023). Notably, the theory in Xu & Chi (2024) does not apply to these samplers, as their analysis relies the properties of the Markov transition kernel specific to their plug-and-play model. In contrast, our approach is based on the tilting-factor analysis from Liang et al. (2024), which is applicable general mismaacest DDPM models. Moreover, the theory Xu & Chi (2244) is limited to cases where the measurement log- ligeliklddoodd function is differentiable and bounded and does not provide explicit dependencies on the data dimension. In contrast, our results do not require
Note, however, that some zero-shot methods, such as DPS CChung et al. might induce additional computational costs during sampling.
2023 and TIGDM (Sog al.
2023a
2

---

Published as a conference paper at ICLR 2025
the measurement log-likelihood function to be differentiable bounded and explicitly characterize the dependencies on the data dimension.
Novel Design of Bias-Optimal Zero-shot Sampler BO-DDNM:
Inspired by our convergence analysis of score-mismatched DDPM, we design a novel zero-shot conditional sampler, called the BO-DDNM sampler, which minimizes the asymptotic bias for linear conditional models. Such sampler coincides with the regular DDNM sampler (Wang et al., (023) when there is no presence of measurement noise, and achieves faster convergence than both the DDNM and DDNM samplers under measurement noise, as shown by our theory and numerical simulations.
Theory for BO-DDNM with Explicit Parameter Dependencies: We provide the convergence bound for the proposed BO-DDNM sampler with explicit dependencies on the dimension d as well as the conditional information y. for various interesting classes of target distributions including those having bounded support and Gaussian mixture. For the case of Gaussian mixture, we further show that three factors positively affect the asymptotic bias: (1) the variance of the measurement noise, (2) the averaged distance between y and the mean of each Gaussian component, and (3) the corresponding correlation coefficient for each component.
1.2 RELATED WORK
We provide a summary of works addressing unconditional and score-matched diffusion models in Appendix B Below we discuss related works conditional diffusion models which are closely related to our study here.
Extra-Guided Training:
In order achieve conditional sampling using DDPM models in practice, one method is to introduce conditional guided training, where one either uses existing classifier (a.k.a., classifier guidance) (Dhariwal & Nichol 202T) or jointly trains the unconditional and condi- tional scores (a.k.a., classifier-fres guidance (CFG)) (Ho & Salimans (022). Here guidance term obtained to "guide" the diffusion sampling process at each step such that the sampling scores correspond to the true conditional scores. On the theory side, Wu et al. (2024) investigates the effect of the guidance strength in CFG on Gaussian mixtures, Bradley & Nakkiran 2024) shows that CFG is an instance of predictor-corretor methods, and Chidambaram et (2024) finds that CFG might fail to sample correctly on certain mixture targets. There are other theoretical works that investigate sample complexity bounds for conditional score matching for variety of target distribution models, including the conditional Ising models (Mei & Wul 2023), those supported on a low-dimensional linear subspace (Yuan al., 2023), and Hölder smooth and Gau-Gausiian conditional models (Fu et al. 2024). Other than stochastic samplers, conditional ODE sampler is proposed and studied in Chang et al. 2024), which also requires extra training of the conditional score function.
Zero-shot Samplers: To achieve conditional DDPM sampling, a popular method is to use zero-shot conditional samplers, with which one generates a conditional sample using approximated scores. These scores are calculated from the unconditional score estimates and the conditional information using simple (usually linear) functions without extra-training (Choi et al. 2021 Chung et al. 2022a b). 2023; Wang et al. 20233 Song et al. 2023a] Fei et al.) (023). The only theoretical works the performance of zero-shot DDPM conditional samplers are Xu & Chi (2024); Gupta et al. (2024). ) In
Xu & Chi 2024). diffusion plug-and and--and-pue and--aadddyyyy play sampler is proposed which alternates between diffusion sampling step and a consistency sampling step. The difference of our results from those in Xu & Chi (2024) has been thoroughly discussed in Section I.1] From alternative perspective, Gupta et al. 2024) shows that the sampling complexity with zero-shot samplers can take super-pol super-polynomial time for some worst-case distribution (among the set of distributions where smooth scores can be efficiently estimated), Instead, our result shows consistent fact that there exists a non-vanishing asymptotic distributional bias within polynomial time.
2 PROBLEM SETUP
In this section, we first provide some background on the score-matched DDPMs. Then, we introduce the score-mismatched DDPM samplers and, special example, the conditional sampling problem and zero-shot samplers.
2.1 BACKGROUND OF The goal of the mctre-matched sampling problem is to generate a sample whose distribution is close to the data distribution. To this end, the DDPM algorithm (Ho et al. 2020) is widely used, which
DDPMS
3

---

Published as a conference paper at ICLR 2025
consists of a forward process and a reverse process of latent variables. Let zo E R be the initial data, and let ne E R", V1 < t < T be the latent variables. Let Qo be the data distribution, and let Qt (resp., Q(,--) be the marginal (resp., joint) latent distribution for all 1 t T.
Forward Process: In the forward process, white Gaussian noise is gradually added to the data: It = vI- 3 + V L1 I t S T. where w, f.d.d NN(,,,) La). Equivalently, this can be expressed as:
(1 = (axs V - 11 3.Ta),
(1)
which means that under Q, the Markov chain Xo + X1 - .... - Xr holds. Define o, == 3c and 0g == II a; for all 1 < t I T. An immediate result by accumulating the steps is that (1) = (Mx V (1 (1-6a ā,) Ha). ) or, written equivalently. It Vā, + -V- R, V1 s t t T,, where We ~ NO, La) denotes the aggregated noise at time and is independent of zp. Finally, since each W; Gaussian, each Q: (t 2 1) is absolutely continuous w.r.t. the Lebesgue measure. Let the p.d.f. of each Qt be qt. and Qt, 1, 4 and 9:-1 for 2 1 are similarly defined.
In the reverse process, the latent variable at time is first drawn from a standard Gaussian distribution: 2T ~ (0, Ta) =: Pr. Then, each forward step is approximated by reverse sampling step. At each time = T, 1,..... 1, define the true reverse process as It-1 = (1(2)) + 2t where
Reverse Process:
(0, la)- Here of 1-a1, For the typical DDPM sampling process, (1+ = yor (Ix + (1 - V log ge (3))). Equivalently, P:-1jt = (N oz Ta). Here V log ge (x) is called the score of gt, and pee (3te) is function of the score. Let be the marginal distributions of x, in the true reverse process, and let pt. be its corresponding p.d.f. w.r.t. the Lebesgue measure. Define Pt- and P 1 in a way similar the forward process.
az
In practice, one does not have access v log q: (z,), denoted as s, (z,), is used, which results in an estimated (2,, and the estimated reverse process: Tt-1
log (zz) and thus (xx). Instead, an estimate of
(x.)) + 0, Let P. be the marginal distributions of xq in the estimated reverse process with the corresponding p.d.f. Pr. Note that Pt-1je = (( Ta) and
Pr Pr. Hence, under P and P. XT 7 XT-1
Xo holds.
Performance Metrics:
In the case where Qu is absolutely continuous w.r.t. the Lebesgue measure, we are interested in measuring the sampling performance through the KL divergence between Qo and Po. defined as
KL(QI = f log dQ dQ = Ex~q [log
x(X) p(X) 2
Indeed, from Pinsker's inequality, the total-variation (TV) distance can be upper bounded as TV(Qo, Po)2 < [KL(Qo)) Po). When go does not exist, we use the Wasserstein-2 distance to measure the one-step perturbed performance, which is defined as
W) {minrens.. S || -ll (I, w) 1/2
where ITQQ, P) is the of all joint probability measures on Rd Rd with marginal distributions Q
and P, respectively. Both metrics are widely adopted (Chen et al. 2023a Benton et al.) (222a)..
2.2 SCORF-MISMATCHED DDPMS
Differently from the score-matched sampling problem, the goal of the score-mismatched problem is to sample from different target distribution from the training distribution with which we estimate the scores. Thus, there will be a mismatch between the target score and the estimated score at each diffusion step. Let Qt (t Z 0) be the training distributions used for training the score. Let Qo be the target distribution that one hopes to generate samples from, and let Q (t 2 1) be its Gaussian-perturie distributions according to the forward process in ( Define the posterior mean [2.). Note that by Tweedie's formula
under the target distributions as my (Zx) == X [X-1]
(Efron 2011), (..) J.m (ā. + (1 ax) log (.+)). Recall that P. and Pr are the sampling distributions of the true and estimated reverse process, respectively. For general score- ecore-mismatched DDPMs, leave the generic definition of (v(x:) without providing any particular expression. An
4

---

Published as a conference paper at ICLR 2025
example of (z(z is given later in c yet our general analysis does not require any particular form for pc. With these notations, the score mismatch at each step 2 can be defined as
Las (x( = I-ar (xx.i~ Xx [ma] Ex - X x =
I-aa. (me((s) - M4l (xx))))
/ar
(2)
The goal. then, is to provide an upper bound on the distributional dissimilarity between the target distribution Qo and the sampling distribution Po- We use the same metrics those defined in Section 2.T to evaluate the performance the score-mismatched DDPM.
2.3 ZERO-SHOT CONDITIONAL DDPMS
One interesting example of score mismatch is the zero-shot conditional sampling problem. Differently from the unconditional counterpart, the conditional sampling problem aims to obtain a sample that aligns in particular with the provided conditioning. Define E Iº to be the conditioned information about zo. Specifically, let y hxxo), where fu(-) is some arbitrary (deterministic random) function of only Tu (apart from independent noise). Note that general exore-mismatched DDPMs can be specialized to zero-shot conditional samplers with the following notations:
Q = 2uqy M.
My,ss
jr = Pr,y1 and A. A..y
Linear Conditional Models: In practice one commonly adopted model is the linear conditional model (Jalal et al. 2021 Wang al., 2023; Song et al. 2023a), defined
y Hag nt
where H E Rpxd (p S d) is deterministic matrix and of is the measurement noise, which is assumed to be Gaussian and independent of zo. For the case where there is no measurement noise, let a² inpainting (Wang et al. 2023), H admits simple form of 0-1 diagonal matrix, where the I's occur only on the diagonal and at those locations corresponding the provided pixels. In these scenarios, both H and y are fixed and given. The linear conditional model is studied in Section
and thus
0 almost surely. In applications like image super-resolution and
Write the Moore-Penrose pseudo-inverse of H as Hf and note that H'H H is an orthogonal projection matrix. With this notation, under 4 we can re-express the forward process in ( as
Conditional Forward Process for Linear Models:
(āa H H) + Vā.H Vā, H n + vI- Cew..
Here, since is independent of wr, for fixed To and y, we have that, for all t 1 1,
(= y) Nax Var H'HH + Va,H'y an (HH)) (1 az) (5)
Also, since the forward process is Markov chain, we have that Qqi-1,y Q for all t 12 1.
Zero-shot Conditional Sampler for Linear Models: We employ the zero-shol conditional sampler for linear conditional models in the following form: Ft-1 (x,) + TeZt, where
(xy( = yau (x, + (x:)), IL.y = - H log (2:) + (z.(
(6)
simple function of y and He computable without extra training and such that 0 for all E Rd. Intuitively, fú, characterizes the score rectification in the range space of H. Indeed, many zero-shot samplers in the literature have such Jr, (xx)'s that satisfy (6) (see Appendix D.. Now, with the linear model in (4) and the zero-shot conditional sampler in (6) the score mismatch at each time t 2 is equal to
Here St.g
(La - H¹ ())..(())
(z.(2)) = (Ia H (V log (t) ) V log (z(z))) + (Hl H)V log (x)(())) fxy( (7)
DDPM UNDER GENERAL SCORE MISMATCH
In this section, we provide convergence guarantees for general score-mismatched DDPM samplers under general target distribution Qn- Throughout this section we keep the generic definition for score mismatch A, in 2 without assuming any particular expression for pt-
5

---

Published as a conference paper at ICLR 2025
3.1 TECHNICAL ASSUMPTIONS
We will analyze general score-mismatched DDPMs under the following technical assumptions. Assumption (Finite Second Moment), There exists a constant M2 (that does depend d and 2) such that E Xo-Qq || I dM2..
The first Assumption is commonly adopted in the analyses of score-matched DDPM samplers (Chen et al. 2023a Liang et al. 2024).
Assumption 2 (Posterior Mean Estimation). The estimated posterior mean 74t at 1,..., T satisfy
E T 1= [=-a))?
21 E xx,-q. (X.) (X.))l S where 8² = =²= (
The above Assumption 2 is made for the score estimation error for the general mismatched setting. where we leave generic definitions of j4, and Tr. While the expectation is over Q2, Assumption 2 is very likely to hold when Qe is close to Qr. i.e. when the score mismatches are moderate. For zero-shot conditional samplers in linear models, this assumption is weaker than that for the estimation error for unconditional scores (see (9). Compared with the motre-macched case, the estimation error needs be achieved at higher accuracy because of the extra error term when there is score mismatch (Lemma)) 2 Such a higher level of estimation accuracy also occurs in previous theoretical studies for accelerated DDPM samplers (Li et al. 2024a).
Assumption 3 (Regular Derivatives). For all t 2 1 where q- exists, E 2 1, and [d] where |a| p 2
E xx,a. 10% lo (X)] Ex,- 0 log q- (x.)
The above Assumption 3 is useful for our tilting-facter based analysis, which guarantees that all (higher-order) Taylor polynomials of log & are well controlled in expectation. It is rather soft, and can be verified when Qo has finite variance (under early-stopping (Liang et al. 2024).
Assumption 4 (Bounded Mismatch). For all t 2 where q:-1 exists, and 2 2,
E Ex.-. |1 (X.)) ((())))=((()))
The above Assumption 4 is used characterize the amount of mismatch each time 2 1. The Gg II 0g. is necessary for the overall bias to be bounded.
In the paper, Assumptions 3 and have been established in two cases of zero-shot conditional
sampling: (i) where Qo has bounded support for any H. using special a; in (8) (see the proof of
Theorem 4); and (ii) where Qu is Gaussian mixture and // = (4, 0) (see Lemma 8). For Case (i), the assumption that Qu has bounded support has wide applicability in practice (e.g., images (Ho) et al. 20200] Wang et al., 2023
) and is commonly made in many theoretical investigations of the score-matched DDPM (Li al. 2024a|c). Finally, note that when ğo does not exist (e.g., for images
(Ho et al. 2020
Wang et al.) 2023)), Assumptions 3 and are required only for L. 2 2.
3.2
CONVERGENCE BOUND
Before presenting the main result, we first define a set of noise schedule as follows.
Definition (Noise Schedule). For all sufficiently large T, set the step size or 's to satisfy
1-ay S 2 V1 VI and āT = II ai = = = T
An example of a that satisfies Definition is at clog T Vt 1 with 1. Then,
clogT) ãT (1 T
- =exp (Tlog clog T' = O
( -= o (T-1).
The following Theorem T provides an upper bound on the KI.-divergence between the target distri- bution Qo and the sampling distribution Po. as a function of (general) score-min acore-mismatch A, at each time >> 1. Theorem iss the first convergence result for score-mismatched DDPM samplers for any
smooth Qo that has finite second moment (along with some mild regularity conditions).

---

Published as a conference paper at ICLR 2025
Theorem (DDPM under Score Mismatch), Suppose that Qu has p.d.f. qu which is analytic, and suppose that Assumptions Are satisfied. Then, with the a chosen to satisfy Definition 7 the distribution Po from the score-mismatched DDPM satisfies
KL(Qol(Po)
S Woracle + Walas + Wyanishs
where
W ΣEs tt= 2a
(1-0r)
E x
Tr 2 log (m + (log T)
Whias = Et=i( - -aa) -x)
|xx())
W
-1=l
1-o6 E
xx Van
V log qt (m= (e
(I-at)* B
(x( log 70
201 1==
Q.
=1=1 3/2
(1-a.)²
E xx
Σ 0% log log
+ Σ j=j= 1 Oj qog::-1 (m)
+ m V (log (1og))
When Qu does not have a p.d.f., similar upper bound is applied to KL((11|P,) such that W2(Q1, $ (1 an (see Corollary in Appendix F.5).
To explain the three error terms in Theorem
Waracle captures the error assuming that one has access to (a close estimate of) V log 0t, VI > 1. This error is independent of the score mismatch A,, and it
decays as Ô(T-1) under Assumption 3 Liang et al. 2024 Theorem 1). The remaining two error
terms Whias and Wvanish arise from the mismatched sampling process. Both terms become zero A, for all 1, which corresponds to the score-matched case. Under Assumptions and 4 Wyanish decays Ö(T-1) under additional mild condition (see Lemma in Appendix G and Waias asymptotically approaches constant. Combining all three terms, score mismatch causes an asymptotic distributional bias between Qo and Pa-
To further understand Waias note that a usually summable under Assumption I (cf. Lem- mas 7 and 10p. Thus, the bias can be further upper-bounded by the maximum step-wise mismatch m E x,-Q. [x.)(
In case that (x(x)) = Jau (æ, + (1 (x)g. (as for the zero-shot sampler in ( ) define measure P. such that ge (z,) V log (z,). Then, from
Êx,- 2x,)) =x
V log X
where P) denotes the Fisher divergence (or called relative Fisher information) between Q and P. In Section 4 this distributional bias Whiss inspires us to design a novel zero-shot DDPM sampler, the BO-DDNM sampler, that minimizes the asymptotic bias.
Next provide upper bound with explicit dimensional dependency, for any Qo that has finite sixth moment such as Gaussian mixture Qo's 's and those Qn's having bounded support. To this end, we consider special noise schedule first proposed in Li al. (2024c):
01 55 az
clog min
(1 elog
1 V2 V2
for any constants (c, d) such that > and 6e > 1. Note that this noise schedule corresponds to
carly-stopping in the literature (Chen et al., 2023a Benton al., (024a). With the a, in (8) we can show that the regularity condition Assumption 3 holds for a quite general set of distributions (see Lemma 5 in Appendix G).
Theorem 2. Suppose that EE Exo~Qo | S d Further, suppose that As satisfies that
Ex,~Q. [x.)( S Ta or det with some 7, 2 1 for all t 2 2. Then, if the estimation 2 and A, satisfies Assumption 4 with the ot in ( such that < 1 and
error satisfies Assumption
7

---

Published as a conference paper at ICLR 2025
cx ex log(1/))), we have, for some Q1 such that W2(Q1. s ód,
KL(QillP)) 200 (1 (og
+ max [ 122 d2 (r-1) (2 (loR (loRT
-2/22- T)..
Note that Theorem 2 provides the first performance guarantee with explicit dimensional dependence for general score-mismateen DDPMs. Here the finite sixth moment is a technical condition to guarantee small expected difference of the first-order Taylor polynomial in case mismatched scores (see Lemma in Appendix G). Later, Theorem 2 will be useful to provide guarantees for zero-shot conditional samplers under linear models (Theorem
SERO-SHOT CONDITIONAL DDPM SAMPLERS
As we discuss before, an important scenario of score-mismatched diffusion models is the zero-shot conditional problem, where certain information is given. In this section, we apply our general results for score-mismatch DDPMs in Section to studying zero-shot conditional DDPM samplers. In the following. we are particularly interested in the linear conditional model in 4 We take the same Assumptions 3 and (albeit with changed notations), and further adopt the following common
assumption the unconditional score estimation (Chen et al., 2023a d: Liang et al. 2024
Assumption (Estimation Error of Unconditional Score). Suppose that st satisfies
+ E' Ex,~Qri [)s)(X,) v log ye (X.)) < 6², where 8² = Ö(T-2).
Note that, with the zero-shot sampler defined in (6. since || H Hi|| = 1, we have, Var E Rd.
(1 ax)²
at)² (1-aa)² ||s, Vloga.l (9)
-
|P Myyl 1|
[La - H H) v Vlog::)||
ax
a₁
Therefore, Assumption [ directly implies Assumption and thus Theorem (as well as Corollary I still holds under Assumptions and to
4.1 A NOVEL BIAS-OPTIMAL ZERO-SHOT SAMPLER
Guided by the performance guarantee characterized in Theorem T we will propose a novel optimized zero-shot condition sampler. With the zero-shot sampler defined (6). the goal is to choose the Sta function that minimizes the convergence error for each E R and 1.
Specifically, it is observed in Theorem that the convergence error in terms of the Kl--divergence will have an asymptotic distributional bias given by Whiss As follows, we characterize an optimal fr,y that minimizes Whias, which thus yields corresponding bias-optimal zero-shot sampler.
Theorem 3. Define Ego.v == a.0 + (1 (1- -d) For any Qo and 2 have
v log 9:ly (Fel)) (Va, xz) + I-an
van
(La H H'H))EQul [oo v).
Also, recall the sampler in 66) and define fū.g as
(i,)( == ETp.. (VorHly H Hx.)
(10)
Also recall Dtyy from Then, fr, satisfies that, for all t 2 and fixed E RP,
Si arg min
|| Qelg-almon surely.
The sampler Jr (2x)) defined in (10) provides a bias-optimal zero-shot conditional DDPM sampler. In the case with o, = 0, such an optimal sampler coincides with the regular DDNM sampler in Wang et al. (2023) (see Appendix DD. Thus, we call this sampler as Bias Optimal (BO) DDNM sampler. With
10 we can also calculate the minimum step-wise mismatch as
V log (1zx(Xe) 2
fuy min
+) Ex,,Qui || = Fx -
(x.).) (1)-HHH)*
8

---

Published as a conference paper at ICLR 2025
0.30
10
0.25
KLIOuuilPPp
20-1
A.L, DONY
DONM by
0.20
0.20
B.0.,
DONM
0.35
0.15
oye 0.01 BD DOWN 20
80 D030
0.30
010
0.05
0.00
0.2
0.6
Each element of
Figure 1: Comparison of BO-DDNM, DDNM
Figure 2: Distributional bias function of the conditioning y (left) and the correlation coeffi- cient p (right) for Gaussian Qo-
and DDNM for Gaussian (left) and Gaussian mixture (right) Qu under measurement noise.
which is the projected Fisher divergence between Qiyy and Q. on range( H²
In the following lemma, we provide the performance bound for BO-DINM when Qo has bounded support. For comparison, we also derive the theoretical performance of vanilla DDNM, denoted as fig.
Theorem 4 (BO-DDNM vs. DDNM). Suppose that ||Xoll² < R a.s. under Qo. Suppose that Assumptions and Shold Then, with the conditional sampler fī, in (TO) Theorem 2 holds with
7 == and 2. Also, with the conditional sampler fly = (1 āz) 1 (Va,lly H H Hx.), , if
further | | S then Theorem 2 holds with y and
Theorem establishes the first result applicable to DDNM-type zero-shot conditional samplers for any linear conditional models on those target distributions having bounded support.
Advantage of BO-DDNM over DDNM and DDNM When there positive measurement noise, Theorem indicates that our BO-DDNM sampler that uses ft.y fi.y enjoys smaller asymptotic
bias than DDNM that uses fiy with the in (8-2 vs. 8 8-.). Note that the DDNM sampler corresponds to fL, fiNy (see Appendix D Such an advantage is also demonstrated by our numerical experiment. In Figure I we numerically compared modified conditional zero-shot sampler (as given in (10) with the DDNM and DDNM sampler for both Gaussian and Gaussian mixture Qo s at different levels of measurement noise. It is observed that the optimal BO-DDNM sampler achieves a much lower bias than both the DDNM and the DDNM samplers numerically, especially when of becomes large.
4.2 BO-DDNM SAMPLER FOR GAUSSIAN MIXTURE Qo
In this section, we focus the convergence dependency other system parameters of the BO- DDNM sampler, including the chosen In particular, we restrict our attention to Gaussian mixture Qo's 's and special conditional model, where // (Tp 0). This choice can be seen in many applications, such as image super-resolution and inpainting (after reorganizing the pixels), where 4p corresponds to the locations of the given pixels (Wang et al., [023] Song al.) [023a). We assume positive measurement noise. We introduce the notation Dollad to denote the variance components that correspond the space
b where a, b (y. 7)
The following Proposition I gives upper bound the asymptotic bias for Gaussian mixture Qo-
Proposition 1. Suppose that Qo is Gaussian mixture with equal variance, whose p.d.f. is given by (o(x)) = EA' (n where Jo.n is the p.d.f. of N(Ho, Ex) and E [0, 1] is the mixing coefficient with EN Tn Suppose that (Ip 0), and adopt fr, in (I0) and an
in Defiiittion I Write X₁ 2 ... 2 Aa > 0 and À₁ 2 ... da-p > 0 as the eigenvalues of Eg and [Eo]g respectively. Then,
E Ex.Q., || (X.)]
S ā.d + a² min()s-
[X 13 max (((Ar + EN Ta ||ll' y H1 HH' 10
S ā. (d+ nm Hty - Ht H Hp
9

---

Published as a conference paper at ICLR 2025
Proposition I indicates that three factors affect (an upper bound on) the asymptotic bias. (i) The measurement noise variance oy determined by the system nature has an increasing effect on the bias.
(ii) The averaged distance Σm=1 xx y H Ho.n 2 between H' and H Po.n captures the quadratie dependency in y. illustrated in the left plot of Figure 2 (iii) The correlation between Hzo and (La - H H) 2 of each mixture component contributes positvely to the bias, which is contained
in the factor ||[o|>=]|1 To see this, consider oy and specific Gaussian example with d [in.Aa..]=
o
p011102) 022
p 1, and Eg-
As the correlation cocfficient p increases, Eg becomes closer to be singular, and thus Xa decreases to Also, ||[ - /² o increases quadratically with p. Hence, this factor |[[olxal|e grows unboundedly as p > 1, as does Ex,-Q., | . (xx,)] min(Aa,1]²
22
Such dependency on the correlation is illustrated numerically in the right plot of Figure
The following theorem characterizes the conditional KL divergence when Qu is mixture Gaussian. In particular, we can show Assumption holds with any ar that satisfies Definition I when Qu is Gaussian mixture (see Lemma in Appendix I.5)
Theorem 5. Suppose the same conditions in Proposition ! hold and oz > 0. Suppose that As- sumption holds. Take fiy in (TO) and on that further satisfies Ei= (1 ar)ā, + o(1) Then,
KI Pa $ (d En Tn HHy H..Hplllll)
(12 + M Tn Hyy Hi HHo. () + vd En FR |iy HtH (log T)..
Although Proposition and Theorem assume H = (4p 0), extension to general H is straightfor- ward by modifying the proof of Lemma 8 and using the fact that HH H = 1 H H|| 1.
This is the first convergence result for zero-shot samplers where explicit dependency on the condi- tioning y is derived for Gaussian mixture targets. Note that the extra condition on a can be verified for both constant ar (Lemma 10) and that in (8) (Lemma)) 7) Among the three terms in Theorem 5 the first term is the asymptotic bias analyzed Proposition T Since the last two terms decrease to zero as T + 0c, the asymptotic KL divergence will also approach some non-zero limit of order d.
The proof of Theorem 5 is non-trivial because from Theorem I we need to figure out the dependency on y in all first three orders of partial derivatives Gaussian mixture density, which is generally hard to express. To this end, we restrict focus to particular linear model where explicit dependency can be sought. The result can be extended to the case of of = with the a in g (see Remark 2).
CONCLUSION
In this paper, we have provided convergence guarantees for the general score-mismatched diffusion models, which are specialized to zero-shot conditional samplers. For linear conditional models, we also designed an optimal BO-DDNM sampler that minimizes the asymptotic bias, for which we showed the dependencies on the system parameters. One future direction to explore zero- shot samplers that use higher-order derivatives of the log-densities, which might achieve better convergence results.
ACKNOWLEDGMENT
This work has been supported in part by the U.S. National Science Foundation under the grants: DMS-2132 -13414,, CCF-1900145 NSF AI Institute (AI-EDGE) 21 12271, CNS-2312836, CNS-2225561, ONR grant N000142412729, and was sponsored by the Army Research Laboratory under Cooperative Agreement Number W91 INFF2232--0225 The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory the U.S. Government. The U.S. Govermment authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly d-linear con- vergence bounds for diffusion models via stochastic localization. In The Twelfth International Conference on Learning Representations, 2024a.
Joe Benton, George Deligiannidis, and Arnaud Doucet. Error bounds for flow matching methods. Transactions on Machine Learning Research, 2024b.
Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is predictor-corrector. arXiv preprint arXiv:2408.0990 2024.
Stefano Bruno, Ying Zhang, Dong- Young Lim. Ömer Deniz Akyildiz, and Sotirios Sabanis. On diffusion-based generative models and their enror bounds: The log-concave case with full conver- gence estimates. arXiv preprint arXiv:231 1.15514,. 2023.
Jinyuan Chang, Zhao Ding, Yuling Jiao, Ruoxuan Li, and Jerry Zhijian Yang. Deep conditional generative learning: Model and error analysis. arXiv preprint arXiv:2402.01466, 2024.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative model- ing: user-friendly bounds under minimal smoothness assumptions. In Proceedings of the 40th International Conference Machine Learning, 2023a.
Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. In Proceedings of the 40th International Conference on Machine Learning. 2023b.
Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ODE provably fast. Thirty-seventh Conference on Neural Information Processing Systems, 2023c.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, 2023d.
Sitan Chen, Giannis Daras, and Alexandros G. Dimakis, Restoration- beyond linear diffu- sions: a asymptoicc analysis for ddim-type samplers. Proceedings of the 40th International Conference on Machine Learning, 2023e.
Sitan Chen, Vasilis Kontonis, and Kulin Shah. Learning general gaussian mixtures with efficient score matching. arXiv preprint arXiv:2404. 18893, 2024.
Xiuyuan Cheng, Jianfeng Lu, Yixin Tan, and Yao Xie. Convergence of flow-based generative models via proximal gradient descent in wasserstein space. arXiv preprint arXiv:2310. 2023.
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. What does guidance do? fine-grained analysis in simple setting. arXiv preprint arXiv:2409. 13074, 2024.
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), 14366 14367-14376, October 2021.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. In Advances in Neural Information Processing Systems, 2022a.
Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come-closer-diiffus--sssee accelerating condi- tional diffusion models for inverse problems through stochastic contraction. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022b.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference Learning Representations, 2023.
11

---

Published as a conference paper at ICLR 2025
Frank Cole and Yulong Lu. Score-based generative models break the curse of dimensionality in learning a family sub-gaussian distributions. In The Twelfth International Conference on Learning Representations, 2024.
Giovanni Conforti, Alain Durmus, and Marta Gentiloni Silveri. Score diffusion models without early stopping: finite fisher information is all you need. arXiv preprint arXiv:2308. /2240, 2023.
F. Croitoru, V. Hondru, R. Ionescu, and M. Shah. Diffusion models in vision: A survey. /EEE Transactions on Pattern Analysis & Machine Intelligence, 45(9):
Sep 2023. Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications to score-based generative modeling. In Advances in Neural Information Processing Systems, volume 34, pp. 17695-17709, 2021.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. arXiv preprint 0rXiv:2105.05233, 2021.
Bradley Efron. Tweedie's ' formula and selection bias. Journal of the American Statistical Association, 106(496):1602-100)) 1614, 2011.
Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 9935-9946, June 2023.
John Torjus Flåm. The linear model under gaussian mixture inputs: Selected problems in communi- cations, 2013.
Hengyu Fu, Zhuoran Yang, Mengdi Wang, and Minshuo Chen. Unveil conditional diffusion models with classifier-free guidance: A sharp statistical theory. arXiv preprint arXiv:2403.11968, 2024.
Xuefeng Gao and Lingjiong Zhu. Convergence analysis for general probability flow odes of diffusion models in wasserstein distances. arXiv preprint arXiv:2401.17955, 2024.
Xuefeng Gao, Hoang M. Nguyen, and Lingjiong Zhu. Wasserstein convergence guarantees for a general class of score-based generative models. arXiv preprint arxiv:2311.11000 2023.
Yuan Gao, Jian Huang, Yuling Jiao, and Shurong Zheng. Convergence of continuous normalizing flows for learning probability distributions. arXiv preprint arXiv:2404.00551, 2024.
Khashayar Gatmiry, Jonathan Kelner, and Holden Lee. Learning mixtures of gaussians using diffusion models. arXiv preprint arXiv:2404.18869 2024.
Ian Goodfellow, Jean Pouget-Abadie. Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, 2014.
Shivam Gupta, Ajil Jalal, Aditya Parulekar, Eric Price, and Zhiyang Xun. Diffusion posterior sampling is computationally intractable. arXiv preprint mxXiv:2402.12727 2024.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851, 2020.
Daniel Zhengyu Huang, Jiaoyang Huang, and Zhengjiang Lin. Convergence analysis of probability flow ode for score-based generative models. arXiv preprint arXiv:2404.09730, 2024.
Aapo Hyvärinen. Estimation of normalzzdl statistical models by score matching. Journal of Machine Learning Research, 6(24):695-709, 2005.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G Dimakis, and Jon Tamir. Robust compressed sensing mri with deep generative priors. Advances Neural Information Processing Systems, 2021.
12

---

Published as a conference paper at ICLR 2025
Yuling Jiao, Yanming Lai, Yang Wang. and Bokai Yan. Convergence analysis of flow matching in latent space with transformers. arXiv preprint arXiv:2404.02538, 2024.
Diederik P Kingma and Max Welling. arXiv:1312.61 14, 2022.
Auto-encoding variational bayes.
arXiv preprint
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. In Proceedings of The 34th International Conference on Algorithmic Learning Theory, volume 201, pp. 946-985, 2023.
Gen Li, Yu Huang, Timofey Efimov, Yuting Wei, Yuejie Chi, and Yuxin Chen. Accelerating convergence of score-based diffusion models, provably, arXiv preprint arXiv:2403033852 2024a.
Gen Li, Zhihan Huang, and Yuting Wei. Towards mathematical theory for consistency training in diffusion models. arXiv preprint arXiv:2402.07802 2024b.
Gen Li. Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards faster non-asymptotic convergence for diffusion-basse generative models. In The Twelfth International Conference on Learning Representations, 2024c.
Xin Li, Yulin Ren, Xin Jin, Cuiling Lan, Xingrui Wang, Wenjun Zeng. Xinehao Wang, and Zhibo Chen. Diffusion models for image restoration and enhancement comprehensive survey. arXiv preprint arXiv:2308. 093888 2023.
Yuchen Liang, Peizhong Ju, Yingbin Liang, and Ness Shroff. Non-asymptotic convergence of discrete- time diffusion models: New approach and improved rate. arXiv preprint arXiv:2402.1390) 2024.
Junlong Lyu, Zhitang Chen, and Shoubo Feng. Sampling is as easy as keeping the consistency: convergence guarantee for consistency models, 2024.
Song Mei and Yuchen Wu. Deep networks as denoising algorithms: Sample-efficient learning of diffusion models in high-dimensional graphical models. arXiv preprint arXiv:2309.1 1420, 2023.
Brian B. Moser, Arundhati Shanbhag, Federico Raue, Stanislav Frolov, Sehastián M. Palacio, and Andreas Dengel. Diffusion models, image super-resolution and everything: A survey, arXiv preprint arXiv:2401.0075: 2024.
Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. In ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023.
Francesco Pedrotti, Jan Maas, and Marco Mondelli. Improved convergence of score-based diffusion models via prediction-coorcction.. arXiv preprint arXiv:2305. 141644 2023.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pp. 1530-1538, 2015.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEFE/CVE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022.
Kulin Shah, Sitan Chen, and Adam Klivans. Learning mixtures of gaussians using the DDPM objective. In Thirty-seentth Conference on Neural Information Processing Systems, 2023.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pp. 2256-2265. 2015.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. seeudoinverseguuidee diffusion models for inverse problems. In International Conference Learning Representations, 2023a.
13

---

Published as a conference paper at ICLR 2025
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In Proceedings of the 40th International Conference on Machine Learning, 2023b.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In The Eleventh International Conference on Learning Representations, 2023.
Yuchen Wu, Minshuo Chen, Zihao Li, Mengdi Wang, and Yuting Wei. Theoretical insights for diffusion guidance: A case study for gaussian mixture models. arXiv preprint arXiv:2403.06639
2024.
Xingyu Xu and Yuejie Chi. Provably robust score-based diffusion posterior sampling for plug-and- play image reconstruction. arXiv preprint arXiv:2403.17042 2024.
Hui Yuan, Kaixuan Huang, Chengzhuo Ni, Minshuo Chen, and Mengdi Wang. Reward-directed conditional diffusion: Provable distribution estimation and reward improvement. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Kaihong Zhang, Heqi Yin, Feng Liang, and Jingbo Liu. Minimax optimality of score-based diffusion models: Beyond the density lower bound assumptions. arXiv preprint arXiv: 15602, 2024.
14

---

Published as a conference paper at ICLR 2025
Appendix
Full List of Notations
16
Related Works on Unconditional DDPM Samplers
16
Details of Numerical Simulations
17
Derivation of Score Bias for Existing Zero-shot DDPM Samplers
17
D.I Come-Closer Cal-CooooDDiitee-eaaaeeee (CCDF)
17
D.2 DDNM and DDNM
17
D.3 Samplers Using Higher- Order Derivatives
19
Proof Sketch of Theorem
19
F
Proof of Theorem 1 and Corollary
20
Step 1: Bounding estimation error under mismatch
21
F2
Step 2: Decomposing reverse-step error under mismatch
21
F3
Step 3: Bounding W oruule, rev-step and hias, nev step
22
R.4
Step 4: Bounding W vanish, rev-step
22
F55
Corollary I and its proof
24
Proof of Theorem
24
Auxiliary Lemmas and Proofs in Section 3
27
Proof of Lemmal
27
Proof of Lemmal
28
Proof of Lemmal3
29
Proof of Lemmal4
30
5
Proof of Lemmal
31
H.6
Lemma 6 and its proof
34
Lemma and its proof
36
Proofs in Section 4
39
Proof of Theorem13
39
Proof of Theorem|4
40
Theorem and its proof
42
44
Proof of Proposition]
46
Proof of 55
48
15

---

Published as a conference paper at ICLR 2025
Auxiliary Lemmas and Proofs in Section
58
Proof of Proposition
58
Prool of Lemma
61
J.3
Lemma 9 and its proof
61
J.4
Lemma 10 and its proof
62
A
FULL LIST OF NOTATIONS
For any two functions f(d, 6, T) and g(d, 6, T), we write f(d, S g(d, (resp. f(d, g(d, 6, T)) for some universal constant (not depending on 6, d or 7") L < x (resp. L > 0) lim sup [[(d, 6, T g(d, 5, T) < L (resp. lim inf [f ( T) 2 L). We write f(d, 6, 6) x g 6, T) when both ,, S g(. and f(d, 6,7) g(d, T") hold. Note that the dependence on 8 and
(g((T) (resp. f(d, T,) (2(g(T))) f(d, 8) S L(d, (T)(T) (resp. f(d, 8,7) 2 2 (dd))) holds for some L(d, (possibly depending on and d). We write f(d,6,T)
retained with S. 2, x. We write f(d,
((g(T)) if lim supt If 8, T) /g(T)) = 0. We write s(d, 6, = (()()) if [(d, 6, T) = (gg()) (l)) for some constant k. Note that the Oig-O notation omits the dependence on 8 and d. In the asymptotic when e- oc, we write f(d, = = (g + if fd. 8, 2
g(d, 6, (log g(E 1)) for some constant k. Unless otherwise specified, we write X' S Kisdi d) as the i-th element of vector x E R and [A]* as the (i, j) element of a matrix A. For a function f(x) Rdd + IR we write 0.f(2) as a shorthand for - (1x) and similarly for higher moments. For a vector (resp. matrix), all norms, if not explicitly specified, are referred to 2-norm (resp. spectral norm). For vector x and matrix P, define ||>||p : Pa. For matrices A, B, Tr(A) is the trace of A, and A B means that B A is positive semi-definite. For a positive integer n., [m] == (I,...,n. n...
(loggf
B RELATED WORKS ON UNCONDITIONAL DDPM SAMPLERS
Given avme-averaged I unconditional score estimation error Hyvärinen, 2005), polynomial-timg -ime convergence guarantees have been established for wide families of target distributions (De Bortoli
ctt al.
2021
Chen et al. (2023d. Lee et al.. 2023 Chen et al. 2023a Benton et al. 2024a|Pedroot Pedrotti
et al. 2023: Conforti et al. 2023). For all target distributions with finite second moment, under L score estimation error, (d log( //)))""))))))) number of steps are sufficient to achieve =² KL divergence between the d perturbed target distribution and the generated distribution using the specially designed aooonential-deaay tteen-ttheccottam step-sizes (Benton et al. 2024a Conforti et al., 2023). The analysis usually involves applying the Girsanov change-of- -fe-uuu framework and the Fokker-Plank equation (Chen et al.) 2023d ab to either the original SDE diffusion process or some transformed process (Benton et al. 2024a; Conforti al., (023). followed by an analysis of the discretization of the continuous-time process. More recently, similar convergence guarantees have been established using non- -DE-tee techniques, such as with typical sets (Li et al. 2024c) and with tilting factor representations (Liang et al. 2024). Here the new analysis introduced in Liang et al. (2024) is applicable a larger set of step-sizes (equivalently, noise schedules) than the ones commonly used in previous analyses (Chen et al. 2023a Benton et al. 2024a Conforti et al. (023). In this paper, we employ the same analytical framework as in Liang et al. (2024).
Some other works analyzed sampling errors using a different measure (the Wasserstein-2 distance) (Bruno al. (2023) Gao et al. [023 Gao & Zhu (024).. Beyond stochastic samplers, another line of studies provided theoretical guarantees for the deterministic sampler corresponding to DDPM (Chen et al. 2023e|c Huang al. 2024). Besides, Besides,Cheng et al. (2023): Benton et al. (2024b); Jiao et al. (2024): Gao et al. (2024) provided guarantees for the closely-related flow-matching model, which learns a deterministic coupling between any two distributions. Also, Lyu et al. (2044); Li et al. (2024b) provided convergence guarantees for the closely-related consistency models (Song et al. 2023b). Finally, in order to achieve an end-to-end analysis, several works also developed sample complexity bounds to achieve the L score estimation error for a variety of distributions (Oko et al.
2023 Shah et al. 2023 Gatmiry et al. 2024 Chen et al. 2024 Cole & Lu 2024 Zhang et al. 2024
Mei & Wu 2023 Chen al. 2023b)
16

---

Published as a conference paper at ICLR 2025
C DETAILS OF NUMERICAL SIMULATIONS
In Figure D compared the performances of optimal BO-DDNM sampler (with the fiy in 10
against the DDNM and DDNM samplers (Wang al. 2023) different levels of og. For Gaussian,
we use μo
4. p = and y (0.5 0.5). We first randomly generate a positive definite
matrix Eig and uniformly sample E [0.4, 0.7), and then this correlation coefficient is enforced for any [Eg where [p] and j E {p (p+1 1...., d}. We use the noise schedule in ( with c 3 and
1. We set 5 = 0.0001 for Gaussian Qo. For Gaussian mixture, we use N = 2, d = 2, p 1. and
Tn (0.4 0.6), diag En) = -0.1 1), and p = 0.6. We further uniformly sample (u ] in the space 1) 1), We use the noise schedule in (8) with 4 and 5 0.02 for Gaussian mixture Qo. We use 150000 samples to estimate the divergence when Qo is Gaussian mixture.
In Figure we numerically verify the exact bias in KL divergence as a function of y and p for Gaussian Qu. Here Qu (0, Eo), d = and 2. Suppose that of = 0. We assume that each element of y has equal values. The correlation coefficient is enforced for any pair of x and x where E [p) and E (p + 1, 1.....d] .d. We first randomly generate a positive definite matrix Eg and then enforce the correlation condition for any x' and x² where E [p) and E {p 1,..., d). We use a sufficiently large number of steps 7 = 20000. The conditional sampler is set as ft. = fiy given (TO). The noise schedule in (8) with 3 and 6 0.0001 is used.
D DERIVATION OF SCORE BIAS FOR EXISTING ZERO-SHOT DDPM SAMPLERS
In this section we show some examples zero-shot conditional samplers proposed in the literature and in particular how they related to the formulation of interest in (6) We recall the notations !!, y. and of from Section 2.3 Also denote
0t V -xx-->( (x [x,]
M4
Tt
Val
which is the mean of the unconditional reverse-step at time 2 1.
D.1 COMF--CLLSSRR--IFUUEEFFFF
(CCDF)
We first examine the Sone-Closer--re Deee--DDfffufaaaeeeee Faster (CCDF) algorithm (Chung al. 2022b). The CCDF algorithm using DDPM samplers gives that
4 = + 0t2t,1
xt-1 = Ht H'H Vo,H'y + v1 0,21,2
1...d. N(O0Ia) Ia) are standard Gaussian random variables. Thus, the conditional mean
where 21,1. 21,2 of the update is
MLy = (I - H H) Vou-1
1
a H (og + Voll'yy 1 H1 HE,
Var
Var.
yon
which
f(o(2))
1 a- (VavvaHHly Hm.)
D.2 DDNM AND D
Next, examine the DDNM algorithm and its modified version DDNM Wang et al. 2023) We first note that the unconditional DDPM satisfies that (cf. (Ho et al. 2020 Equations (7) and (11))),
Q4 az)
az
-.t-1)
==
ToJe"
+
1 - ā
Fojz
Ft
v log (xt)
Exo~Qo ((p ( [o]
(11)
Voi
Vou
17

---

Published as a conference paper at ICLR 2025
Combining these two lines, we have 14 vaa (2g + (1 an Voog)((z))). In DDNM, 20/e is projected along the direction of the given y, which yields
to H'y y + (La H 2ole
and the corresponding conditional mean of the update becomes
vat-1(1- as) FLy 1 at (H (Ia H H)))t)))) + vas(1 -(t-1)
Toley + var(1 - a1-1)
Dt.
1- at
Thus,
Vār- ai)) -
Pt.
1-a; 8
(var-i(l-a)) ar)
y var Ō(-1)
(Ia
H'H) + H
HIa
1-a; a.
1 1- ār
ft + - a:
(Ta H) log (t4))
V
vai
Var-il1 - or)
a - ā4-1) (1 H Hz,
H
Ht H
1 1-
Vau
where (i) follows from 11 Thus, to express this conditional mean in the form of (6).
(xz( = vāl Hi y 1 az. 1-āc
[a((1 [a((1- 5(-1) 1) H H
1
(VorH'y y Ht Hs.)
ā
Here note that (1,) supported range(H H H). Also note that for DDNM, Jty = Jīy which the BO-DDNM sampler defined in (10), when there is no measurement noise (i.e., o/ = 0).
Next we investigate modified version, DDNM+ particular when = (Lp 0). To relate the notations of (Wang et al., 2023 Section 3.3 and Appendix I) with ours, note that Σ = A H, U Ip. V Id, $1....: Sp 1. spt1...., Sd 0, and a (1-o) o ao we have
1-a:
(o - -do
E ld.
p =
0
of Tapp
Otherwise, if og < yo we have
0
Σ,
ad
of T
Id-p)
Observe that the only difference on the space that supports HiH H.
From Wang et al.
2023 Equations (17) and (18)), write
sopLy
(Ia Ht HDrop + E,H.y + (Ia H Hoooe supported on range(HH H)
Thus, with similar arguments above,
P,y = Vat- (1 (t.).
Vai(1 (i.-1)
Dt
1
Cze (Ta H1 HİV log (144)(
1 -
+
/ 04t
Vou
+
- ai
(Vor(E, (Ia- F,) Hoog))
- Hzoo) Hi H
0 - ā.)
supported range
where Je is again supported range H1 H).
18

---

Published as a conference paper at ICLR 2025
D.3 SAMPLERS USING CORHER DERIVATIVES
Before we end this section, we note that the formulation in only uses (estimates of) first-order derivatives of (unconditional) log-p.d.f.s (a.k.a. unconditional score functions). This might not correspond to the optimal zero-shot sampler, and in practice there have been methods that use both first- and second-order derivatives (namely, in (xze) (3x)) to achieve hetter zero-shot sampling performance (Chung et al. [023] Song al. 2023a). Nevertheless, the second-order derivatives might be hard to obtain, which require extra machine time and memory in the calculation. We leave investigations use second-order derivatives in zero-shot conditional samplers future work.
E PROOF SKETCH OF THEOREM
We now provide a proof sketch of Theorem 1 to describe the idea of our analysis approach. The main technical challenge due to mismatched scores is analyze the expected tilting factor under a mean- mecan-petturbed Gaussian, providing an upper bound of the asymptotic orders of all Gaussian non-centralized moments. See the full proof in Appendix F
we decompose the total error as (L(Quul S E Ex [log or (Xx) For (r((X))
To begin, with Lemma
E E three terms correspond respectively to the initialization error, estimation error, and reverse-stey step error. The initialization error can be bounded by ārd in order using (Liang et al. 2024 Lemma 3) under Assumption 1
1x,,x 1-Q..-1 log Pt 11c(X, [Xx)) Pi (1x [X.]
+ + Ex, Ex,,x..1
log 1n(X, ,1x;)) These Pi 11(()_1(X+e)
Below we focus the remaining two terms.
Step 1:
Bounding estimation error under mismatch (Lemma 2). At each time 1,...,. T, (e((:--je(Ec-lll (
has an explicit expression since they are conditional Gaussians with the same variance. However, differently from the typical matched case, the mean of Pj-
(i.e., (xx)) is no longer equal to the posterior mean of Qi-1je (i.e., mxx))). Their difference is contained in (xte, whose asymptotic order needs to be upper-hounded in light of Assumption 2
Step 2: Decomposing reverse-step error under mismatch (Equation
15 First we decompose
the tilting factor as Čt,- ( ze x1-1) = Emix (F2, 1t-1) + Evan ( Ft Ft-1),
1), where
Emis == (a, (x()))
8an = (Vlogū-
log (o) (m)(x)) +
Σ
T (log 14, m. p=2
Here Emis captures the factor that contributes to the total bias within 7, Define the oracle sampling process as Pe- = N (m of Ta). Then, the reverse-step error can be decomposed as
E [,,-11] - Egg XP 1l1 [2,0-1] = (( (G.t-1) B 1iixPP-ii(ee.--1
Wench me-orp
+ E i1 [miss] - E a/e Smisl)
+ EQixP in (Sraal - E Ir (raan)
Wiece nocstrp
Wumsh rovwerp
Step 3: Bounding Woraele, rev-step and Whias, rev-step (Equations 16 and (17). Under Assumption 3
the dominant term of W Worsele rev-step is given by (Liang et al. 2024 Theorem 1). Also, the calculation
of Whins, rev-step is reduced to the difference in conditional mean, which is proportional to || (1:))]
Step 4: Bounding Wvanish, revistep (I (Lemmas and 4). To upper-bound Wvanish, rev-step- with results on the matched case in Liang et al. (2024) we need only to characterize the mean of Evan under the mismatched posterior Pt- We determine the dominant order in the expected values of all Taylor polynomials, which includes calculating all non-centralized moments. We first calculate the first three non-centralized moments (Lemma 3) and then determine the asymptotic order of all higher moments (Lemma 4.. With these, we finally locate the terms of dominating order in W Waniih, ste-
19

---

Published as a conference paper at ICLR 2025
F PROOF OF THEOREM AND COROLLARY I
Overall, the structure of the proof of Theorem is similar to that for (Liang et al. 2024 Theorem 1).
To start, note that with similar arguments (iang et al. 2024 Equation 13), an upper bound on Po) given by
KL(Ool|Po)
-KL(Or\PPT)
Σ
E
t=1
T
Σ
Ex 1 ~
T + Σ
$ (L(0rl|PPr)
E Ex.m
KL
t=1
log (Xr)
Log Pt Pt- (X, X()
XxT@r
B X Qu-
PT
P X+))
error
Term estimation error
T
log t (X
X, X X
Pt 111 (X
(12)
Term 3: reverse-siep
The last equality holds because Pr = Pr. Now, we provide an upper bound for the reverse-step error that is ready for further analysis. In the following lemma, we show that the mismatched Ø:-1j an exponentially tilted form of Pt-1/e-
Lemma 1. Fixed 2 1. For any fixed E Rdd, as long as q exists, we have
Pt- (1-1)((0t-1200ee ee (e0
q. 11 (It
Ex, (c6ca-1
where
(m) + (Vlog
(1 p=2
Here we define the p-th order term in the Taylor expansion of f(x) around μ as
T=(
xx' n
p!
where a E [d] are the indices of differentiation in which the multiplicity of E [d] is
Proof. See Appendix H.I
We abbreviate Č,.- = -. 1 (1x, 1.-)). Given the expression of Č,,- the conditional reverse-step error be bounded for any fixed x; as
(X,- (X, log
E Ex
Pt-1je (X, [xc)
=Ex, -Ex-1-Qu-1
( - log Ex, (ep-
20

---

Published as a conference paper at ICLR 2025
(8) I E x Q [(2.511 + Ex.- P
log e
Ex, 1- [1,1-1]] Ex-- 1/1 [Gr,c-1]
(13)
where in (i) we use Jensen's 's inequality and note that - log(.) is convex. Thus, from 12) we have an upper bound as
qr (XT) PT (XT)
Pr- (x E Ne X log (X- (xx)
(L(01)) s Ex Xy xr
log
+
Temm iniitalization
Term 2: estimation
E Ex, 1 - Q. Gra-1] - Ex, Pm 1
Term 3: reverse step emuur
Here, using (Liang et al. T xc,
2024
Lemma 3), the initialization error can be upper-bounded as, when
4r (XT)
E Xr~Qr
log
S E Xo
|ooll or + +0 (a7)
(xxx)
This implies that, under Assumption
I and if
Tr (Xrr
E Xr
QT
log
pr pr (Xr)
= o(T
Also, under Assumption 3] the higher-order Taylor polynomials enjoy exponential rate of decay in expectation, which is contained powers of (1 o1). Thus, we are allowed to exchange the limit (of Taylor expansion) and the expectation operators (ef. (Liang al. 2024
Lemma 1)).
Now, we upper- bound the estimation error and reverse-step error under score mismatch separately.
F.1
STEP 1: BOUNDING ESTIMATION ERROR UNDER MISMATCH
The following lemma provides an upper bound the estimation error under score mismatch.
Lemma 2. Under Assumptions aadd4 4 with the on satisfying Definition 7 we have
T
log Pt-1je(Xe-11 I max (
Σ
Ex,
VE (1)(( T)e + (log T)
121
Proof. See Appendix H.2
F.2 STEP DECOMPOSING REVERSE-STEP ERROR UNDER MISMATCH Now, decompose (2,, 1.-) Emis + Eyan where
Emis = (a, (mm (x:)),
a Vog (x)) -m(z))(( + E = (V -
Th (log Tt-, mz
p=2
(14)
Here Evan is the same tilting factor without score bias (ef. Liang et al. (024 ) Also, define an auxiliary conditional probability Pr- such that
Pr-je : ---(maaa))))
(m)((z))
1 1-a1 aq
at
which corresponds to the oracle reverse process that knows the true scores the perturbed target distributions. Thus, can decompose the expected value of (13) in the following way:
Ex,-Q. (Ex, - Ex, I-Pe-sje) [G.r-]]
21

---

Published as a conference paper at ICLR 2025
= Ex (Exmi~Qu-1 - -E Ex [5c
Wards
+ @xi~ (Exe - EX.. [misk]
Whss spy
+ Ex, 1x
Ex.-inPe Pe s1e)
[Svanl
(15)
Wusiss.ressty
F.3 STEP 3: BOUNDING WORACLE, REV-STEP AND WAIAS, REV-STEP
Among the terms above, Liang the a₂ in Definition] 1
2024 Theorem 1) shows that, under Assumption and using
Woraele, tev-stop Σ - 2 ( log ( (
lz (16)
Also, for Whias, seep since direct caleulation yields
E Ex Knmis (Ttt, xt-1)] = E. Ex Km ( Xt-1)] = 0,
E Ex, i/c Smis Xo-1)l = -(1 - ax) Ac((2))1
have
-x, P Ex prPe- ) (mis (X Xc-1)l = (1 - (x))Ex, (x,)11
Whins, rev-step
(17)
F.4 STEP 4: BOUNDING WVANISHI REV-STEP
Next, for Wvanish, rev-step- we first note that P/-1 is conditional Gaussian. Thus, under Assumption 3 we are able to exchange the limit (from Taylor series) and the expectation due to Gaussian-like moments (cf. (Liang et al., 2024 Lemma 11)), which gives us
E Ex Px._>>P (Sra X,-1)]
1- al Tr( log (m
Σ E Bx (--
20k
Here the expected value at p zero because all odd-order centralized moments of Gaussian vanish.
Now it remains to characterize the expectation of (1e, under Pt-1je- To this end, introduce the following notation.
Definition 2 (Big-O in L space). For a random variable Zr, we say that Zx(x) Ozrro (1)
(Ex-q (x) 1/p O(1) for all p 2 1 T cc. Define Ô C)(()) likewise.
One property is that if Zz Oz)(o) (1) then Ex~q (x)) O(1). Another property is that if 7, = (0(())) (aT) and Zg2 = Ocr(q) for all 2 1, applying Cauchy-Schwartz inequality we get, for all p 2 1,
1/(2p) =((abr)).
Ex-q P/ 1Ex-QZZ10 2 x~ 220
which implies that O (=(()) (0))) (br) = O (arb)) Now, with this notation, the first lines of Assumption 3 can equivalently written as
- ar)" (1-ar)" lat log qo (Xx)) = O cx(()) - ((l)),")), Vp 2 1,
ar) 10t log tz (X.)) O Or( ((1 -a,)" Vp 2
22

---

Published as a conference paper at ICLR 2025
Also, Assumption 4 can be equivalently written
(1-ax) ax) | (Xx)ll crrt. (a)))))))") -(xra)(((1-aa"")) Ap 2
With these notations, the following lemma characterizes the expectation of Svan 24-1) under Pt-1je. which involves non-centralized Gaussian moments.
Lemma 3. As long as 9 is defined, with the definition of Suan in under Assumptions and have vl N L,
_x 1 [Snan (Tt, Xx-1))
at (V log 1 Vo (
Vau
1 at TT( (?2 log (
(1-a,)2
+ 201
D (o
20t
1
(1 (1-as) 0z)2 (1-ar)²
3
0i log (m +
Σ
0%
log 1m-1(mm)))))
3!
3/2
i=1
+
8
Ex
(or +0
c)())) ((1 - ac)
p=
Proof. See Appendix H.3
The following lemma provides the rate of decay of the difference in expectation of all Taylor polynomials with order p 2 4.
Lemma 4, As long as q is defined, under Assumptions and 4 we have, Vp 2 4,82 1,
(Ex Ex--IPP-u ) (log 1, X,-1, mx)) O (((
((1-ay)) az))))
Proof. See Appendix H.4
Thus, with the help of Lemmas and identify the dominating terms in Wvanish, rev step when 1 0g is small. The dominating term is
Wyanish, rev-step Ex,né (Ex-inRun Ex, imPe-11e) (X,, X.-1]]
1 - an E
Exing.
log (o
Vai
(1 (1- E
Bx,n
(x)
20t
1
(1 - (1-a))
+
E ExinO.
3 >
0 log q (m, (X:) (X,))
3!
a4 3/2
0gg log d (m)))
+ (1(()a)))
(18)
Therefore, with the decomposition in 15 in mind, an upper bound on the reverse-step error is achieved by summing up (16), (17) and 18
The proof of Theorem T is now complete.
23

---

Published as a conference paper at ICLR 2025
F.5 COROLLARY AND ITS PROOF
Below we state and prove corollary of Theorem [ when Øo does not exist. By Liang et al. Lemma 6), Q1 always exists, which provides with the following convergence result with ctrly-stopping
2024
Corollary 1. Suppose that Assumptions 7 too 4 are satisfied. Then, suppose that the at satisfies Definition ! at t 2 2, the distribution P1 from the discrete-time DDPM under score bias satisfies
(L(
T
Σ
1
a xx.qq. |1xe()))1
at
(V (Vlogū- (mlgūx-(m((()))) -oo
Xin@i
Cr.
T
a1)
+
E8
Tr p2 log क ( (F2 (X)) - ((XXA
20
T
a4)
d
E xx
3 >
( + Σ Oig log q. (xx))
3la,3/ 3/2
2=
V Exx-q. 14c(X.)) (log T) + (log T)
max
where Qo) 2 a,)d
Proof. The result directly follows with the same arguments in the proof of Theorem I The only difference is the guarantee under the Wasserstein distance, which can be obtained using (Liang et al.
2024 Lemma 12).
G PROOF OF THEOREM
We first recall some of the properties of the noise schedule in 8 By Lemma 6 the noise schedule
in 8 satisfies that [1-6.1)7 X
1-a,
log Tlog(1/6) for p 2 while or (T and thus such oz Lemmas 15 and 17)
5P-1T
satisfies Definition I when 2 2. Further, with the a in Liang et al. 2024
show that for any Qu with finite variance under early-stopping, Vp. 8 1.
Ex Ex,~Q. 10 log qc (X.)) =
(1 - az)
1
E Ex.nq. 18% log -t- (m (X.))
Thus, using Lemma 6 Assumption 3 is satisfied (since constant). the following, we further verify the last relationship in Assumption holds.
Therefore, since Assumptions ! and have been satisficd, we can invoke Corollary ! and get (L(Q,1| S Womele + Whiss Wynish- Now, we investigate the dimensional dependence for each term of the upper bound in Corollary 1
To start, from (Liang al. 2024 Theorem 3), for any Qo having finite variance, with the a, in g we have
d² log" (1/5) 2)
W
(log T)e..
T
Also, since by assumption E @. 1 |x, < My with the ar we have from Lemma that when << 1,
Whins S di (1 (1-
2 (1/)) log
5r
T
24

---

Published as a conference paper at ICLR 2025
Now investigate each term in Wvanish- The following lemma is useful to determine the rate of difference of the first-order Taylor polynomials.
Lemma 5, When E Xxoau |ooll & d,, with the ar in (8) we have
2 at) 2 (1 -1- 1-a 13
(1 V As a result, Assumption 3 holds.
E X. (xx)))) Va V log (X)Il
Proof. See Appendix H.5
In other words, combining Lemma 5 and Lemma 6
we have
1 |V (o Vo (X)) =
(1 ar)
E Ex.-Q.
(19)
T
Now, by Cauchy-Schwardd inequally and Lemma 5
at
Ex Ex
02
au
V
/Exmei (Ax(
Vas
1=2
IE
|I 7 (Xi)) Vo,V (X))
11/2
( )2 )2
az)
(1- xx-1)² )3
T
1og(1/5)2
(log T)2
81-r/272
log(1/8)((log)) T)²
814r/2T
To proceed for higher orders of Taylor polynomials, first note that from (Liang et al. 2024 Section G.2), the second and third derivatives of log & are
1 1 la + (1-aa)² --ac
~ log (z())
(xom@oo((la) [(x vaiXo)(I va(Xo)"
(Exo~ |o VorXul) (Exomion(te)) Jo vorXol)
03 log (z) - /
22'22 2d
/ 20 (00 / 222 =02
Σ
a1 jiijk
02 ug. .03
/ 2" [dQo)(())))))
/ / =dQo (xnla))
/ 2 (
where: :=
Thus. in order to provide an upper bound on the expected norm of the
I-ar
second-order derivative of log (m.)) we can calculate
Ex,n Exo ((mm)) m.) [(m, - vār- -o0
< &Bxpeeexoo (m1m) [[ - V0)- - V
= (11m)
||
1Xo1
25

---

Published as a conference paper at ICLR 2025
2 -(1--a-11)
and
Ex,-Q ( (Bxo-Qocis [m - vāl-iXol) (R
(x (ime - v xol) "12
=Bxndê EXon@on- (+1me) [m vāc-
-0x- (1me) lime /
(9)
3 (1-
where both (i) and (ti) follow from (Liang al. 2024 Lemma 16). Thus,
E xx.m. 2 (o x a
For third-order derivatives, similarly use (Liang et al. 2024 Lemma 16) and get (ef. (Liang et al.
2024 Section G.2))
1
E Ex
Sh log t: (mm
B(Z
(1- -
(1 1)-1)3 1332
1=1
d
E ExInQ
(0% log 9: (x.((m
1
(B(Z))9) 2/3 (B(Z))6)143 1/3
W==1
i,=1
d
(1-û,-1) - 13
Here we denote ~ NO, Ta), and note that (Z))))) ,E(7)) Therefore, by Cauchy-Schwartz inequality,
$ 1.
az)
E EXIN
log (.(
204
011)
Ex,O (X.)ll V Bx |/ log (m (X.))m(((())))))))))) 11
204
(1 0;)²
di
az) V Exia | log q. (m
20h
t=2
01)
d"
d
201:
- (1-at))
(1 ai
d +
log( (108)
17
and
(1
)
E ExInQ.
3 Σ
The log (m,
3la,
3/2
T
3(1 - 02))
BE (xiIl
E Ex,nQ. Σ
(03 log qt (m,
3/2
1=2
T
d'/2
d
S
(1 a+))
(1 -. /1/2
1)3
2=2
26

---

Published as a conference paper at ICLR 2025
d (lg) ?
8
and, with M being a matrix such that (x) = 0i log (m)))))))())),
a2)
E X
Σ
Big log go- (m)))
310,3/ 3/2
111=1
(1- 02)
E x ||Mx(X))(xxx)ll
3lay 3/2
T
04)
dd E Ex,nQ. |x)(()))
E Ex.vQ. (X,)1
3la;
(1 0+))
Vd E Exmnq |Ax(X))1
Bxna
(0% log q (m(()())))))
3l., 3/2
2
T
d((+))/2
11 a,)²
(1 a1) V (1-
1=2
(
8 8
and
d//2
E Ex-a. (xx() (log T)s
8r/2 (log T)e.
(22
Therefore, combining all the above, we get
(1-2log( log( 118) log T
KL( 2000 KL(Q:IIi)sar00
T
1 (r-1)))-))))-0 (log (logT)² T)2
+ max =
.d' + dd'
T
+ (
H AUXILIARY LEMMAS AND PROOFS IN SECTION 3
H.1 PROOF OF LEMMA II
remind readers that throughout this proof I is fixed. For brevity write m. = me(()), Me (xz)). and (x) Ac. Recall that m, xt/ Voz + (1 0t)/vay (xx). By Bayes' rule, we have
T. 1/e (Z+ (xTe)
26/11 (Ira 2 - ) (I
ãe (x+)
x q (xx-1) )00c0- (Ie [1-1)
(1)
x d: (1)-1) exp
2(1 an)
|| - -1el1 |,|11 2 Voil
x qt (It (8-1 (xx)) exp
2(1 an)/as
[t-1 - 22///a + x// calvar
/a5 14
04-1 - 52/ /ozl
- 1PP 1le (Ir- 1xx) exp
2(1 ox1) /14/0r
27

---

Published as a conference paper at ICLR 2025
(x (0t-1-It/ve xt/ (a. mi)
x Φ
exp
(1 (1- (1-wr)/ck
where (i) follows because the forward process is Markov and 0tje-1 = 9t/t-1- Here, the exponent equal to
x L2/ 22///1)) (21/ Vai- -HI)
(1-04) /ar
(x1-1 2// va.) [(m - Hi)
(01-1 10--1-I//aae Va: (1-a)/) var log (x2) (1 (11 w2)/aa
(1-04)/ur
- 01)/04
Va,Ai( 21/Var) - Var(x1-1 - vaz) log qe (21).
Thus.
( x -1-21e(001111t)) exp (x1,
where
( mz) 3 ( vai (x+11 mz) TV log o (xx).
Finally, since all partial derivatives of ūt-1 exists for any / 2 2 (See (Liang et al. 2024 Lemma 6)), the Taylor expansion of log 9--1 around m, gives the desirable result.
H.2 PROOF OF LEMMA 2
For each f I.
Pt- 1/ (14-1112111111 log
a2
(112-11- (x:))1 1|0--1 -1(0)1--((()))
P 11-1 (Zt-1111))
2(1 ar)
0t
(xt-1 - (e(xx)) [((()t)) + 2(1 - az) (1 - (x:))1
at
(1 - a1)
For the first term above,
E x.. X I [X,-1 (((X)) (x(x.))
Ex,g ((m((x) (X.) -xx(x))
at
"EE Ex,Nă. (xx (o(Xx))
Vai
04 V E 1
IX) E Ex.nQ. (X,) (X)11
Va;
Here we recall the definition of A, from where me(x) - (x(x)= 1-a1 (z(z). Thus, var
log (x
E X, 0..-1
(x-
T
Vary Boa/Ex,ue (l (x,)) Ex Ex,n
- -
+ 1- Ex, (lue(Xe)) - The (x.))))))
Cr.
1=1
T
11 a4)
1 V E X,mO, |1x((1)11
B Bx, (X,) (X,,)1
V (1 (1-a,)² a,)²
1=1
(1))
04
Bxnq [ (Xx)11
(1- 02)
log T
Ex,aq [xx()))] x v (1- Bxna. (l,)(X,) - (X.))1
az
T
log T
at
Ex Ex.0 [|xe(Xe) - Txx22
T
02
28

---

Published as a conference paper at ICLR 2025
log 7
$ max t1
V
B xx | (X,))
at
x (x,) (xx))1
T
(1 - o,)²
+
log! T
at
E
x
~ (lx) - (X:)l1
T
a+p)
)
1=1
(86)
X
max
E @ [1(r(x))1((0gg]
T E
(1 - a.) p Bx (lxe(x)) - Tee
a
121
log T
at
E Q (lx)(X)) (X+)l
an))
(18) max 121
B Bx [I,)(X))]l (Xx,))1)((0 (logT)= T) + T)
where (i) follows from Definition
(ii) follows from the fact that for any non-negative sequence as + Ei= V S / Et= ae by Jensen's incquality, and (tii) follows from Assumption The
proof is complete.
H.3 PROOF OF LEMMA 3
Recall that Pj- = N (M,, 1- Ia), and thus Ex (X-1 me(xx)) = م-ا A (zt).
you
Note that we can change the limit and the expectation under Assumption 3 Now, we can calculate that
E Ex Po-
Sran (Iz, Xx-1)l
at log Gt ( Voi log (x
Vai
20
Ex, 1 (To( Xr.,
2=
Below we write m, = (1)). Since T/ is in quadratic form, the expected value under Po-1je for this term is
E x P Ex--
(2og q 11,,
at Tr log (m+)) + (1-
(z( (m))))))))
20n
2a
Recall the formula for Gaussian non-centralized third moment. If _N(u, then E[2 4² + 0² and E[Zz| = 1² + 3 Thus, the expected value under P,- for T3 is
E Ex, 11t (log q X,-1, m.)]
0 log q Bx. in (Xi_1 m()²
3!
1
0 log dr- (m. an (X m m!)
31
Mi
1
+
log ( mmx mt)
3!
(j,k=1
i.j. all differ
d
((-1-102)
3 -3(-1-au at 4:) ( (1-aa)
att ))
3 log qi-1 (m))
Vai 4:)
+3
31
vai
ai
29

---

Published as a conference paper at ICLR 2025
1
si) ((-t7m01)
are
+ (
at ) ( a
+
3!
0i log qt (m))
Var
1J=1
1
4;) (-1
1 at Ai) -
at
(-1- d
+
Li log qm-1(
(-a
3!
Vas
/a
Vai
6.j.k differ
1
{ (1 ax)
3
d
0% log q1 (mm)
+
Σ
d
By log 91- (
3!
3/2
1.7
+ + (a( ((1 az) ve 2 1.
Here the last line follows because (1 - ar) 1095 log log = O2(())) ((1 - aa)))) under
Assumption and (1 - ||Ar| = 0 c+( ((1 - -ax)²) under Assumption both for all 2 The proof is now complete.
H.4 PROOF OF LEMMA Fix 24 Rdd For brevity write my = mx(x1), 1t = (xx)), and A, = Ax(x)). Recall that
d
(log TI-,, m.)
1
0% log qr 1 (m+)
II (o m))²
p! TEN
where [d] the indices of differentiation in which the multiplicity of is 7.. First, for the expectation under Pr-1je (i.e., Gaussian centralized moments).
d
E P
[x - m²)r²
E P.
[xi__ (xi=1-m;"**
x
71/2
(7'-1)!11 (7* is even)
Ot
p/2 d
at
II
(n - 1)!1 is even),
Ot
where we use the convention that (-1)!! = 1. Next, for the expectation under Pt-1je (i.e., Gaussian non-centralized moments),
Ex PX P1
II 1 m2)²
(x 11/ 1
1- ar
Aijs
Vaa
(- at
x'-e
(1-00))
att
e/2
(E - 1)!!
Eeven
To investigate their difference,, we divide into the following few cases. Note that under Assumption 4 (1 ax) [ = O ( (0:--a;))) "²) for any m 1/2 and f 2 1.
1. Case 1: is even and all elements of r are even. Then,
Ex
II [x:
mt)
30

---

Published as a conference paper at ICLR 2025
d
(1-at) at
7//2
- O ( (11- a1=
1=1
p/2
II I(
1)!! - O cc((2))) (1- p/221
Qt
2. Case p is even and 3i" such that 7d is odd. Since Ex p. there exists j such that 7 also odd. Then,
(x m; )²²
O cr(())) (11 -1- -a--a)) az) ((r" +1//2)
E Ex-IIPPLaIe
Ex, 1~ ip
(X - -m²² )m (c((()) (11 (11-ax)('' +1)/2))
which implies that
[xx
mby² -
=c(1ss (11- (1-ac)p/22t)
P
3. Case 3: p is odd and 3i* such that 7" is odd. Then,
Ex aa (x 18--mm m yr] =((()))) (11 (1--a)) ac) (11/2)
which implies that
d [[ -mi)²
0 (c((án) (11- az) (1)(0+))/2)
Ex, P-Pe-
Combining these cases, we get
[xx - mm)
(Ex.InPPnn - Ex. P
(Deiss ((1 (01-aa))))) 8+1 (cnes (11 0+) +
vp > 4 even
Vp 2 odd
The proof is complete by noting that the rate does not change when we take the expectation over Q under Assumptions 3
and
H.5
PROOF OF LEMMA 5
Note that ())(())))) (x/0)))))) is the p.d.f. of ((Vārzo (1 - Ta) Thus, the gradient of log (x) equals
log (x(x)= = SnpERd (2
1 āe / (x - (z) (20) MOEER
(x)
Thus,
V log Tr- (mi) - Vau, log (xx))
(m+ (101))))))))))))0m11)) + Vai
(x-- V
ōi-1 JzgERd
Ee ZOERR
1
(1-aa
āz - 1 /
(m; vo1-120dd@01-
āi
+
(m4 )dl0 Vai /
(Ir ((uro)dQuu
SgERd
SOERd
31

---

Published as a conference paper at ICLR 2025
(1 āi -1 11-0--1)) 4t (og
(m+ - 100 - voi /
(ot - Varzo )d
- αt
IDERU
IDER
Ūi 11 ac)
(m, - vaizs) + v 0 1( -cre)
/ JIDERS
log 1og9:-1(m()
- αt
ae
αe
term
term
temm
Va a1-1
(21)
at
ZOERR
term 4
where (i) follows from Tweedie's formula. Among the four terms in (2I). the first term satisfies that
S (1 āt) ) āx-1) 04 1 at) i-- (
d(1 a,)²
E x
Ōt
by (Liang et al. 2024 Lemma 17). In the second term in 2T). by Tweedie's formula,
my Varaa
V log q (2:4 varzi
Vai
Va
1 ar
(Iz + log (x:)).
Vot
Thus, by (Liang et al. 2024
Lemma 15) and Assumption
I the second term satisfies that
S d(1 d(1-a,)²
E x
(me
I- Ōe
The third term in 21 satisfies that
0t ac)
d(1 - dr)=
E Ex
2odQo)((00(11)
x (1 x,)
ZoERa
by Jensen's inequality and Assumption To deal with the last term in 21 note that
I
g (1-11(()))zo)
T-- ao-1p((pgoo))
(
dQu(zo)=
dQo(vo);
(m=)
SyeR= qt- (--po(mml)ddQQ00
( d (xx)
(no(=z(=0)
(Z)()
JNERS ((o(=xl))dQ((())
Thus, the last term in 21 is equal to
Val-1
( 1 mm)) - . SudQo)r(2oozz)
1 1- ā.
V
1- āi (m=)(m))(((x)) ( ( =:00 qaddal//Wie-noomm/b/l00000dddddd
where
-2)
= (m (0= -1)
(2)0
||,, - vār- 2
|x, / /
mm vo.-1311
||x, v 5,
-1 1
exp
2(1 (4-1)
2(1 (x))
2(1 ā.-1)
2(1 (i.)
32

---

Published as a conference paper at ICLR 2025
=: =0 - 1) in which we have defined the exponent as A. Now,
m, v 14-14 20 - v Viyll
mm v va-11211
| - v v
2(1 (1-1)
2(1 (4))
2(1 x1-1)
2(1 (t)
v 1a--1(2 1 y) Tm + 0--1 ||y|1 δi-1 ||0||
Vāl(1 (x
-y) + ãr. |1
0z. ||
2(1 Ō1-1)
2(1 ac)
1 Vār(1 a1)
2t + vai-1(1 ar)
log log (((x)) (x y)
2 a4((1 - 01-1)(1 a+)
Vai (
0111 04)
(1lvl |
(1 - a-11) - āt)
Now, with the Or defined in following from Lemma 6
vā ac)
1 at
log I T
=0
0
04 δ1-1)
(1
2)-1)²
1 -
log'T
04-1
T
0g 11-1(11 (1))
1- ai
0 log T
0
(1 0; a+.)
(1
-a-1)
Thus, for fixed x, y, 24, + as T + co, and thus when T becomes large,
1- 0(A3) $ |A Vo, Rd.
Also, since (2)((=)/ and - (m.ll) decay exponentially in terms of and y (for any fixed xt), we have
/ Te @no(=rl)))pol(()dd(( < cc,
[ (m (oly < cc.
Thus, the limit and the integral can be exchanged due to Dominated Convergence Theorem. Thus, the fourth term in (21) gives us
Var-1
(f JZOER
20d
Var-1
Uc-11 (m=) ( (0m ((Q()))(QOQ))
1
04-1
Ve (a (A))) (ro1AA)daootttttrtddd00---0111
and, from definition of A and using Cauchy-Schwartz and Jensen's inequality, we have
V Va-1 {
E Ex,-q
(x (00
1 5i 2.9ER'
(1 (1-c))
dr 22 _ar)
[Ivaxx] (1IXx11++ + (1 - āz)² log (X))))
E
Xi~Qi
x-Qo)(-1xx))
(1| || + +lvār-n + +1|Vaxx11] + [[va)))11)] ([[vaxx)-=+Vvvaax11)))
(1 - (1
1
(1
33

---

Published as a conference paper at ICLR 2025
||vaxx|| (11x,1² (1-11000) āx)² || log (X.))
E
XimQi
Y YaQai (1-mmm1mmm
+ ||VaxX1
Vva- (1xxel + +(1 -1--a)) 2 || log कe (x))
||
|| 1I
(1 (1-ay)2
(1 - 1 - -,)4
E (xxmá 1Voxx 2/3
(xxree (1x +11 āx)% 6 | log 2: (x.))1 1/3
1/3
(Bxnáolle |Vaax 1/3
E
Xem@e
1l 11
Y~Quie (x)))((x))
9 || log 2t (x)))1))) 1/3
[ + (1==)=)
2/3
1/3
+ Ex yx Varx +
E
Xir v~Qof (x.))
1V
(1)
(1-
(1- (1-)a-1)(11-00 a.)
-
where (it) follows because, following (Liang al. 2024 Lemmas 15-17) and by the lemma assumption that Ex | S dd, we have
Ex~a0 || S dd²;
E Ex,~Q, |X < E Ex
Voo +(1
||| S d=,
d3
Ex | log (X,)l1 x (1-
1110
Xi~@i
(1m
E
X (1
|| Vā + +Ex, Ex,ng lmmlll dd d.
Hence, combining the rates of all parts, obtain that
13/2 -a1) (1 11-ash cacl (logm- Vog < (1-0,-1)3
H.6 LEMMA AND ITS PROOF
Lemma 6. The ai defined in ( (with 1) satisfy
1- @
log (1/8))
while ãr = o(T 22SIST,p21
8T-17
Proof. The proof is similar to that of al. 2024c First, note that if T is large,
Eq (39)), We first prove the second relationship.
clog I'
ing 7
1
x 6e 1.
T
Thus, with any fixed E (0, 1) such that t rl' (>
Jog
have
clog T
clog T')
c log T
1 - ar
min
8
1
1
T
T
T
34

---

Published as a conference paper at ICLR 2025
As result,
(1 clog T
(1-r)21
eexp (ra--))) (-closT) -1
II [=[rT]
T
T
(22)
Given any > 1, we can always find some r such that (1 - r)c > 1 (say, r r= (c-11/2ife (1,2) and r = 1/4 i 2 2). This shows that on satisfies ör (T-1 if 1.
Now, for the first relatinoship, define such that
8 (1+ log S
T T)
log T T
< 14
(23)
Here 7 unique since 1 - Org is non-decreasing. In other words, T is the last time that 1 - 0z is exponentially growing. Assume that T is large enough such that 2 2. Below, we show that
clogT T
1 1 1- 2 36
1+
42 SEST
(24)
I 2,
1 0 =
( 1+
clog T
1- 1-1-11= ā1 a1
3
T
Here the last inequality holds when T is sufficiently large. For t > 2, suppose for purpose of contradiction that there exists 2 < to I T such that
6 11 3
loog T
1 1 1
while 1 - ā.
0 1 T
v2 5 to 1
T
3
In words, to defined as the first time that (4 is violated. To arrive at contradiction, we first write
- = (1 (1-āso-2) 22) (1+
[+
ato-2(1 Pto-2(1-ata- (ita-1))
1 1
1 5 1
clog T 4a-1
1+ Ckto atto-(11 ato-
3
T
Here the inequality holds because to is the first time that t to Also,
24 is violated, and thus (24 stills holds for
(
0 ( 1 3
c clog 7
to (6) 1 5 (1
c clog
(ar)
1 ā - 5 51-60 ōt
T
2
T
2
where (i) holds because
[4 is violated at /. = to, (ii) holds when T is sufficiently large, and (iii) holds because to and by the definition of r in (23). Thus,
clog t0-1
clog T
Otg 2(1 Otto
T
clog T T
10-1
18 1 +
clog
2
1
and thus
Cito 30 (1 11 clog T
10-1
(1
Cigy Qtg- 20n-2(11 at
1 0
clog T T
to
1+
ato-2
We have reached contradiction. Therefore, we have shown that Now,
24 holds.
24 implics that
04-1 2 1 (1
(1+ 1+
c clog T) t 1 8
(1 1
clog T
1/p clogr)
2
AA SEST.
T
3
T
There cases:
35

---

Published as a conference paper at ICLR 2025
ff2 then
clog 5
clog
1 a2
3'c T 8p-1T
(1 a1-1)
1 dP
[ + clog
I t > т, then
c clog
e log T
(1 + clog
1 at
T
(1 0t-1)p )
(1 - -
(1 + 50
<log
(1 (1+ clug 8 r+1
37
7
3-88p-1
3 3elog T 8 IT
1 clog T
In both cases, if T is large enough, noting that 2 loolll1//000 (8), we have
1 1 Ct
4oclog T
log T [o /8)
(1 - 11-))_1) S
8P-17 2
V
5w-1T
because p and are constants (that do not depend on T, d, and 8). The proof is now complete.
H.7 LEMMA AND ITS PROOF
Lemma 7. With the Or defined in [). given any p > 0, óp < 1,
T (1-ai & (1 pp + (1 8)
pó lop(1/15 1/8)
(
log T
T ))
1 1+ 0
1 1p
Further, when å < 1.
T
a - (1 (1+
p+1
clog? T
+
Ō
2p
T
7
Proof. Define the sum ST. Recall that
clog T
(1(1 clog T
a = 6, 1 a-
T min
1
V2 StST.
T
We first note relationship that for fixed 5 + 0 and
> 0. As + 00.
(1 82 1== pz log(1
8z-1) 1---006=--+-- 2/= 9)) 60 + 82 12)
/2) + 0)=
(25)
We also use the fact from binomial series that
p(p 1)
- 2-100 1 p2
-2 + 03
(26)
2
Define t sup { t [1,T]: : (1+ clog
< 1
Thus, 04 1 clog T for all t > t². Note
that when T becomes large, 1" = A
To further understand the big-O term, note that using
log
25
(log(1/4) clog
log(1/8)
5 clog
T
(1 1
log(1/5)clogT
1) (1
log(1/0)clog
+ +Ō (0(1) (1)) ) T2
+ δ
2T
T2
T
log(1/8)clog
+ Ô
1
2T
T
36

---

Published as a conference paper at ICLR 2025
<1 as T 00.
This implies that
clogT -1) (1).
T
2 log(1
(27)
To start, we suppose T > 1s is large enough and decompose the sum as
8T
(1
a4 a +
Σ (1
az)ag
1=2
+1
P
clogT T
clog T
clogT
clog T
6(1 - 8)P
+
8
1+
T
T
T
T
1=2
1=
clog? T
clog T
p(e-t")
+ di.
(28)
T
T
1=1+11
Now we first focus on the second term in (28).
clogT
T
T
clog T s(t-(")
1-
T
T
+1
1 (1 - clug T' ((T-e")
(1
clog? 7 T
clog T T
clog T P
1
p(2'-1")
1- (1 clog
c clog? T)
clog? T
1
T
polog T
+ +p(p - 1)
cloog
+ +0()))
T
27
(8)) 1
polog T
(p - 1)clogT T
1
1
1+
+
- 0
T
T2
2T
T
Tpe/2
1 (p + 1)clog
2T
T2
where (i) follows from 26) and (ii) is because to' = ((T log T) and thus T' - > T/2 for large T. Also, for all t = 2,
- 8 clog T
clogg] 7 ))
of = (1 8)P
II
1
T
T
im2
1 5:
scloog
log T
1(t-1)
$ (1 8)P
T
Thus, we have
-1)
1 clog T
aG. s (1 8)P
T
c clog T
1
clog T T
pt
= (1 - (1-
1 + 6p
+
T
T2
(81)
clog T
x (1 - 8)8 (1+ ōp(1 log(1/)))
x
T
c clog T
1
c pólog((1/))
1 1 11
+
27
T
(1 -88e po 2- log((1))) (1+ 5p log(1/0) + (6/2)
(1/5)) T
clog? 7
+0
1
T
37

---

Published as a conference paper at ICLR 2025
Here (tit) follows because using 27
and 25
have that
1 0 log T
pr
clog T
P plon(1/60) plog(1/0((
1
T
clog T
(1 pllog(11))) clogT)
1 + 5p (olog(1/0)
po 1oR(1/6)
+0
(29)
T
2T
T
Thus, the second term in satisfies that
Σ (1 )a
1=
1 S (1 5) e pō log(1/8)
(1- [-+1- (1-(2=1 8p (1 log(1/6) + (6/2) log())/)111))))) 2
clog T
p
T
+ δ
(30)
Now we turn to the first term in (28).
in which the summation can be upper-bounded as
clog
II
8
clog T
[1 clog 7
T
T
T
t=2
loog T
(1-
clog T
p(t-1)
=(1
clog I
+
=:
T
2=
where
clog T
clog T
1 +
0
T
T
c clog? T
8p c olog T
flog T)2
1
-5pppp 1)
+0
T
T
2T
T
clog T
8 p'p(p-1)
(log
T)2
= =1 +++00 (1 8p)
+
8p)
+
T
2
T2
73
Note that by assumption 8p < 1. Also, by definition of and
29
óq" < 1 0 clogT pt
T
< e p6boo((11
(1+bplog(1/0))) 0/2)
(1+
clog T
+0
T
72
Thus, we have
clog
T
6'-1
clog T
pa -ōg
5
T x 9-1
T
e p6 Log((1/)) (1 (1+ 8p log(1/0)( (1 8/2) e log
Ô ( ōg
clog T
T
9-1
eopolog(1/0)
( 8p log (10)(1)) 8/2))
clog
+ (112)
(1 5p)
clogT
+ +
(Iv) clog
x
T
(1 (1- (p) clog +
8ppp-11
6p)
c #logg 7 + +0
pô (og(1/6)
-1
(oplog(L/0)(1)) + 8/2)
clog T
+
-
8p
1-5p Sp
82p(1-1) 1)
8p
clog
+
+
1 - 8p
T
T
38

---

Published as a conference paper at ICLR 2025
e e-d (- -
8p (pplog(//)(( 6/2)
e po loR(1/8) -1
6p(1 8(p - 1)/2)
c clog
+
1- 6p
1- óp
1 d-óp
1 6p
T
+ 0
T2
where (dv) follows from 26). Therefore,
Σ (1 an) = (1 8)
(1+ clogT)
( (cclog 11
Σ
T
T
1=2
pd log(1/)) -
$ 0)p
8p
(pplog(1/00) (10)(1+
+ - 8y
-p
log(1/6) -1
6p(1 8(p 1)/2) ))
clog? T
+0 T2 +1(
(31)
8p
- 6p
Combining (30gc and 31 we have that
S ( - 6) PPee-ee (og(1 + (1 - 8)0 e pá po1oR(1/8)
log 7 T
1 1+0
1 d-
Also, for all large T"s, since 800 + and log(116) + 0 as 8 + 0, when < 1,
8T 5
1
1
1
clog? T
+ Ô
p
2p
T
72
The proof is now complete.
I PROOFS IN SECTION 4
1.1 PROOF OF THEOREM 3
Fix & 1. Using the forward model in (). have that grjo.g the p.d.f. of Nf HH'
Va, H.HH.., y, E. Thus,
V log (x)) (2)(2) I ZRRRR
1
vārlla Ht HH=o yod (2) Egjo.n (=)
ZOERa
1ooyy (x (i.Hiy)
+ vāl x-1 STp (La - HH) (2)
/ JnoER qejo, (xlo)()/aa)ddd000pp
Thus, the equality for V log Øyyy established because by Lemma
(o? + (1 - dt) H*#) = (1 (1- (1a H))
To see the optimality with fī,: fix t 2 1 and x Rdd. First note that (Ia - HHH
=
(La HHHHE (VāgHty y Ht Hz) 0 by Lemma Now, suppose that fu.y = fi, + such
that (Ia H H)f = (Ia H H y
0. From the definition of Ar,y in 7
(x) (Ia- HH H) log 4eju (x: V (x) (H) H) log (x) fe (x)
where
(H+H)V
(ogVVlogeoo log (2)))))))))))))))))))))))))))( (2)
39

---

Published as a conference paper at ICLR 2025
(H' (H' H) ( x) - y -
-(H' H) 1 (Ta H H)) (Ta H Hi H Hi
where the last line follows from Lemma 9
Thus, e 0, then fay fiyy and we have
D.y (Ia H HV log (1)(() V log (z)))
(32)
Also, v + 0, since v is orthogonal to the space induced by (La H H), we have
| (+))l² = (1a - HV log (1)((x) V log (o))) + ||0||
(33)
which is minimized at 0. The proof is now complete.
1.2 PROOF OF THEOREM 4
Fix t 2 2. Recall that the unconditional score V (log((()) is
log (x() = (((x)) /
mg ERd
1
(1)(<=/20) - vārzo)dQo(zo)
(1 at)g((x)
AOERR
Vār
/
(1 ā+) + (1- (z)
JIDERS (10(=1)))2odQ(((m)
since yoo is the p.d.f. of M(Varto (1 aa)la).
In the first half, we consider the case where of is known, and thus JU,y St. in 10 Note that from Theorem
V log = Enjo.n (VāiH x)
aa
/ JADER (1l.y(///
41y u.y (la - H'H)
Here we also recall from Theorem 3 that
E,o,, == ā.o ( + ad)
Since (HH)T is positive semi-definite, all its eigenvalues are non-negative. Write the eigen- decomposition (HI)T
Da) where D1 2 ... > Da 2 0, vi E [d]. Then,
Amin(2ejo.y)) 2 Dio" + 1--- āi 2 - āt, and we get
|
S 1 πi
(34)
Also, from ( with the fi. in 10
(x, (x) (Ia H log (xx
110 (VorHty Ht Hx)
(1- (Ia H H'H
Var
(Ia H H'H) /
(1 (x)
ZOERR
(
(
V -H -H'H))= - E $1,0y H,)
10.yy
vat
110, (1a 11HH) / JRDERS (10((120)22dQ(((z0)
(1(x)
= =Eioy (Vāht y - z) + vāl
talo (Ia - H'H) /
(Ia-
(x)))
40

---

Published as a conference paper at ICLR 2025
where (1) follows from Lemma
Then, the norm-square of the score mismatch at time t 2 2 is
A I|
V 1o
"llll || H log 4:ly v log 9)ll
|2
SMDERS (1000
I JZgERe 4s (z0(
$ 0z
1 10,
(2)
(c(xx)
(18)
2
01
Σ
|0aa rbal dQoje (2r (1a)dQ(
ERd
Σ 1
||20 =
Ze Esupp(Qolp xy Equpp((Q)
(wi)
d. )x)²
(35)
Here (it) follows from Jensen's 's inequality, and (tit) follows by (34) and from the assumption that Qo has bounded support (and thus also for both Qoje and Qojt.g). Therefore, with the a in (8) (cf. Lemma 6), since 1 - 0₁ 2 - 8 which is a constant, Assumption is satisfied for all of 2 0. Thus, Theorem holds with
and r = 2.
Now, we consider the case where o is unknown, and the conditional sampler of interest is g (z) =
fi (z) + (Ia Ht H)V log (x) where f (z) = (1 āz) HVā H Hz). With the same notation in the proof of Theorem 3 we write fiy. fū,, ((1 ) la
2.dyy (Va,H Ht H Hx). Note that v still satisfies that (Ia - H HYv Using the result in
33
we have
I1(Ta - (V log (x)()) - V log (=)))1 I
+ 1 (1 (1-aa) (1-aa))11 -a) (Vaillyy H² Hx)
where the first term the same as in (35) which can be upper-bounded in similar way. To upper-bound the second term, note that by Woodbury matrix identity.
1 Is E0N 1 Cs
aioy (1
Ip + 1 1 ā Hi
oy
(HI)
(HI)T
)2
(36)
(1 - ōz)²
where the inequality follows because ||ii|| S is constant and the minimum eigenvalue of o
(Ip + a-a.
(H+) (tt) is at least 1. Thus,
(Eid.y - (1 -11) (VoiHl H' HX.) |
EQu
(t0)
0.00
EQu I|Va,Hly - HXII HX.]
(1 a²)
0700
EQuiy E2jo. |1 Vaaltly H² HX,
(1 )z)1
Eeuiv
BQq VorHty - -
11 az)²
20 4
< (1 )²
EQuiv
@ Hly y H H l + E | H(x vaxxo)l
20/o1
vl 2 + a |ool + d( - az)
<
EQaiv
(1 a,)²
41

---

Published as a conference paper at ICLR 2025
0 0
(1- āz) d
where (iv) follows from (36). and (v) follows from the fact that 2oly has bounded support. Similarly, for general moments 2 2.
2/2
a, - (1 at)) Iaa) (Vantly - Hx)) 2 (1 xn) (2n0.v
Therefore, with the a; in since a_ which is constant, we still have that Assumption 4 is satisfied (see Lemma and Theorem 2 still holds with = 1 and = 4. The proof is now complete.
1.3 THEOREM AND ITS PROOF
Before enter the proof of Proposition I and Theorem 5 we first state similar set of results for Gaussian Qo. which turns out to be useful for analyzing Gaussian mixture Qo's. 's To begin, the following lemma investigates @x--Q (X,)) when Qo is Gaussian. This quantity is proportional the asymptotic bias Whins
Proposition 2. For Qo NHo: En). if ft.y fi,,, in (10) and H (Lp 0), with the o 's Assumption fis satisfied, and
according Definition 7
max Hy m - H H I| + d(A1 + o.), d}
Ex,~Q0y (X))l < a
||Loolue [iol
(da, 1)2 min (da-p;
ot C[ H Holl" + d)
where x is the largest eigenvalue of En,, and da and da-p are the smallest eigenvalues of Eg and [Eogg.. respectively.
Proof. See Appendix
With this lemma, the following theorem characterizes the conditional KL divergence when Qo is Gaussian.
Theorem 6. Suppose that o² 0. Suppose that Assumptions and 5 hold. Under the same
conditions as in Proposition if or further satisfies E (1 a a = + o(1, we have
KL(Qot (PPp)) S Hiy H Huol
122 +d)
(log T)²
+ +( - H H d)
V | H HiHppll d (logT)e
T
Note that similar result can be obtained for (L (Plp)) (where W2(Q 201y) 2 z 8d) for any general of 2 O using the a, in ( (see Remark
1.3.1 PROOF OF THEOREM
Throughout the proof we use the same notations in (55) Since Assumption is satisfied from
Proposition 2 order to invoke Theorem [ we still need to check Assumption 3 Since each
Quyy (Vi 2 0) is Gaussian, all partial derivatives of its log-p.d.f. higher than third-order equal
zero. For the first and second-order, note that Egly G.a Ht H) H H (1
La + ā HHHH H. Thus, when of > O, Amin (E--1ly) 2 min(1 - ãe + G.o 1 - ā. + azAa) >
min oz da} 0, which yields
M 1-1 1y
S1, 1,
(37)
Thus, we have, AA 2 1,
EQu
V (lo)
42

---

Published as a conference paper at ICLR 2025
-Qu | (X+ memill - ||<
EQei | (X2 mmw1 -x
$ d = 01),,
EQu
V log at 1/0 ( (x+))]l
= -e (m (x.) - 14 1|
< | ||
(X, Erui - (X, -m + +||=AT
Een
V
V log 9:lo (1) (X,)
1 =(1),
V2 log y (x,)) = -|| 011),
EQut
2 log 4t 1/y (m (x - || O(1)
EQu
Thus, Assumption
holds when - ag satisfies Definition !
Now, we can invoke Theorem and get (L Poyy)) S Woracle Weins + Wvanish. where
T
Woracle
Ex, Ex
( log (x)( + (log 7)
201
T
Whiss
11 ax,
(X,)11
t=1
Wvanish
at E
Exi
(Vlgga-11)(aa,(())) Vog 9t Ax
Vai
201
(1
a²)²
3l0x3/ 3/2
Ex, Rely
3
03 log Ui 111 (m (m, (X,)
tss
1=1
+
Σ
Ogf
log at 11-1111 (m),,))) (X.)
y==1
+ max
Ex (x,) T)e.
121
We first consider the estimation error (in both Waracle and Wvanish), which can be upper-bounded as
Exi-Quy | (X,) (log ()o T)= + (log T (| [| H HHol' + a) (log T)
max
r21
from Proposition
2
Also, when Qily is Gaussian, we can calculate, for any oe E IRdd,
2 qt (zx) log 1gg Tr(E,11 )Tyy
= Th (a 1/0 + - az) (1)
1 1- at x.
Tr Tr 111 (aj و
+ +0((1 - -a)))))
at 2111/y
0:
where (1) follows from Taylor expansion when 04 is small. Using (37). this implies that
d(log T
Woracte S
(log T
T
43

---

Published as a conference paper at ICLR 2025
Also, from the condition dt..
T (1 oz Qx, a (x.)ll s (y H H 12 + d).
Now we focus on Wyanish (except the estimation error). Since Qtly is Gaussian, all third-order partial derivatives zero, and only the first two terms in Wanish remain. In the following we fix t 2 1. Also recall from (56) that when H- = (Ip 0).
l. -ā HHH)) (Ia H (x+ vāruo) - [Zolgg + (1 - dd-p) Eolyn 25 In - v(ruo))
For the first term of Wyanish, we first calculate for each 24 that
v log 9-- 11 (me.g) va V log (1z(
vanz (1 (0e vāellolu) _ 112 (mey vao-1140l)
fly
Recall that
1
Et + 1- at V log (11(((()))) (log
1-a0
1
1- at
2/1y (xx vārloll);
myy Vax
Vat
Vot
Vat
E-1 Ecoy + (11 - +1-aa))aa) = A1p
-
1
1- al
5-2
+0((1 - 33)
Oz
1-1
at
2-11y
Thus,
V log (t (110) (m),)) - /0, V log (())((zt)
= Vai
--1
al -
(Ex - vārHoro)
1-118
1-11y
at
D-1
1|1
Vat
-x +
1
1 1-a at
Vai
v
log (1)(())) - var-inul) + +0((1- - =at)2)
ai 2-
(xi - varpou) 1- au 1--0s,11 -
Voz 111 Di (vre vārlool) + O(1 (11)),)))
3/2 111
01
Combining with the definition for Dry in (66) and using Lemma 9 we have
Ex [A. c (x,) (VV log tt 9--1 ( (X,)) - Vo, V log 9: (x,))
=xi (Xx - Vaijo)) TE,
HH 1, (Ia 11
Ht H)
5-
1 1- ター1 1-11y -11
(X. voT-oo/00)) +0((1- - oz))
3/2 1-11 01
Vat
E N,Exx~Q111
(X, -V-apu))) 1 ( H) (Ix - H H) (Id H)
1 1 ar 5-1
1- -ー v (Ia- -HHHH)
3/2 111 (Ia - H H' 1-11y
a,
Vai
(Ia - H (Xx Varua) +0((1 22
(X, V 1,2 EE0] uy (āx Eolss (1 -a
1 -
1- at p2 1os 1 10 (0 Ta-p) (Xe vāoHo) Vau
[- -10-000
3/2
a,
+ +0(11 -ax)²)
44

---

Published as a conference paper at ICLR 2025
(1-9)
11 at
T;TT 11 [ooobs(( (2oly (1 -at)lapp)
12
at
1 الyا
3/2
11 1
Vac
O4
(0 Id-p) Ex [(x: - Vorno)(X, -vas(po)
+ +0((1 -a,)
||| || (ā, Enlog + (1 -ā, p.) - x
| 1 1-at at | -
|| 11)
3/2
Vot
Tr (Ex,~0rl
[(Xe vāsuo)
(Xe -(al
(10)
-
at
1-ac
max (1e1'y 11 H
d(A1 og)
3/2
Vale
0f
S az) (IlHtyHHHHa + +d)
where (it) follows by Lemma 9 and from definition that (Ia H H Moly (Ia Ht H HO
(iii) follows because (Tr(UV) S |U]| Tv(V) if V is positive semi-definite, and (iv) follows from (58) and the same reasons for (57 In particular, we note that 2 Pi
(oolg + (1 T and ( (ools + (1 Ta) S min(As r. < 00.
For the second term of Waanish. we use the fact that (2.11/y11 = - + (1 - -i-1)) and have
- De
= - (X,
H - H H
(Ia - (1)
Vonuu))
(ā,- To;rr [Σ,') I (z Eolas + (1 la - 1 + (1 - ā2 1appp
(Gr(Eolsa + (1 - la-) (Eol E2, 11: 1x Qu (X, - Vinmo) v(rsHoo))
(e)
max { {1 11 11 woll 1| + d(>i og), d (2olos (1 -1)
[(āt-1|Eolã) + (1 - ld-p) 11 | 112 |
X max
|| y Ht H p 2 + d(>i +oz).
H Hi Ho 2 d.
Here (v) follows from the fact that ['Ir(UV)] < (T]V) if V is positive semi-definite and (88) The proof is complete by plugging all the results above into Theorem [
Remark 1. Before we end the proof, we leave a note for the case of o = 0 (indeed, for any general
of > 0). The only difference is how to upper-bound Womele. In particular, if of 0, (37) no longer holds (i.e., we can longer upper-bound
constant). Instead, can obtain upper
bound
| I (1 (1-aa-a -1 Then, with the a₁ in 88. we have
d(log 2)² log(1/8)2 + (log T)
Woraele
T
45

---

Published as a conference paper at ICLR 2025
The rest of the proof still follows because the a, satisfies Definition when t > 2. Combining with we would finally obtain
Lemma 7
(L(Q11111PP S H| H Hool l +d) (1 11 3 (og( T
T
(log [)g(1166)
+ +(IlH'y H H 2 + d)
+ Vlllly H H + d- (log T).
T
Here W2(Q1 Quy) x öd.
1.4 PROOF OF PROPOSITION
We first introduce some useful notations for this subsection. Recall that Qo has mixture p.d.f. in which the mixture prior Ta is independent of (= Hro + n2). Thus, using the fact that zo = (Ia Hi H) + H n, we can define Q0 as (cf. Fläm 2013)
Qoly
nue Qu,ty
Σ
Th H Po. H' y. (La HLH)) H² of
Note that when H = (4p 0) and oÿ > 0, 401y exists. From the conditional forward model in (5). further define
N Qi
N Qrnngy Qen == N(e Ee), Qt,nly == (H4, Edly)
Σ n=1
Tu Qu.
Qlly Σ
m=1
E1 = C la) Memuly = van H H) + varlty
Eny == Er,sig + a Σg,eig āi (Ia H H)olala H1H) + (1 āc) (38)
Similar to 37). we still have
|
S1, 1
We can also calculate the scores of Q₁ and Qnly in as follows.
N
1
V log (x())))))==))))))
Un (xx))ET( (xx - Mr,n),
9r (xre)
1
V log (1)(())))
Tu nt (z: (x:))2 Me,
(39)
n=1
Now, with St.y Jū (in 10 from the expression of A., in 32 under the assumption H (Lp 0), the score mismatch at each diffusion step is equal to
Dry = (Ia - H H) log Aels
V log 9:)
1
n th (2x (Ta Hi H)E, (xz - Vosso.,)
qr (x+)
n=l
(3rz) (2o H1 HT) Vorpo.nlo)
Unlt
Otly (xt)
N
(xx)
(Ia H H' vonso..))
nn
Ot (xte)
aea u((re)
(La - H'H)
(x, (xr vorua.)) Eij( vorpo.nly)
(1))
46

---

Published as a conference paper at ICLR 2025
N
(mmlu(ae)) (21)
=-vag 02
Tn
(Ta Hi H) P0.2
(24)))
n=1
9r,0 y (Tr)) (2t)
(Ta -HHHH (x vorMo.n) -211y v(rloonl))
((ml)(21))
HH- n0,a
Tn
(e((Te)
(2)((
It,n y (2))
Un
A A, H) (0z - varpo.))
(40)
where A, (Ia H Hi (Ia Hi Here (i) follows from similar arguments as in (6)
Note that since H = (Lp 0), we have equivalently At = (T H H) (La - H H). Since
H' ( ))
we can also re-express the second term in Dr,y such that [Ar, 0 and
N
((,n(Tr) (o(e))
((mll((t)) (1)(1e)
α
Σ
[ Mom
n=1
(ar (āl(Eols @.)Td-p) " [2, 1u: Vārpo,n)
nn
(41)
1p
A: A:=
since when HtH
0
(aa +
Eoo
(1- -a)
Now, for the second moment, we follow similar analyses in 57 and get
Ex.~Quv ||
I 44
max ||- mo.m|l
nEIN
+ 2
(av (Soloo (1 - Tap) ||=,|||] 1y:
|| Soluvll 11
N
Mt.r v (X1) (X,)
Ex
Σ
Tn
||x, - Vasuo..
where
Ex Σ Tn (X,) (X.)
|| - -Voppon 2
X, Vārmu,N |12 2 2
-ExiQQ11 ENNILIN
EN-IL Ex Ex ||X, Vārmo.N
(4) EN Tr(Celp) + ār || y Hr'HPoNN 1²]
N
Tr(Engp) +
nn | Ht H |
1=1
where (ii) follows from 588) and note that Qt.N is Gaussian for each N n. Denote \ 2
Aa > 0 and 2j 2 ... 2 Xa 0 to be the eigenvalues of En and [E] gg. respectively. Similarly
as the proof of Proposition
we have |[Eo]ay|| < | = Å1. [[ 1y $|1 S (ā,da +
(1 - (1-z)))- < min/A....
[a, [Fols gg + (1 a.) Ta -1 < a, +1-aa))
min(Ax p. and
Explll < ae( tof) + (1 oz). Therefore,
E Ex,~Quy ||
47

---

Published as a conference paper at ICLR 2025
X āid + t [olpell 1
min Ad-p: 112 min Ad 1
N
d(1 - az) + ard(A1 + +oy) + ae
Σ
nn
11 y- - 11t po.
n=1
Ht H.HHu Hoon Il (
< āid + t
max d(A1 tog)
Σ
Tn [
min Ad-p, 1)2 min 1]2
n=1
The proof is complete.
1.5 PROOF THEBRRE
We first recall all the notations in
38 under Gaussian mixture. We also recall the scores from 39
log (x)))
In (2g (21))2 (x (xx pe,n))
lt (xrr)
N
=-
2t +
T Tn ( ke.0
go (xte))
R=1
N
v log (2))
It.nl y (z() > (xt - Me,lly)
at (xre)
n=1
-
Tny (Drs ( Mlyy
Atly (2t)
n=1
Also, we recall explicit expression Ar.y from (41), such that [ 0 and
(t,(( (e(()))
Atuyl
v
TT.
[E 18:
(2))
N
(t,n)y((1)) (1r)(
(ar + (1 --)aa))pppp) Eolyy E "y (1 v voe)oom)
Tn
In order to invoke Theorem Lemmas 13 and 14), since
we need check Assumptions 3 and From (Liang et al. 2024
for all the absolute values of any-order partial derivative bounded by O(1) in expectation, and thus Assumption is satisfied. The following lemma verifies Assumption 4 using the a, in Definition
Sholds if the ag satisfies Definition Lemma 8. Under the same condition of Theorem Assumption holds
Proof. See Appendix J.2
Now we start to upper-bound the conditional KL-divergence of interest. Recall that from Theorem KL Po S Weracle + Wbias + Wvanish. where
(1 - (1-a))²
Worsel
E Exi
Tr
22 log at (m (x))) + (log 7)
20
t=1
Whias
Σ ar
Ox.
Bx (x,)l1)11=
1
at Ex
(Vlog - Vor. log (x,
Wvanish
Ex
T
ax)
Ex,~ (Ar,(
Ex,
log qt-1 (X)
200
48

---

Published as a conference paper at ICLR 2025
a²) 3l0, 3/2
Ex Qelv
3
S. log 11y
1=1
Djgy log 9t 1/y (m). [r (X,))
+ max t21
E Ex (x,) T)
From
Liang et al.
2024
Theorem 2) (and by assumption N & d), if the oz satisfies Definition | (log T)² + (log T)
Woracle
T
Also, from Proposition
T. under the assumption on 0t,, Whins can be upper-bounded as
N
Whias S d
||
H' H
12
Among the terms Weanish the last estimation error term can upper-bounded using Proposition) as
121 max V ExinQu, , (X.)l (logT) 2 d
Tn Hiy - H HHHHoo. (log (log T)
It remains to analyze the rest of the terms in W In the following we fix 2 1. We remind readers of the notations (38). For the first term Weanih: we first provide the following useful calculations. Note that by exchanging the order of expectation, for any function fn we have
N
Ir An. fnn nn =
Ex R
Σ
Σ n=1
Tu TT Ex ~ (n(X,,
2//y (X,)
Thus,
(xt))
Ex,~Q.v
Tn (1))
1(x, - (
N
Tn B TxpBxrr.. Qr.io
1xx: vārHo,n)
n=1
Σ
nr Ex X Qunil || Vāsuon 2 + va Σ
nn || V Bxenqenn ||Xe Varuo.n
n=1
==1
Tr( Tr(Enly)
Fy Hy Ht HtH Ho + āt
N
N
+ 0t
nn || ll |
Tr(2i(n) + āc Σ
|H nn
H H po
n=1
n=1
S <d+ Tn | H H
(42)
where (i) follows from 58
Also, note that (La Hi HHE (- 1/8 (H using the following simple induction argument.
For the base case, have (Ia H H (HtH)
from Lemma
Then, suppose
49

---

Published as a conference paper at ICLR 2025
(la (Ha- H) = 0, we have (Ia H (H H) = (1a- 10 (Ia
112
H'H + H ( H) = (La H Hー (La - H H H) + (Ld
1
H #))
H) = 0. Thus, for all 2 and any fixed vector v, with the
definition of L.y in 40
and 41).
Ex,-Qu
12,-11" 110 A k.
I ||a||| Bxi
- k. = ||01|| Ex Bxi
<4 ||l| |
max [5als 10 (1 0 Td-pp |1
|| 152,"10:oo. 10:14.n
<e(N]
+ 2 ||ll| ( + (1 - - -1 ( [ools + (1 - -a.) (Eolov
E 11=10=11 x
N Qa
Hnmnly (X1) 4oly (Xt) (Xt)
Ex
|x+ Vosponm1| 2
N
S d+ Σ
nn H
H H Hoo
(43)
==1
where the last line follows from (58) Similarly,
N dd+ Σ nn [ Ht H po.n
Exi~Qeiv
E2iyy e) Ar.y
Also, for all 221, 1,
Ex
(2,-1
1/y A1.0
= Ex X (ta - H H Ac.
= -Ex,-0ip
X[A A/- A.vl
S
54|[(ar1
| 00 (1 02-1)) -r max
|||ا 1==1000 1g:Mo.n Ex ||+||
NE
+
( + (1 - ōt-1) (d- 11 [1 Eulys + (1 -a) ( lo Ex 1 x
N
Ex
Tn
(X, Vospo,.))
by (x²)
1=1
N
Sd+ Σ Tn
Hly H HH'Hponn p
(44)
n=1
where the last line follows from
44 and the fact that, from (58).
Vd. Exi-Qul |x S
d
n
2E X
1 Men 12 + 2 ||
d
Tr(Eolp) + or 1Htty H H + d
Tn
n
2 d +
Tn [Ht- - 11 H
n=1
N
S d
ny H H H Hpo
n=1
Similarly, we also have Ex,-Q.
( X T
Sd+ - Tn
N
Hlyy - H" H po..!
ky
Also, for all r 2 1, using the expression of log a y in (3) and noting that by definition (Ia -
H Hx - Me,nly) = (Ta HTH)(I; - (,,)), we have
Ex,~Q4 (E V log (o.g))(())))T (X.))
50

---

Published as a conference paper at ICLR 2025
N
(nnly(X))
A/-1 Σ In n=l
4tly (Xi) Ar H HX: - Momlee Se.y
$4 < 200500 + (1 âc-1) (c-1) -71 (as + (1 - ar de)pp 11 x
Σ Tte Yellly (Xt)
Ex
[X, vāsHo,els
max nE[N]
[ 125i"lgMMu 1g:Mo,ml
4lyy (X,)
+ 221(@rr-( (1 cz- (dd [oo9 + (1 (
[ - Vārmo, Eol lx, vāusoo.N)
XinQtly N [1
Tn E. Fr ||x, - VāeHon,1 112 vd
|X - Vasuo,,ll | - voNHo.N
X11l (1,
Sd n=1
Tn | ( H po.lll
(45)
where the last line follows because, from (58)
E
[]]x, - Valua. X| Valuo,N11
E XimQry (x
||X1 - vārmou_||
x
E XimQuty
||x (e
Vārmo.N1l 1
Xe VāeMo.NN|
Xi-Qey N-
$ d +
nn Hiy H H
1==1
Now, we start to analyze the first term of Wvanish- Recall that m4,((x)) = Ex [x,1]] [x-]] Jou + 1-ae V log 9tly (zz). Using the score expressions in (39), calculate that given 2t vor (and thus myy = m4 2)))
you
log 91-1/1 (1) - ai V log (2r)
Vat Enj di 1e
-E,1 21-1ly m4,0
5-1
gt.n (Ie)
N
--- I
Σ
2ely (1+) Mem + --1 2-1
Tn Σ
Tn
11 (ma.y) M1 1,nl
n=
n=1
(ITT)) )
Varii
1 1- ar 5-1
(ote + V log (xz))
Vot
2-11y
N
g (
+ (1
at) 2
Th Σ
p1 1,11y
Q (k-1ly(Mt.)))
n=1
Entee ne
[t (t-1 y (m,
90, E - 9 (0 pr 1.m1
+ 02
4- y ( (12)
Here, using similar analyses as in the proof of Lemma 5 we get
1 -
ar
(mix Me- Me-1, (I1 Me,nly)
+
v log (x))) - (1 (ai)M--1,al
Vas
Va:
1/1 = 1 a4 2
1 1- a1
1,1
+
21-2 2
2 111
+ +0((1 - )
ar
aj
51

---

Published as a conference paper at ICLR 2025
9-1,, m 1ace ( 9t- -1,1, (ml, (16)) (2t)
)
+ (g-
(m 10.00 - 2 2--1, 1, (e.
-0t-1.ll Mr
(mma -1-1,00) (xx (1,,lo)) (mey. 1,-1,
(0r 14,A1w T (mp.) - Ph (1-1,8l))
2
1
(ot (1,81l) (E - 14-1,80)) (x) (1,01a))
2
1
((mLy Pr- p - (xe Me, w)) 1-11/ (mery He 1n
2
1
(ot Ma,) ( (EE (mg) Pt pt-1,)ly)
2
1
2 (2t Me,)lu) T (mer 14-1,)ly) (xt Memlll)) Edl
+ +0((1 (²)
Thus,
Ex
(Vlog log q- (1 - -aaaaave log T
|XX(ETT -
< Ex
+
at Ex.nQix
|x, Vog 91lw)
T ((=,1 ) (
T
9t 9111,nl (Me.z)
(1 (k.) -xx)
Tn
21-1/y (m,,y)
11-1,1
En Anw
(me (X,)
-1-1,899 (m Ie, (Xx)
AT
5--1
52-1
Tn te
01-11/ 9:11 (
1-11/
4- (me. Jtly (X.)
pe Me-1,m
n,E=1
Among the four terms above, the first term S d + ΣN Tn
Hiy -
HH H p0, || from 44 (along
with the similar result for D the second term S d + En Tn
H Ht H po.n
from 44
and (45). and both the third and the fourth term S d + N Tn
Hty H H from
(along with the similar result for ΣTTy). Thus,
N
Ex Ex-- (V log (m Vou V log 2ol (X.))
nn Sd+ Id+
Hi
Ht HH
n=1
and the first term in Wvanish satisfies that
at Exiaeen (Vlog VarV log 41 (Xx))) A (xx)
/a
N
H HtH po 122 (og) T)²
+
T
n=1
For the second term in Wvanish, we first provide the following useful calculation. Similar to 43 for all 2 1 and any fixed vector v,
Ex, Ex,~Qay
( A..g)
52

---

Published as a conference paper at ICLR 2025
||ell Tell/Exxwee Ex | 4 || ||w|| E Qx1 || A 1,2
s ||ull (ā. 5v + (1 - ac)ld-p) -1
||= 1::''lgPP00 18:Mu
max nE|N
|el| (1 Eools + (1 - T.) 1
(ov Eolgy + 1 (pp))a-))-1pa00 Er 11 1v:11
nn Itmly (Xi)
Ex
E
| varuonm!l
4uyy Xt)
N
S d2 + Tn
|| H
(46)
1=1
where the last line follows from (58)
Also, similar to (44) for all r 2 1,
ExIQQ m A
2
$ Ex.mQry
X (.,0)
+ (1 - ax)
(Vlog log gely (Xt)) 22 A
Ex
x7 11y (Id
H H b.
= Ex [XT ATTAAc..)) c. ?
$ [a, Eol + (1 az) || 2
|>n 10. Mu. 1| Bxi~Qejy ||xell
max DE(NN]
+ [ Eooes b3 + (1 ā,) ( 1
1|
[1 [Eols + (1 - σz)) - [Eo] Gollvrl E., 1): 112 x
nn hnmly (X1)
Exi~Q1l
4ily (xt) (X, )
S d2
nn Hty Ht H
(47)
==1
where (ii) follows from the fact that Ex,~Q (V log qe (x Ar. 12 x d2 (using similar
argument for deriving (45) ). and the last line follows because
N
d. Ex |x < d
(2Ex, ||X, - 1 2
R=1
(ti))
N
(Tr(Eolw)) + ar H Ht HHon (+()
d
Tn
2
d
nn
| y Ht H | 1|
where (ii) follows from (88 and also
lt.nly (X,)
(X, Valmo.)) 2
Ex
nn (X)p(1))
((X, VālHo.N) xx,)
= Xi-Q+iy
(e
2E X (-1
||Xe VāsHo.N11* +2 E Xe
|x, Vārpo.N EN ||
V
d²
Hly
Hi Hpon.l
nn
53

---

Published as a conference paper at ICLR 2025
where the last line above follows because for all and each [N].
Ex 1| VārHonn ll"
ExxmQunil
|x=, VasHlo,nivll" + VVlfoonlly VālHo.nll
E Ex,~Qenn | (Xx - Vonkoonlu ||
+ (ax) /|
H H
d0 H H pu..nll
(48)
Now we are ready to analyze the second term of Wemnis Note that
2 log Qt 11 (m+x)
N
( - Me,nle)(ma - Me,nly) (2 - -/ qt (M
41 (
n=l
Tn 4t Inly (M
q (1-1mmle ( S1p/y (may Menel))
52 (ma.y Menllw) (t-1ly(Me.g))
Σ
on qt-1ly (Mw.g)
Thus,
Ex.~Qul
A.p 2
log q: 111 (me,))
N
( - Hemlle) 2
< 31Exx0
10-11 (ma
+ 3Exx~Quiv
1. Diyy A..9
2
Tn l 1.11 (m4,))
+ 3Exi~Qriv
AT L.y Pity - Me, w)
90-1/y (me,y)
< 3Ixx-QQu
L 1/ Ae.y
It (ma,))
+ 6Ex,~Qivv
(AT ET (mex Memlu))
q: (m(.p)
To determine the rate of these two terms, get
Ex.~Qu1
AT, - A..y = Ex.~Qu
H y H H nn A,A,, dd+ Σ
n=l
and
nn 4t 1,mly (m4,)) 9 90-11y (m..x)
Ex, Q
(mew - (e.mllu) 2 Dr.y)
==1
Ex NEIN
max
(Meyy
Me,lly) 22 Ar.
Ex
m 2 f/y D..
max + +Ex WE(N]
Ac. 2 Ac.
$ N
nn [Hy- 11 HP
where the last line follows from (46) and
47 Thus, the second term of Weanish satisfies that
T
(1 az)
Ex,
log qt (X)))
201
54

---

Published as a conference paper at ICLR 2025
N nn H'y H
c² (log 2) T
S
N &
H HHoolll
n=1
For the third term of Wvanish- we provide the following useful calculations. element-wise (Hadamard) third power of vector v. For each n [AV. we have
Denote yad as the
-x, (mayy - Me,nly) (Ia H Hr.) = -x, |1Arylymm - pr. 103
x -x ||A.,1](X - Mt,nl 103
+ (1 - only (1-wo))
Exi A Bxi / Vlog (
(10)
Ex | ( Xx - m 103
max ||> 1o:Ho.el
Bx Bx I|Xe - Me, ll +
Xi Lull
||x - Me,21la 1Xc,-e,
S E x,~Qu1s [
1|X+ - me,ell + ELAH l1
n1,1l1
+E E
Xi~Qi1a bo)l)je((().))
||x - ,e,1,l
d +
Ta H H H
(49)
where (áv) follows from Lemma 6 (using the in last line follows from (48). With similar argument,
8 and (Liang et al. 2024 Lemma 15), and the
Ex,~Qul [mey Me H H)
H)
S
XemQriy L
| mr.1ll
d +
H H H po
(50)
n=1
Now, employing the notations from (Liang et al. 2024
Section G.I), we define
(x) = -ETj(r Mennly);
(x( i) = max (x)(()))
E = max [
When H = (Ip 0), we note that
1 = (1 ās + t
(ar Eol 50 + +1 oz) -a)
Thus, we have Σy 0 whenever (i.j) [1
1, d or (i.j) 1p+ + 1, x [1,p] and
jel+ 56
(1). Since og 0, we also have (TT $1.
From Liang et al.
2024 Section G.1.2), an upper bound for third-order partial derivatives
ijk log 40 (x) < 66,(2, (z, (.. k) + 2 Ex k) + 2 (x, 2Σ% (x i).
We also remind readers that At,y is supported on rango [a H H). namely that [Ar.plo
Now, the third term of Weanish can be upper-bounded
Exxx
03 log a1 (M)
= =Ex,<Quv
Σ
log (1- (X. (X,)²
as
ip+1
d Σ
< ExI~Quy
(x
(X,,
+ +X
S (X),, (X)), i)
1
1
55

---

Published as a conference paper at ICLR 2025
(as Eoola9 + (1 -1-1 pp) -11 Ex
Σ
(m mi Se.yy (Xx)²
imp+l
[2olo0 + (-=r))dappp-11 1xx_0x (Ix
(Xea(())) Pi, A
d Σ Tn H H H' (00mmll))
S N
Here the last line follows from (49h and 50
We provide the following useful calculations to upper-bound the fourth term of Wvanish- First, for all r 2 1 and any fixed vector v.
1 1 cz Xx +
ExyaQuy |mmay
111" = -ri
V log 2tlu (X,) -u
Vo+
Vas
Qxi
1xe-vll + (1 ar (x))ExinQrn
V Vlog 20 (ul (X))ll
243
XinQtia (
|x -0,1,l + BL l| - wll"
S dr/2
Tn H H$ H H
(51)
where (v) follows from Lemma last line follows from
6 (using the a, in 8 and (Liang al. 2024
Lemma 15), and the
(48) Now,
d Σ 1=pt1
E Ex
Ex,
Ec mm
E Qei ||mea Mem
(o (me. Dew
m,(=1
N
Ex || Meenllll (Exieen llmaw - Melel Ex 1 (X.)1(+) 1/4
<
f=
NN
$
N2
de
+
nn Hty - H HHHHHoon H
(52)
==1
where the first inequality follows from
| S and the last line follows from 5T and Also,
Ex Ex,QQu
Σ
(m) (1
ijj-p1
Siz 2
Exi~Qui
El(me.g 5) De, (X,)²
1=+11
j=p+1
N
E Ex [mmy - Me,)) D (t.((()))
4
*=
==1
N
S
E Ex.~Qui ||me me,ll Exerqy
#+
1=1
56

---

Published as a conference paper at ICLR 2025
n H
Nd
d+
Σ
n
H H po
(53)
n=1
where the last line follows from Proposition
and (51). Also,
d Σ
Ex,~
E
Em 8) a) (Xe)
w=ptt
x ExiQQ
Σ ()
Σ s=p+1
(X,)
i=pt1
d. ExeeQuv
Σ Ee(megg:
v d. Exiequ (x,)
imp+1
N d.
Ex |mmay - Mt, V Ex (x,)l
n=1
d Σ Tn n=1
H'y - H Hpo.mll
(54)
where the last line follows from Proposition and (51). Now, the fourth term of Wanissh can be upper-bounded as
Ex,~
Σ
L log (X))
1==1
= =Ex,QQiv
Tiey log q: 1--%
j=p+1
< Ex
(m (
(X) (X,)
s=p+1
d Σ
+ E Ex
d
+ Ex
Σ
D (x))
1-j=t1
For the three terms above, the first term S N2 ( EN on | 11 po. It') from
the second term $ Nd d + ΣN n. Hty - Ht HItHoon H 11') from (33 and the last term
S Nd (d+ EN Tn H - H H po... from (54).
Thus, overall, with the a, in & (cf. Lemma 6). the third and fourth terms give us
T
a.)
d > 0 log g.- (X.,)) ( (x,,))
Ex,~Qrv
3la 3/2
Yy log 91-11/11 (M..)
57

---

Published as a conference paper at ICLR 2025
N H
(log T)?
S
N22 &
+
Σ nn
- H H.Hoo
T
n=1
Therefore, combining all the above, since N constant,
N
nn HHy - Ht HyHHpon! 11'
(T,(Qo)/
Poly) x
d+
n=1
N
(logT)²
T
+
d²
+
Ty H H H
T
nm
|
d
+
nn |Ht Ho, (log T)e.
Remark 2. In case of general σy 0, the same upper bound can be applied |
as detailed
in Remark Thus, with the or in we similarly have
(log log(1/0)2 + T
Wornel
T
The rest of the proof is similar. Combining with Lemma
we would finally obtain
N
(1-
2 loo log T
KL(Q11111P1I)) S d+
H y H" H
T
n=1
d
nn
[1l
H mommlll)
(log 7)2 ())(1
d
N
Ta
lHll
11 H1 11 (log T)s.
T
Here W2(Q1 22 $ öd.
J AUXILIARY LEMMAS AND PROOFS IN SECTION 4
J.1 PROOF OF PROPOSITION 2
Given Qo NN Eo), from the conditional forward model in (5), we can calculate
Q: -N(Vaa aLL. + (1 - -a) =: N 2.)
Q NVVaall - H H) + H',
TilTa H H'H H' H) + Hn +(1 aa).
Note that when H = (Ip 0) and o > Øo1y exists. Define
Prly Vo H HP Vo,H'y
Σr,sig = ār - H HI H*H) + - aa)la
Edly = En,sig + (
(55)
Here Mi is the signal variance at time t, and May is the total variance of the signal and the measurement noise. Note that when H = (Ip 0), EEpuls = Eigliglos. We also calculate the respective scores of Q. and Qtly:
V log (x(e) = -2, (xt - pt),
log (x;) = - -M (2: - Mely))
Since Jt.y Si, (defined in 10 from
32 the bias at each time is equal to
Dry = (La - H H)
log (xe) - v lo (2x))
= -la- - H1 H)
[ - Vāruo) ETy vorHoly)
58

---

Published as a conference paper at ICLR 2025
(Ia - H H) (x: vāruo)
-Ia- H)
varHiy).
Now, define
+ A, = (Ja - H H) (1a (Ia H
V8 H)
+ (H))
Thus, we have M Lt, + Cee V1 and Mgly Lig + t of ( (Hi)T. By Woodbury matrix identity, for any matrices A and B, their sum can be inversed as (A B) B)- A- A B(A+ + B) Thus, we get
x (Er,sig + avVi)) 200i T
t,0ig ViLa
Σ (Et + t H (H))H)) )T) 1,
0, 2.
H (H))
Eو 2 sig (H H)
where (i) holds under assumption H- = (1p 0). Thus,
De.y = (La H) [T (Ix - vārHo) - Iiyy (2t vauHolu))
=(Ia- H H) (It,d L, ViE, 1) (a's vāauoo
(Ia H'H) H)
Σ-1 0, coigg (H) H) (ot Vorlla H H'HO voiH
(
(Ia H1 H) Ligg σe 1,31g V1SI (xt vo(loo
(Ja H H) sig (It (Ta H -a,
= -ā,(T - H H| 1ig V,E, (z+ - VorHo)
+ (la H| (Lo 11' ()) - vosuo) 11 H (x+ - varpo))
(Ia H) (Io H H)( - VāsHoo) + (HHH)) vā(Hiy) v)
(10)
-ār(la H H - s0 Vi2i (xt vāeHO)
(iv)
-á - H H| sig (La - H HiH) HY (xt - vāruoo
= - (H H)Σ, (It Vās(Hu)
(56)
where (iv) hold because (Ia - H H) L,sig (H = 0 by Lemma 9
(4p 0 we can re-express A, and Dry as follows.
Now, since H =
A
(Ta
Ht H
E (Ta
-HH)
0
0
(1 Tp
1(0
0
0
Ta-p)
0
on Eooys +1 a.)
Ta-p)
(Colas + (1 5c ad) -1
D (n Dolly + (1 da) - [5, 1T (x Vonue))
0
and we have
Ex,~Qri || 112 = a2 Ex (it + (1 αxe (d - Eolog 12) Iy: - vāsuo)l?
= = Tr (1rr/oolss + - (1- Tapp) [Eolsv [r, 1T (X, - Voruo)(Xe - voruo)
59

---

Published as a conference paper at ICLR 2025
[2 1 Volus (ār (ol99 + (1 ai.) (z))a-p)-1)
I at [( Dolos + (1-1111) -s) 11 12, 10:
[[ Ex || - vāruol
/| (57)
where (v) follows from the fact that [T(VV) & [r(V) V is positive semi-definite. To
analyze each norm above, denote À .... 2 X > 0 to be the eigenvalues of Eg, and note that | < Eoll = λ1 The largest eigenvalue of E, is (á, + (1 - (x)))=+ and note that
[ 11-1g=11 | Also, since [Eolyg is positive semi-definite denote \ 2 2 λd-p 0 to be its eigenvalues, and thus
[[ + (1 Ts)
I min(dd-p.1) 13 80.
aidd-p (1- āz)
Also, since
Ex,-Quiy (X, - Voaspo) Vānuuo)
=Ex,0Qn (X, - -tll + Vor(H y Ht H Mely Vor(Hy H Ho)))
= =Eel + @H( H -HiHHO)
we have
Ex,aQr |x, Varuol 1|2 Tr((-nny) at H Ht Holl
(01)
a + +gg) + (1) + āt [1 y Holl
max {1l H1 H H1 uoll + d(Ai + o)
(58)
where (vi) is because Egjy is positive definite with
[ = =Ilār(Ta Hi HIH) - o + +1 Ta)
Sā |Eoll + āzo + +(1 (1-
=a((A + og) + (1-a.).
Therefore, for the second moment,
max H - Ht Holl 1| + d(A1 + +
E Ex,~Qniy ||
< a3
||2iolus [2olovl22o
min (da, 1}2 min da (da-p. 1]2
S a max || - 11 Hool + d(Ai + +o))
since [Eolyg
Eg Gvll <
1 x7. Also, for general moments with 2 2,
||
m
(ā. Elyg (1
( vorno))l
< 0
( [Eo] yg + (1 a.) Ta |
1x 11
100 voluo
S a
[ Va.uol
m
and thus
Ex,0Qux || m S ax m
Ex
| (X, mmw)
+ (x))
+Il H* H)))
< a m (m- 1)!! d |Vaa(Hl H Hoo)ll O(ā))
Therefore, Assumption 4 satisfied if the oz satisfies Definition
1
The proof is now complete.
60

---

Published as a conference paper at ICLR 2025
J.2 PROOF OF LEMMA 8
We continue from the expression of Ar, in 440 when Qo is Gaussian mixture. For m 2 2, we have
Ex,~Qelv |
20 (a.) m/2
12 mo,n
max NEIN]
+ 2 (at) "E
[1(Ta - H H) t, (Ta H -Vāspo,N))!
XenQely x
X (04) (at) m2 qm/ (at) "E
Xi-Qeu [=
||,, VassooNN1
(ō:) m 2m/2 + (a.) "F
Xt+ Vor/oNNN 1l
N-I1
X
(38)
mm/ (
dm//2
Tn H y H H Ho
(59)
= =(
Here (ii) follows because
(Iaa 11 H) (1a H H'
[[(rr[olaa + (a-=))aa-pp-11 22 1 "
A1
O(),
min (ain(As-pi 1) (
and (iii) follows from (48). Therefore, this verifies Assumption The proof is now complete.
when the satisfies Definition
J.3 LEMMA 9 AND ITS PROOF
Lemma 9. Given a positive semi-definite matrix Σ, 2 0, and E (0, 1),
+ ( - H) 1-a (Ia H
H' H H) + ao H H + (1 a))a) (H* H)- =0
Proof. The key of the proof is the Woodbury matrix identity, which states that for any matrices U E RdxP, V E Bexd
(Ia +V) Ulp + VU)-IV.
05 For the first equality, we apply Woodbury with = I-a
H and V
ad (H and we get
I-a
1 (aOH'(HH) +1 1 a Ia H!
11 ( -1 (Er)T).
a
Since p < d, the pseudo-inverse equals H1 = (T(HHT)T and by the orthogonal property we have
(Ta H H) (HH H 0.
We have thus shown the first equality.
For the second equality, we first consider the case where o = O. Write S = (1 a) + a(La
H' (I H H). Since E is positive semi-definite, there exists matrix L such that Σ = LLT. Thus,
8-1= (1= 11- )
(11- HT)H (T6-)(1))))))
61

---

Published as a conference paper at ICLR 2025
la
((Ta H H)L
i [Ta Htm))) LT(Ta HtH,,
la+ 1-
where in the last line we have applied Woodbury with VT VT= - (Ta (Ia- H H)L. The equality achieved because
(1a H (H)S-
1 1- (I) - H H
(La mmL) LT (
(Ia HiHHL
la 1-a
=0.
When σ > 0, we can apply Woodbury identity to sum of matrices and and get (A+ BB= A BBA + B) B) Thus,
(S + HO - (H' H) H
and
(Ia- H H ao Hn (H
H
ao (Ia - H (nn
The proof is now complete.
J.4 LEMMA 10 AND ITS PROOF
Lemma 10. With 1 a₁
log
Vt 2 0 (which satisfies Definition 7h given any p 0.
T Σ 2=
az)
111- 2pc log
+
1 T2
T
Proof. Define the sum ST. Then,
2p 1 - (1
clog p(T'-1)
clog? T
clog T
c
pt
clogT T
clog T
8T
1
T
clog T
1
clog
clog T T
2p
F (1 + O(T
T
1
clog T 1
(1
2pc 2pclog? T
Ō(T-2).
p
T
62


Reviews:
[{"review_id": "2J7DkgMc43", "reviewer": "Reviewer_w6Vp", "summary": "This paper studies the convergence guarantees of zero-shot conditional diffusion models under certain regularity assumptions on the target distributions and assuming small score estimation errors. The authors provide distribution estimation error bounds in KL divergence for score-mismatched DDPM and establish convergence rates under several settings. They also introduce a new zero-shot conditional sampler designed to accelerate the convergence rates by reducing the asymptotic distributional bias in the obtained bounds.", "questions": "Q1. How does Assumption 3 relate to the standard smoothness and regularity conditions typically required for Langevin dynamics sampling, such as Lipschitz continuity and bounds on the gradient and Hessian of the log-density?\n\nQ2. Considering high-dimensional data, how does Assumption 3 ensure that the expectations involving higher-order derivatives remain well-controlled?\n\nQ3. Are there any connections between Assumption 3 and functional inequalities like log-Sobolev or Poincaré inequalities?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "S1. The paper is well-organized and easy to follow.\n\nS2. The convergence results for zero-shot conditional diffusion models are new and address an area that remains largely unexplored in the literature.\n\nS3. The authors propose a new zero-shot conditional sampler that accelerates the convergence rates of the zero-shot conditional DDPM sampler by reducing the asymptotic distributional bias in the obtained bounds.", "weaknesses": "My main concern revolves around Assumption 3, which appears to be relatively strong and may limit the applicability of the results. Specifically:\n\nC1. **Smoothness Requirements:** The assumption requires the existence and control of higher-order derivatives of the log densities. Distributions that are not sufficiently smooth (e.g., those with discontinuities or singularities) may not satisfy this assumption.\n\nC2. **Scaling with $(1−\\alpha_t)^m$:** When $\\alpha_t$ approaches 1, the factor $(1-\\alpha_t)^m$  becomes very small. If the expectations do not decrease at the same rate, the assumption may not hold. This can occur in practice when $\\alpha_t$ is close to 1, as in the noise schedule Equation (8) in the paper.\n\nC3. **High Dimensionality Issues:** In high-dimensional settings (large $d$), the behavior of the derivatives can be more complex. The curse of dimensionality may make the assumption unrealistic because the number of derivatives grows exponentially with $d$.\n\nC4. **Applicability to other distributions**: The assumption may not be valid for distributions outside certain classes, such as those with multimodal densities where modes are not well-separated, or distributions with skewness and kurtosis that affect the derivatives.\n\nAdditional concerns include:\n\nC5. **Avoiding overstatements of contributions:** In Section 1.1, the paper claims to provide convergence bounds for general target distributions with finite second moments (e.g., Line 92-93). However, their analysis requires regular high-order derivatives for the score functions, which significantly restricts their results to smooth distributions. Adjusting the claim accordingly would provide a more accurate representation of the scope of the contributions.\n\nC6. **Insufficient Discussion on the obtained bounds:** The paper lacks sufficient discussions on the practical implications of the bounds. For example, how do the obtained bounds compare to conventional conditional sampling methods, such as Langevin dynamics? How can the bounds explain the success of conditional diffusion models over other sampling methods?\n\nC7. **Clarification on early stopping in Theorem 2:** The bound in Theorem 2 is given in terms of the KL divergence between the target and learned distribution at time step 1 rather than at time step 0, implying that early stopping has been imposed. Clarification on this point would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wXWcN08KSn", "reviewer": "Reviewer_nM2R", "summary": "The paper derives KL convergence bounds for diffusion models when the model is not trained on the target distribution but on a mismatched one. Under some assumptions, the authors exhibit a bound with a term depending on the mismatch. Building on this result, the authors propose an algorithm for conditional sampling that minimizes the mismatch term.", "questions": "What does the final line in theorem 2 signify: \"Here $W_2(\\tilde{Q}_1, \\tilde{Q}_0) < \\delta d$\"?  Is it an assumption or a result of the theorem ? It is unclear from the proof as well.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper participates in the effort to provide theoretical characterizations of guidance and conditional sampling in diffusion models, an area which has not yet been well explored.", "weaknesses": "1. Assumption 2 is not realistic: Assumption 2 measures how much the approximated conditional expectation deviates from the true $\\mu_t$. The expectation in assumption 2 is taken with respect to $\\tilde{Q}$ the target distribution, which is unknown during training. Assumption 2 can thus not be guaranteed to hold. Moreover, the authors state mistakenly, as far as I understand, that Assumption 5 is the same as Assumption 4 when is taken under different distributions. Assumption 5, under $Q_t$, is a classic assumption and, as it relates closely to the training loss, is a realistic assumption. Assumption 4 on the other hand needs more justification.\n2. A minor modification of prior work with assumptions to control the additional terms: Theorem 1 is obtained through modification Liang et al 2024 by assuming the target distribution is different from the training one. The simple additional terms that appear due to the modification are controlled using a series of assumptions. This limits the technical novelty over Liang et al 2024.\n3. The paper is not easy to read. This can be fixed by reducing the large number of inline equations (paragraphs line 513-523 or paragraph A in the appendix). Unnecessary factors $(1-\\alpha_t)$ appear on both sides of an equation in crucial assumptions 3 and 4 which I found to be confusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DpVdAQu1Xy", "reviewer": "Reviewer_n4bv", "summary": "This paper mainly focuses on the convergence analysis of zero-shot samplers when training scores are based on the unconditional distribution while inference samples for a conditional distribution. Under these conditions, the inference required scores vary from the predicted one, which is considered as the score-mismatched diffusion.\nAlthough various practical works have appeared, from a theoretical perspective, this is the first work to analyze the inference convergence under score-mismatched settings. The core step of this paper is to recognize and minimize (Theorem 3) the mismatch scores (Eq.(7)) by considering the conditional reverse process (Eq.(5)). The conditional reverse process is deduced by a known connection (e.g., linear connection) between the random variables and the conditions (Eq.(4)).", "questions": "1. Theorem 4 is somewhat confusing. Specifically, it claims the establishment of Theorem 2 under the specific choice of $f_{t,y}$, while Theorem 2 has additional assumptions for $\\E[\\|\\Delta_t(X_t)\\|]$. The problem is whether the upper bound of the mismatch-score, i.e., $\\E[\\|\\Delta_t(X_t)\\|]$, is required for Theorem 4 when the conditional target distribution is general.\n2. The BO-DDNM is too dependent on the linear structure of $y=Hx+n$. Considering a general mismatch-score setting, i.e., $y=h(x_0)$. How can we adapt BO-DDNM to the general case except for a Taylor extension?\n\nI am willing to raise my score if the authors solve my concerns.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The problem setting considered in this paper is interesting, especially regarding the effectiveness of the zero-shot sampler in the score-mismatched diffusion.\n2. Authors denote the gap between ground truth and generated samples lies in the accumulative mismatch between the target and training distributions. This qualitative result is useful and aligns with their sampler design.", "weaknesses": "1. I am a bit afraid that this paper overclaims the problem settings it actually solved. Specifically, this paper claims to consider a general score-mismatched setting. Although it provides a convergence analysis (Theorem 1) for such a setting, the assumption, e.g., (Assumption 4) required, is highly artificial. Besides, it can hardly find an algorithm to achieve Theorem 1, since authors do not provide any approximation for $\\grad \\ln \\tilde{q}_t(x_t)$ under a general score-mismatched setting ($y=h(x_0)$). Instead, I think this paper more focused on the linear setting, i.e., $y=Hx_0+n$ (Eq.(4))\n2. I am curious about whether the problem setting is really useful beyond the super-resolution problem. Since the conditioning (y) is usually correlated with the random variable (x) implicitly. I suggest authors provide more examples of utilizing zero-shot samplers to solve score-mismatch diffusions.\n3. The establishment of Assumption 4 requires strict assumptions. It not only constrains the unconditional distribution $Q_0$ but also constrains the $H=[I_p, 0]$. I hope the authors can demonstrate whether the results can extend to a general $H$ and how.\n4. The final KL divergence bound between ground truth and generated samples is unsatisfactory. Although it is an upper bound, it contains a factor O(d). Such a result will make me concern whether a zero-shot sampler is really effective for mismatch-score settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zaaloLBiZm", "reviewer": "Reviewer_oUs6", "summary": "This work provides the first guarantees for conditional sampling (within the inverse problems setting) via diffusion models, which approximate the guidance term via techniques that do not require additional supervision (e.g., x,y pairs such as in CG and CFG), which the authors coin zero-shot as they do not require to observe any additional data to fine-tune. \n\nThe authors then provide a preconditioned linear-approximation-based approach to their proposed decomposition of the posterior score, which minimises the mismatch; the proposed estimator is very cheap and could be useful to the community. Furthermore, the authors provide a very modest numerical benchmark showing significant improvements to denoising diffusion null space models. In general, the strategy/sketch used in Theorem 3 can be of great use as a template to practitioners looking to derive quick yet accurate estimators.", "questions": "1. I am slightly confused by theorem 2 following from theorem 1 (more specifically, Corollary 1 following from theorem 1).  Theorem 2 bounds the KL between Q_1 (the marginal distribution of the zero-shot sampler at time 1) and P_1 (e.g. the time 1 marginal of the VP-SDE at time 1 where P_0 is the conditional distribution). Typically for all practical purposes in diffusion models Q_1 is simply a Gaussian and this bound can be obtained from exploring the ergodicity of the OU-process and prior work on the convergence of diffusion models. So two questions \n\n     * Why is this bound for a general Q_1 relevant? We typically fix/choose Q_1 , so Im a bit confused about what specific aspect of the zero-shot diffusion models considered in this work induces/requires a particular Q_1 which needs to be analysed \n     * How does corollary 1 follow from theorem 2 , do you simply just preform the same analysis in reverse time ?  (e.g. flip the time indices) more detail here would be useful \n\n2. How realistic is assumption 4 for zero-shot models such as Chung 2022a-b, Chung 2023 where the score mismatch cant really be controlled as it makes rather unrealistic Gaussian approximations of the transition density in the backwards process.  Assumption 4 is a rather specific bound on the mismatch it seems its form is required to make everything else work so discussion on its relevance outside of a Gaussian setting for the prior (data distribution) would be extremely helpful.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clear and well written\n2. The short summary sketches of the proofs allow us to for a high-level verification in the soundness of the results\n3. The work is novel and clearly relevant to the community; with some small modifications and clarifications, this work can be impactful.", "weaknesses": "1. The reviewer found the presentation of the paper a bit confusing. Calling methodologies such as reconstruction guidance (Chung 2023 in the paper) zero-shot (whilst potentially accurate) can be confusing and misleading for the following reasons: \n\n\n      *  Whilst many of these methodologies do not require additional supervision (e.g. observing x,y pairs such as in CFG or classifier guidance) they do often still approximate the same quantity e.g. the h-transform / guidance term [1], ultimately with the goal of approximating the posterior score.\n\n     *  Many of these zero shot approximations induce costs that are comparable to training when one considers (see wallclock times in [1]) , furthermore with methods like DPS / reconstruction guidance we can see that they are effectively fine-tuning/minimising a reconstruction loss across timesteps, whilst there's no extra data (thus arguably zero shot) , there is certainly a significant extra cost arising from this \"online-finetuning\". \n\nIn short, I think the paper would improve its reach/impact if it revised its presentation of what it considers to be zero-shot, or make it much more precise from earlier in the manuscript. \n\n\n2. Assumption 4 seems quite strong and rather non-practical for some of these zero-shot methodologies, which make Gaussian approximations on the reverse process and thus do not induce a mismatch that can be controlled. \n\n\n[1] Denker, A., Vargas, F., Padhy, S., Didi, K., Mathis, S., Dutordoir, V., Barbano, R., Mathieu, E., Komorowska, U.J. and Lio, P., 2024. DEFT: Efficient Finetuning of Conditional Diffusion Models by Learning the Generalised $ h $-transform. arXiv preprint arXiv:2406.01781.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XoqjdUCpzV", "reviewer": "Reviewer_n16U", "summary": "This paper provides a non-asymptotic convergence theory for diffusion models with a mismatched score function. The results can be applied to zero-shot conditional samplers, especially linear conditional models. The authors also propose the Bias-Optimal (BO) DDNM sampler and provide a comparison with the previous algorithms.", "questions": "- The big-O notation in this work is confusing. To name a few, in Assumptions 3 and 4, should $(1-\\alpha_t)^m \\mathbb{E}[\\cdot] = O((1-\\alpha_t)^m)$ be equivalent to $\\mathbb{E}[\\cdot] = O(1)$? Or do the hidden constants in these assumptions implicitly depend on $(1-\\alpha_t)^m$? In Definition 1, the noise schedule needs to satisfy $\\bar{\\alpha}\\_T = o (1/T)$ which is defined as an asymptotic bound $\\lim\\sup_{T\\to\\infty} |\\bar{\\alpha}_T / (1/T)| \\to 0$, while Theorem 1 presents a non-asymptotic analysis. How does the asymptotic assumption apply to the non-asymptotic analysis? I suggest the authors clarify the notations or even explicitly write the constants' dependency on the hyperparameters if possible. \n- Does $1-\\alpha_1=\\delta$ in equation (8) contradict with $1-\\alpha_1 \\lesssim \\frac{\\log T}{T}$ in Definition 1? What can we obtain from Theorem 1 if $\\alpha_t$ is chosen as (8)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The convergence bound of DDPM under score mismatch applies to a variety of target distributions without restrictive assumptions. \n- A bias-optimal zero-shot sampler is proposed, and its convergence rate is analyzed.", "weaknesses": "- Assumption 2 requires an upper bound on the score estimation error $\\epsilon^2 = \\tilde{\\mathcal{O}}(T^{-2})$, which could be restrictive compared to the previous literature (e.g., Li et al. 2024b) which applies to any $\\epsilon$. \n- The in-line equations in Section 2.1 and Section 5 are hard to follow. I suggest the authors re-organize the equations for better readability, especially by highlighting the definitions and differences of $q$, $p$, and $\\hat{p}$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)