Understanding Guidance Scale in Diffusion Models from a Geometric Perspective
Authors: Zhiyuan Zhan
Liuzhuozheng Li
Masashi Sugiyama
Venue: Accepted by TMLR ()

Abstract: Conditional diffusion models have become a leading approach for generating condition-consistent samples, such as class-specific images. In practice, the guidance scale is a key hyperparameter in conditional diffusion models, used to adjust the strength of the guidance term. While empirical studies have demonstrated that appropriately choosing the scale can significantly enhance generation quality, the theoretical understanding of its role remains limited. In this work, we analyze the probabilistic guidance term from a geometric view under the linear manifold assumption and, based on this analysis, construct a geometric guidance model that enables tractable theoretical study. To address regularity issues arising from multi-modal data, we introduce a mollification technique that ensures well-posed dynamics. Our theoretical results show that increasing the guidance scale improves alignment with the target data manifold, thereby enhancing generation performance. We further extend our framework to nonlinear manifolds, and empirical results on real-world datasets validate the effectiveness of the proposed model and are consistent with our theories.

Published in Transactions on Machine Learning Research (02/2026)
Understanding Guidance Scale in Diffusion Models from a Geometric Perspective
Zhiyuan Zhan
shon@ms.kkutoogoo....p
The University of Tokyo, Japan RIKEN Center for AIP, Japon
Liuzhuozheng Li The University of Tokyo, Japan
fhcchuuzhengg-k@@ooloooock com
Masashi Sugiyama
wugi@k..utokky..o.. ac.
RIKEN Center for AIP, Japon The University of Tokyo, Japan
Reviewed on OpenReview: https: // openreview. net/ forum? id= id=nfHammo6
Abstract
Conditional diffusion models have become a leading approach for generating condition- consistent samples, such as class-specific images. In practice, the guidance scale is a key hyperparameter in conditional diffusion models, used to adjust. the strength the guidance term. While empirical studies have demonstrated that appropriately choosing the scale can significantly enhance generation quality, the theoretical understanding of its role remains limited. In this work, analyze the probabilistic guidance term from geometric view under the linear manifold assumption and, based on this analysis, construct geometric guidance model that enables tractable theoretical study. To address regularity issues aris- ing from multi-modal data, we introduce a mollification technique that ensures well-posed dynamics. Our theoretical results show that increasing the guidance scale improves align- ment with the target data manifold, thereby enhancing generation performance. We further extend our framework to nonlinear manifolds, and empirical results on real-world datasets validate the effectiveness of the proposed model and are consistent with our theories.
1 Introduction
Diffusion models Ho et al. 2020 Song al. 2021a) have achieved statc-of-thh-art performance generative tasks across various domains, including images (Dhariwal & Nichol 2021 Rombach et al. 2022 text-to- image synthesis (Saharia et al.
2022), videos (Ho et al. 2022), and audio (Kong et al. 2021 As result, their empirical success has led to increasing interest in understanding the theoretical foundations of diffusson
models (De Bortoli et al.) 2021 Loe et al.] 2022 Chen et al. 2023c a Gao et al. 2025). In particular, under the manifold hepothesis (Bengio al. 2013), the ability of diffusion models to output high-quality samples in high-dimensional spaces motivates rescarchers investigate how these models can generate distributions
supported on low-dimensional manifolds in high-dimensional ambient. spaces (De Bortoli 2022
Oko et al.
2023 Li & Yan 2024 Wan et al.) 2025).
Controlling diffusion models to generate conditional distributions is another active area of rescarch. Based on the theoretical framework proposed by Song et al. (2021b), both classifier guidance and classifier-free guidance models [Dhariwal & Nichol, 2021 Ho & Salimans, 2022) apply a probabilistic guidance term derived from Bayes' rulc to guide the sampling process toward the target conditional distribution. These methods also introduce a scale adjust the strength of the guidance, and they showed that the performance depends strongly on the choice of the guidance scale and appropriate value significantly improve generation

---

Published in Transactions on Machine Learning Research (02/2026)
quality. Recent empirical studies further demonstrated the importance of the guidance scale in conditional
generation tasks (Dinh et al. 2023 Sadat et al. 2024 2025). However, the theoretical understanding of how
the guidance scale affects the generation remains limited (Chidambaram et al. 2024 Wu et al.) 2024).
In this work, we propose a new geometric guidance model to enable the theoretical analysis of the role of the guidance scale in conditional generation. A key challenge in studying the guidance scale in classifier(- free) models is the analytical complexity of the probabilistic guidance term. To address this, we replace the probabilistic guidance with a new geometric guidance term. Specifically, under the linear manifold hypothesis study the geometric property of the original probabilistic guidance term, building
(Chung et al. 2022). on an idea introduced by Chen et al. (003b)), and construct linear geometric guidance term that plays the same role but more tractable for theoretical analysis.
As a next step, the analysis of the geometric guidance model requires certain regularity conditions on the score function, such as the Lipschitz continuity. However, because of the multi-modality of data distributions, these conditions generally fail to hold (Lee et al., 2022 Gao al. 2025). To overcome this issue, we introduce a mollification technique inspired by mollifiers in mathematical analysis (Evans 2018 to construct surrogate score function that satisfies the required properties for our analysis.
Building on this, we construct well-posed geometric guidance model through which we address two ques- tions: (i) whether the model can recover the target data manifold, and (ii) what is the upper bound on the distance between the generated distribution and the target conditional distribution. Our results reveal the effects of the guidance scale: increasing the scale encourages the generated data to lie closer to the target manifold, and large guidance scales do not significantly increase upper bound on the generation error.
Finally, for the nonlinear case and real-world data distributions, we extend our framework by constructing nonlinear geometric guidance model. This model builds on the same principles as the linear case, with the theoretical foundation obtained by extending the results Chung et al. (2022) to nonlinear data manifolds. Experimentally, we evaluate the nonlincar geometric guidance model on CIFAR-10 (Krizhevsky) 2009 and demonstrate its effectiveness for conditional generation. We also report how performance varies with the guidance scale, providing empirical evidence consistent with the behavior suggested by our linear analysis.
In summary, our contributions are:
1. We construct new linear geometric guidance term to replace the original probabilistic guidance term by studying its geometric property under the linear manifold hypothesis.
2. To ensure the regularity of the unconditional score function, we apply a mollification technique to construct a a surrogate score function, and build a well-posed geometric guidance model.
3. By analyzing the geometric guidance model, we uncover the role of the guidance scale: a large guidance scale encourages the generated data to lie closer the target data manifold and does not significantly affect the upper bound of the generation error.
4. We propose principled nonlinear geometric guidance model and evaluate it on CIFAR-10; the experi- ments demonstrate its effectiveness in conditional generation and illustrate guidance-scale effects beyond the linear setting.
The remainder this paper is organized as follows. Section 2 reviews related work, and Section 3 summarizes the technical background on diffusion models. Section introduucs the construction of the geometric guidance term, and Section presents the theoretical analysis of the geometric guidance model. Section G exetends the model to nonlinear settings and reports experimental results. Section concludes the paper and discusses limitations. Nolation is summarized in Appendix A
2 Related Works
Convergence analysis: A number of recent works have analyzed the convergence properties of diffusion
models under various assumptions (De Bortoli et al.) [2021] Lee et al.] 22022 2023 Chen et al. 2023c a Gao
et al. 2025). De Bortoli et al. (2021) established total variation bounds under C Ce-regllarity assumptions
2

---

Published in Transactions on Machine Learning Research (02/2026)
the score for the target distribution, Chen et al. (2023c) relaxed this requirement to Lipschitz continuity of the score function but for each intermediate density, which was further weakened in Chen et al. (002aa) to the Lipschitz contimuity of the score only for the target density. Using functional inequalities, Lee et al.
(2022 2023 and Gao et al. (2025) have derived convergence guarantoes under the assumption that the target density function is log-concave, with results in both total variation and Wasserstein distances. In contrast, our setting involves multi-modal target distributions for which log-concavity and smoothness assumptions do not hold (Lee al. 2022]. To address this, we introduce a technique that constructs a surrogate distribution satisfying the required regularity properties while closely approximating the original target.
Geometric structure: For real-world datasets, it is widely believed that high-dimensional data lie low-dimensional submanifold of the ambient space, a perspective known as the manifold hypothesis (Bengio ct al. 2013). When generating such data distributions, deep generative models often encounter challenges such as the curse of dimensionality [Bronstein et al. 2021) and manifold overfitting (Loaiza-Ganem et al., 2022). However, the strong empirical performance suggests that diffusion models can avoid these issues. As result, understanding the theoretical behavior of diffusion models under the manifold hypothesis has attracted increasing attention. For example, De Bortoli (2022) established a Wasserstein convergence bound assuriing that the target distribution is supported on compact set. Under the additional assumption that the target data manifold is linear, Oko et al. (2023) showed that diffusion models can avoid the curse of dimensionality by providing a Wasserstein bound that depends only on the intrinsic dimension. Chen et al. (2023b) further derived a total variation bound in lerms of the intrinsic dimension based on a decompasition of the score function under the linear manifold assumption. Following this line of work, we further investigate the geometric structure of this decomposition to clarify the role the score function in recovering the target data manifold, which in turn helps us construct the geometric guidance model.
Conditional generation: To control the generation (Song al. 2021 Dhariwal & Nichol (0211 and Ho
& Salimans 2022) applied the probabilistic guidance term to generale conditional distributions. Following their works and based on the geometric structure of noisy data manifolds under the linear assumption of the target data manifold (Chung et al., 2022), Chung et al. (2022 2024 and He et al. (2024) proposed using a new time-dependent guidance in conditional generation to constrain geometric structure of the generation process. From a different perspective, (Song et al. (2023) and Bansal et al. (2023) constructed a time- independent guidance constructed by a loss function that is designed to enforce desired constraints on the generated data. Instead, our geometric guidance is constructed by studying the geometric property of the probabilistic guidance, with the goal of replacing its role in conditional generation.
To adjust the strength of guidance, Dhariwal & Nichol (0021) and Ho & Salimans 2022 also introduced a guidance scale, and their experiments showed that selecting appropriate scale can significantly improve performance. However, there are limited works on theoretically analyzing the effects of the guidance scale in conditional generation. Chidambaram et al. (2024) studied one-dimensional case and showed that increas- ing the scale not only reduces diversity of generated distributions but also leads generated data to drift to the extreme points in the support of the conditional distrriuttion. Wu al. (2024) theoretically analyzed the influence of the guidance scale in the context of Gaussian mixture models, demonstrating that a large guidance scale diminishes distributional diversity while boosting classification confidence. Due to the ana- lytical complexity of the probability guidance term, previous works have focused on special cases. Therefore, we propose a geometric guidance term that plays the same role as the probabilistic guidance but is more amenable to theoretical analysis of the guidance scale.
3 Background
3.1
Diffusion Model
Let X~P Px E P(RP) denote the target data distribution. The forward process in denoising diffusion probabilistic models (DDPMs) (Ho et al. 2020) is governed by the stochastic differential equation (SDE)
dX,- = - (t) + (B(t)aW, Vte [0,T],
(1)
3

---

Published in Transactions on Machine Learning Research (02/2026)
with the initial condition Xo- Px, where (Wi) is standard Brownian motion and 8: [0, 00) is smooth; see Song et al. (2021b). This SDE admits the following analytical solution:
x, - Xo + -1 arE.
AA E (07),
(2)
where & (O, ID) standard Gaussian, Og exp (- B(s)ds), and " means equal in distribution. The derivation is provided in Appendix B.1
The reverse process of DDPMs aims to generate Px, which corresponds to the time-rrevrssl process of To this end, we need consider the process
XI-t
and study its stochastic dynamics. As shown in Anderson (1982) and Haussmann & Pardoux [1866), the process (xE))t satisfies the following SDE:
dx;- - (H x)x; + 3(T t)V log po dt + 3
(3)
where pt is the density function of X,, and W ()(...))) is the Brownian motion in reverse time. A simplified proof can be found in Tang & Zhao 2024).
In practice, a neural network st(t, )) with parameter trained to estimate the score function V log using the score matching method (Vincent] 2011). By substituting V log with the estimator sø (f.,)) 3 experiments (Song Ermon 2019: Song et al. 2022bb Dhariwal & Nichol 2021 showed that DDPMs achieve state-of-the-ate performance in data generation tasks.
3.2 Probability Flow ODE
Instead of simulating the stochastic process (3) denoising diffusion implicit models (DDIMs) (Song et al. 2021a) employ deterministic approach for generation, which corresponds to the following ordinary differ- ential equation (ODE):
1 di x = -)) + Va loggr- V E [0,7].
(4)
with the initial condition X₁- ~ Pr, which is called the probability flow ODE. The evolution of the density functions of X under this deterministie process equivalent that of the stochastic reverse process the continuity equation associated with the ODE coincides the Bokker-Plandee equation corresponding to the SDE [1 ; see Song et al. 2021bb for details.
In this paper, we focus on the deterministic dynamics, as the Wasserstein distance used as the main metric makes analyzing the ODE formulation more convenient than the SDE. It naturally extends to the SDE via Itō's formula (Gao et al. 2025). Following Chen et al. (2023c) and Chen et al. (2023a), we consider the Censtin-UUlenbeed process by setting (t) = 2 in Equation for simplicity, where this constant choice is unimportant, as varying it merely rescales time.
3.3 Conditional Diffusion Model
When working with paired data (X,Y) Pxy, the goal of conditional generation is to generate the condi- tional distribution I' (- Y). In Song et al. (2021b), diffusion models are directly applied to Fx Y). is first run with the initial condition Xo Px Y) to obtain the density functions P of X,. Then, the stochastic reverse process (3) or the deterministic process is simulated to generate samples from P YY).
Specifically, the forward process
Moreover, these intermediate densities P admit more explicit expressions. Suppose (X,Y) ~ Pxy and we run the SDE with initial condition Xo ~ Px = fFr
obtain X,. Let pe (r y) denote the joint
density function of (Xe, Y). Then, it can be shown that
pe(3 v) = r. (xx);
4

---

Published in Transactions on Machine Learning Research (02/2026)
see Appendix B.2 for details.
Therefore, the score function for generating (x (-1 Y) can be decomposed as
v (ogg vn log v) = (:)) + V log x),
(5)
where Pa( is the marginal density of X, obtained by running ① with the initial condition X Px- This term can be estimated using standard methods from unconditional DDPMs. The remaining term, v log Pxy æ), is known as the guidance term, and there are two main approaches for approximating it: classifier guidance and classifier free guidance (Dhariwal & Nichol 2021 Ho Salimans 2022). In classifier guidance, a time-dependent classifier is trained to approximate (y on all noisy data. In classifier-free guidance, a new neural network st æ; x, trained to estimate the conditional score V lo (a) y), while so(1, x. approximates the unconditional score V log pa (xx). The guidance term is then computed as
V log a)) x sn(1, 2, 2,, v) - so(1, ax, 0).
In practice, a scaling parameter 7) > O, known as the guidance scale, is typically introduced to control the strength of the guidance term (Dhariwal & Nichol 2021). When using the deterministic dynamics this modification is mathematically expressed as
d = xt + Ve logpT- + 1/ log IT), VtE [0,T], x
x
(6)
di
with the initial condition X (r(-11)1)
As mentioned in Section 2 although setting 7/ * 1 may seem counterintuitive from a theoretical perspective,
empirical studies (Dhariwal Nichol 2021 Ho & Salimans 2022) have shown that selecting an appropriate value of 17 can significantly improve performance. In particular, increasing the guidance scale v enhances the distinguushaablitty of generated samples, but at the cost of reduced diversity (Ho & Salimans) 2022;
2024). However, theoretical understanding of how the guidance scale y influences generation remains limited, due to the analytical complexity of the guidance term V log (r x))
Chidambaram et al.
2024 Wu et al.
(Cidambaram et al.
2024 Wu et al. 2024h.
Therefore, the main objective of this work is to provide a theoretical analysis of the guidance scale 7, under the assumption that the target data concentrate on low-dimensional linear subspace My C RP, called the target data manifold, i.c., su Y Y y) c My- This analysis consists of two main steps:
(i) First, we replace the probabilistic guidance term with a geometric guidance term in order to avoid the difficulty of handling V logg (r x) (see Section
(ii) Second, we analyze the modified dynamics under the geometric guidance from two perspectives: (a) how 7) influences the recovery of the target data manifold, and (b) how it affects the distance between the generated distribution and the target distribution (see Section 5
A central technical challenge in analyzing the geometric guidance dynamics is that V po) may fail to satisfy desirable properties, such as the I-Lipschitz continuity (and the log-concavity of (x)), due the fact that (x) arises from a diffusion process initialized with multi-modal distribution (Lee et al., 2022 Gao et. al. 2025). To address this issue, we introduce novel technique inspired by mollification in mathematical analysis (Evans 2018), which yields surrogate distribution pf (æ) for which the geometric guidance dynamics is well-posed.
4 Geometric Guidance Model
In this section, our main objective is to construct a new guidance term to replace V log x)) in Equation from a geometric perspective. Specifically, the key idea is to understand the role that Va log x) plays in recovering the target data manifold My.
Note that Va pr | 2)) appears as a component of Va log p (x) by Equation 5 This motivates us investigate the geometric interpretation of the score function Va log Pe in the setting of unconditional
5

---

Published in Transactions on Machine Learning Research (02/2026)
and basic property of Po(y | x), then propose replacement for v log Pc k),, which preserves its geometric role but is more tractable for theoretical analysis (see Section 4.2
DDPMs (see Section 4.1 Based on Equation G
4.1 Geometric Interpretation of Score Function
To study the geometric properties of the score function Va log):(a:), we first examine the geometric structure of the noisy data manifolds that arisc during the DDPM process. Chung et al. (2022) showed that, under the assumption that. the largel data lie on M RD, a linear subspace of the ambient. space TRP with significantly lower dimension, the noisy data X₁ concentrate on hypersurface, i.e, (D 1)-dimennionall manifold embedded in RD, for any t > O. We generalize this result (Chung et al. 2022 Proposition 1) in the following proposition; the proof is provided in Appendix C.1
Proposition 1. Assume ~ pd on R and X = AZ ~ Px on R for an E ODxd, i.c., AE A Dxd and A la. Define
M fo E R: (LL AA =
where r(t):= : V(D:= d) a4) and e-21 Let Xx be generated by the DDPM forward process with the initial condition Xo = X. If < D then X, concentrales on M with high probability.
Based on this result, the next question is how the score function V (o)) contributes to recovering these noisy data manifolds M during the reverse process
Under the same assumptions as those in Proposition Chen et al. (023b)) showed that
V lo A v logy (z) ==
at (ID
AAT )a,
(7)
where pf the density associated with the forward process ① initialized from p². An alternative derivation of this formula, along with an analysis of its geometric properties, is provided in Appendix C.2
Based on this orthogonal decomposition, we observe that the role of Va (e)) can be understood two components: (i) the first term serves generating the distribution pZ in the latent space, and (ii) the second term controls the reconstruction of the noisy data manifolds M in the ambient space. Informally, this decomposition can be summarized as
V log (xx) = Generate Latent Distribution + Recover Data Manifolds M
We formalize this intuition in the following theorem; see the proof in Appendix C.1 Thcorem 2. Under the same setting as that in Proposition
let x = AA X; and X = X,- X
where Xi -t-
(a) Let X AZi with Z A X Then Zi satisfies
d z z
log logoo (Z;))
dt
which implies that Z. = AT X, = ZF_, follows the forward process
initialized from p².
(b) X satisfies
d
x X dt
ar- X
Moreover, X 1o r(T to) implies
| 1 = r(T to 5), where r(t) V(D d)(1 ag)
In Theorem statement (a) demonstrates that the parallel part Vz (zg in the decomposition is responsible for generating the target latent distribution p² via the reverse process of DDPMs, which has been thoroughly studied in Chen et al. (2023b). Mcanwhile, statement (b) shows that, since
[[T AAT)X:
||
the orthogonal part (ID AAT)a j plays a key role guiding the recovery of the noisy data manifolds M which provides insight for designing geometric guidance in conditional generation.

---

Published in Transactions on Machine Learning Research (02/2026)
4.2 Geometric Guidance for Conditional Generation
Let us return to the conditional diffusion model. To apply the results from Section 4.1 in studying the role of V Po æ) in guidance, we first impose the linear assumption for the target data manifold.
We consider a two-class dataset (X, Y) ~ Pxy on RD x {1, 2} for simplicity; the following analysis readily extends to the multi-class case. Let P(Y = 1) = wi and P(Y = 2) = w2 so that
Px w1 xIy Y = 1) + w2l x Y = 2).
The linear assumption states as follows. Assumption For i 1,2, there ezists a Zi ~ p. on Rd and an A, E ODxd, such that
X; = A,2
and we further assume A A =.
Remark 1. For this assumption, we provide two remarks.
(i) basically means that the support supp Px Y) M, == Im A,, the image x + Ax; in other words; P Px Y =) is supported on the linear space Im A,. The definition of the support of a probability measure is provided in Appendix
(ii A A2 indicates MM M2. This orthogonality assumption is introduced to simplify the sub- sequent analysis, but it does not significantly affect our conclusions regarding the guidance scale; see Appendix E.I for further discussion.
Next, we fix Y = and our goal is to generate the conditional distribution, which needs to consider the geometric structure the condition score function Va log. y =y=1), 1). By combining the results Section 4.1 with Equation (5), the conditional score function has two different types of decomposition:
V (x y== = = Generate Latent Distribution + Recover Data Manifolds Mi
(8)
V log v log = ())
We will show that Væ log æ)) plays the role of recovering the data manifolds Mi with respect to the first decomposition.
For the first decomposition in S based on Assumption and Proposition I generated by the forward process starting from Mi are given by
because the noisy data manifolds
M = E R:: [| - - = r(t)]
(9)
the orthoglnal part of V. lo (x
1) in the first decomposition responsible for recovering M is parallel to (ID A] Jæ shown in Section 4.1
Intuitively, for the second decomposition in since Pe(y = 1 x) acts as classifier for Mi, we have (y = x:) ~ for any x E M,, i.e., log (y yy = 1 x)) is approximately constant on MM. Therefore, by x) almost normal to M
Lemma 1.1
V log
V lo = x)) - A,A] )æ, for some > 0,
because (ID-A14) ja is normal Mt by Lemma 1.1 Rigorous details are provided in Appendix C.3
Therefore, the guidance term log. = x) partially contributes to the recovery of the data manifolds Mi during the reverse process. Consequently, it can be replaced by (Tn A₁ A Ja. we propose the following geometric guidance model for conditional generation:
Jm). Based this insight,
d X + Va log - n
X = dt
P A,AT
(10)
7

---

Published in Transactions on Machine Learning Research (02/2026)
5 Main Results: Analysis of Geometric Guidance Model
In this section, analyze the geometric guidance model 10 with the aim of uncovering the role of the guidance scale 7)- To understand its effects, consider two related questions: whether the model can approximately estimate the target data manifold MM (see Section 5.2), and how to quantify the distance
between the generated and target distributions (sce Section 5.3). These two problems serve as a lens through
which we investigate the influence of r in conditional generation.
Before addressing these two questions, it is necessary to ensure the well-posedness of the ODE [T); that is, we must establish regularities of V log P(()) such its Lipschitz continuity and the log-coneavity of pa(x), which requires careful analysis (see Section 5.1 because it is obtained from multi-modal distribution Px.
5.1 Well-posedness of Geometric Guidance Model
In general, the Lipschitz continuity of Var (og)((x) and the log-concavity of p<(2:) induced by the DDPM forward process depend on properties of the initial distribution μ.. basic requirement is that μ admit a density p(x).. Log-concavity of p(æ) then implics log-concane cog-coitavity of (x()) (Gao al. 2025), and Lipschitz continuity of V ((x) implies the Lipschitz continuity of V (o()) (Chen et al. 2023a
However, in our setting, it is clear that Px does not admit a density function. We therefore first deduce the necessary conditions on the latent distribution implied by Px; see Sections 5.1.1 and 5.1.2 Second, the multi- multi-modality of Pxx introduces irregularities in Pr(x)) (Lee et al. 2022). which we discuss in Section 5.1.3
By solving those two problems, we construct a surrogate pl (x) for use the gcometric guidance model 10 which is well-posed; see Section 5.1.4
5.1.1 Problems in Latent Distribution
When j does not admit a density function for instance, when the support of p lies a lower-dimensional manifold in the ambient space De Bortoli (2022) showed that the score function Va log is Lipschitz continuous under the assumption that supp μ is compact, i.e. closed and bounded. This setting aligns with our problem but guarantees only Lipschitz continuity. In contrast, we establish a stronger result in the following Proposition 3 which does not require the compactness, under the assumption that the target data manifold is linear. The proof is provided in Appendix D.1]
Proposition 3. Let Z be random variable on R with the density function p², and let B E Rank Assume there are mn, A > such that
V logp logg=(()) molk. |B SA,
and A= Amin (B (BTB) B) 2 0, the minimum of all eigenvalues of BT B. For a E R and 3 > 0, let
X =BBBZ BE, & Ia,
with the density function px on R", We have
aAA Be + 32)
v2 loggx(x))l L, L L:= 32 +
Remark 2. A direct application of this proposition is that it extends the result of De Bortoli 2022 to non-compact setting, under the additional assumption that the latent distribution is strongly log-concave, i.e., - (2) ž malk- If we are only concerned with the L-smoothness of logpx, the log-concavily of p² can be relaxed to the L-smoothness; see Appendix for details.
Corollary 4. Using the same notations as in Proposition we have
a²A A log 3
) In.
3
1 smsmoothness f and I=Lipechita continuity Vf are equivalent C2 functions; use them interchangeably.
8

---

Published in Transactions on Machine Learning Research (02/2026)
Remark 3. Note that, if a < a²A + mo such A X, then
(1
1
a²A
logpx(#)) z mglas My
> 0,
8
a² + mo,82
which implies that px is mo-strongly log-concave. Therefore, it shows that the strong log-concavity of p² ensures not only the L-smoothness, but also the concavity of logpx.
Based on results in Proposition score function can still be guaranteed, provided that the latent distribution admits density and satisfies strong log.concavity. However, in our setting, these two conditions are not satisfied:
g even if Px does not admit a density function, the desired properties of the
(i) For the latent distribution of Ixx because Z; ~ P on RR,,, we first lift them on Rd with d == di + d2 by defining
ž = (Iz, Odixdd Z1 ~ P77 D2 = (Odxx Iaz) Z2 ~ P2
Let A = (A1, Oxd, It follows that
A&. = A; ~ Pxpr 1 =i,, i.e. A+ = P Y =).
Therefore, by Lemma H.7 if Z ~ p2 == : w + W2222 we have
AZ Px wi Y 1) + w2P Y 2).
But the problem is that the latent distribution 1 does not admit density function on R
(ii) For log-concavity, even if the latent distribution admits a density function, it typically does not satisfy strong log-concavity due to its multi-modality (Lee et al. 2022
Therefore, in the following, we first introduce technique to address the log-concavity of the latent density (Sections 5.1.2 and 5.1.3,, and then apply Proposition 3 to establish the desired properties of the score
function (Section 5.1.4).
5.1.2 Mollification Technique
Mollification (Evans 2018) is a standard technique in mathematical analysis to address non-smoothness of functions. When dealing with a non-smooth function f, the idea is to find smooth kernel function & such that the convolution := f * k which is clearly smooth, is closed to f.
Following this idea, we choose a Gaussian distribution of with some a as the kernel, and consider its convolution with μ2, see Remark B.1 for the definition of convolution between measures. Let
p = p2 * NN0 La) =w + wyP2.
where P.Z, := ?07 * of Note that both Pz and p? admit density functions, denoted by p and pŘ. respectively, and
pe = wipL + w2p2...
(11)
Moreover, by the definition of convolution, if Z p,, then
z Zi (Zi,0) + oci pf.. Zz := (0, + P2,07
(12)
for Ci NO, Ia) independent of Z, Therefore, we obtain smooth density 1h the latent space Rd.
Next, for Px, the following Proposition 5 addresses the question of whether sampling from pe yields a P == Ag P2 that is close Px- The proof is provided in Appendix D.1
To measure the distance between probability measures, we use the 1-Wasserstein distance in this work for analytical convenience. For u.v& P(RP), it is defined by
((,,)) = inf
ylldr(a, 9): 1
E T(u,)) v) = inf [E Y X n4 Y N~
9

---

Published in Transactions on Machine Learning Research (02/2026)
where V), == {YE x : (A R ((A),, 7BP x B) = =B{)); see Chewi al. (2024) for more details.
Proposition 5. Using the above nolation, if o ~ P/z then for X = =AZ ~ Px: we have
Px = == Y 1) + w2 x = = 2).
where P x (.- = A for = 1,2, and it follows that
Wi (x),,, S &ovd.
Therefore, the mollification technique provides smooth latent density function p that induces a distribution P approximating xx.
5.1.3 Log-Concavity of Latent Density
In general, even if p is smooth, cannot directly assume that it is strongly log-concave, as it is multi-modal by Equation [1]]. However, we can still assunne that each of its components P is strongly log-concave, which, in fact, follows from the assumption of strong log-concavity of the original latent density p.¹ Assumption II. Let vf be the densily funclion of IP? defined on Rd4, There evils large > 1 such that -V logp? =mla;
i.e,, p. mn-strongly log-coneave for 1,2.
Assumption II ensures the strong log-concavity of cach component ph,,, but does not guarantce that the overall mixture p is strongly log-concave- this common difficulty in the case of multi-modal distribu- tions. However, due to the mollification construction, the parameter a can be freely chosen, which enables establish the strong log-concavity of p under the following assumption.
Assumption III. For chosen o, we assume that
M = = V= log (og V log 1og.2. (z)1l < 2/m
Remark 4. This assumption novel but essential for addressing log-concavity multi-modal settings. Characterizing the classes of pe that satisfy is nontrivial. As a concrete example, each pf (2) is a Gaussian truncated to compact, convex set, then compactness implics that the difference of V Vlogg?. is uniformly bounded by a quantity depending on Therefore, with an appropriate choice of a, Assumption holds. The details, with a sufficient condition for Assumption III are discussed in Appendix E.2
Assumption
required to obtain upper bound V log p, even when the density p is multi-modal, I
shown in the following lemma; see the proof in Appendix D.1
Lemma 6. Let P1,Pa P2 be two probability density functions on R such that 2 log p: I L for some constant L, E IR. Suppose that
sup V lo - V Vlogp2(x)) & M < oc.
Then, for the mixture density p wpi + (1 w) with w E (0, 1), it holds that
2 log V max (T,, T2) + 1/2 I,..
By Lemma G and Proposition the strong log-concavity the multi-modal latent density function p can be guaranteed.
Theorem 7. Under Assumptions 11 and for pz = wiPf. + w2ph.a. i.e.,
III
if o² (4m M²), then po is strongly log-concave
m = 4m - -M + mo²)
v3 logg? 7 (x) 2 mjla.
4(1 mo²)
10

---

Published in Transactions on Machine Learning Research (02/2026)
Proof. Note that
= = (Ta; (Id;: Z + ooi Pi,o
2
By Assumption II
and Corollary
with the choices (Ia1, 0) mg m, 1, and B = a, we obtain
1
1
log 3 o²2(1 + mo²)
I
0²
For pá,a. we similarly have
2 log 22, (z) 1 02(1 + mo?)
(2)
Ia.
o²
Then, because p = =wiph. + w2p2.. it follows from Assumption
III
and Lemma 6 that
v (2)
1 0² (1 mo?)
M2 7
Ia= --mpa
02 +
5.1.4 Smoothness and Concavity
Before proceeding, let us recall that the latent distribution μ22 of Px is not "good". To address this, we construct new distribution P whose latent distribution P admits "good" density function p and which is close to Px. Consequently, instead of considering the score function associated with DDPM initialized from Px, we consider a DDPM initialized from P i.e.,
X = AZ, + -V-afk
(13)
where Zo ~ Ph and & ~ NN(,, ID). We then modify the dynamics in 10 to define our final version of the geometric guidance model: Definition 1 (Geometric Guidance Model). For any E [0,T 8],
X, = X, + Va log log n1p - ~ ~(0,
dt
where P1 = 11= A
Remark 5. (1) The initial condition is taken as (NO Ip) instead of pr(- y) to reflect practical implementation
settings. (ii) The time interval is chosen as [0, T 6] for some ö to avoid the singularity at time T.
Therefore, our main objective now becomes establishing the Lipschitz continuity of Va log pt (a) and the log-concavity of p8 (z), which follows from the strong log-concavity of the latent density p" (z).
Theorem 8. Under Assumption 4 and the same settings as in Theorem 7 for the density function pf defined
in Equation [3). have
20 + (1 - 0z)
logni (x) S Lt, L ==
(1 a-) (ar + (1 a mm)
and
m
V logpi (x) m,ID. mi at + at m
Proof. First, by Theorem
7 the latent density pe is mā-strongly log-concave. By the definition of pf
Proposition 3 implies that
2a, + (1 -1- or )m
2 log (x))lop < (1 a1) (ar, + (1 a.) m))
with the choices B = A, ma m.. Var, and 8 vI ac. This follows from the fact that AT A = L (Assumption Dh which indicates ||A|1|, and Amia (A A) = 1.
For the log.concavity, Corollary directly yields
1 11 (1- (1-
a, + =11 ar m )) ID.
22 log po (x: 2
11

---

Published in Transactions on Machine Learning Research (02/2026)
Therefore, we have established the desired properties of pl. which ensure the well-posedness of the geometric
guidance model I Moreover, from the definition of in Theorem g
we can derive lower bound that
will be useful in the following analysis; see Appendix D.1 for the proof.
Corollary There erists a small o such that my and my == inf my > 1.
5.2 Estimating Target Data Manifold
For the geometric guidance model the first problem is whether it can estimate the target data manifold Mi. Specifically, we aim to show that the generated sample X approximately lies in Mr. Since Mi = Im A1 is linear subspace by Assumption it suffices examine whether E [[] Yr-o|]] 0, where
Y, P,X,, Pi In AIA
Multiplying both sides of Equation by Pi. we obtain that Y, satisfics the following dynamics:
dt Y. Y P.Va logpl- nY, Yo N1),
(14)
for L E [0.T - 5]. By analyzing the dynamics 14 the following theorem provides a convergence rate of E Y + 0 with respect to the guidance scale 17.
Theorem 10. Consider the dynamics
14 under Assumptions and III Then,
E [ [ll] s o (
In porticular, for any 0, by choosing 7 ((max (18), 1/e)), E [[ <e. Proof sketch. We provide sketch of the proof here; the full proof is given in Appendix D.2 The key idea is to derive differential inequality for EE [[] [[]]] First, have
E s (1 [L-n)EB]])
[|] E V log lo (X.]ll.
(15)
To bound E [|| log pf Xx) ]]]] the smooohhhnsss of log is required, which follows from Theorem
The smoothness implies that
Vx lo S Ls X, +
for some constants Lg and C. Therefore, sulfices to bound E X, By deriving a differential inequality
we obtain E [ < M, for some from Equation (+) and applying Grönwall's inequality (Lemmna H.1l
constant Mr. and thus E [|| log (X.))l < M2. Substituting this bound into 15) and applying
Grönwall's incquality once more yiclds the desired result.
Remark 6. For this theorem, provide two remarks.
(i) Note that this result depends only on the L,-smoothness of log and not on strong log-concavity, Therefore, Assumptions I and III can be relaxed; see further discussion in Appendix
iii) The universal guidance model,
dx;-
X Va log -X- -
dt
was proposed by Bansal et al. 2023 to control the generation process such that the generated images match the prompt g( X c. In their setting, f(x) = ((c,g(x)) for some loss function f. A similar idea used in the proof of Theorem I0 can be extended to theoretically analyze the universal guidance and f is strongly convex,
model. If the L-smoothness of log holds (sce Appendix
[[((F)) + min f. as 77 - oo;
Appendix
D.3 for more details.
12

---

Published in Transactions on Machine Learning Research (02/2026)
Theorem 10 shows that the geometric guidance model can approximate the target data manifold. Specifically, as the guidance scale yf increases, the generated data increasingly lie close to the target manifold. This result is consistent with empirical observations on both synthetic datasets (Wu et al.
2024
Chidambaram et al, as well as with the
2024) and real-world datasets (Dhariwal & Nichol 2021 Sadat et al.; 2024
2025
theoretical results in the one-dimensional case studied by Chidambaram et al.
(2024).
which demonstrate
that increasing o causes the generated data to move toward the extreme points the support of the target conditional distribution.
5.3 Distance to Target Distribution
Let Pt be the density function of X₂ in the geometric guidance model The second question is how to mea- sure the 1-Wasserstein distance between the generated density pr-6 and the target conditional distribution (x 1). Specifically, the goal is provide an upper bound W1 (fr-s, Px Y =))-
First, require an additional assumption: the boundedness of the first moment of each condit distri- bution (x i), i), which can be reduced to the same condition on the latent distribution p.
Assumption IV. For fori=1, 1,2 and Zi p m == E [1 2 co. Theorem 11. Under Assumptions 1 III and
IV
obtain that
Wi ( = 1)) s 0(e- 5 +o+ +C
for some constant Č.
Proof sketch. The proof consists of two main steps:
(i) Let Qı = AiA be the orthogonal projection onto Mi Im A1. By Theorem 10 we have
Wi (xr-s,Pxxx(( Y= = s Wi ((Q)) Px ===11 =1) + 0 0(= + +
(ii) For Wi ((Q))#9T- Y = 1)), it has
Wi ((Q))
=)) S W/ (fT- Wi (x IY = =)),
where the first term Wi (fr-s, Px) can be bounded by comparing the geometric guidance model with the unconditional reverse dynamics, and the second term is directly bounded by Lemma D.3
The full proof is provided in Appendix D.4
Remark 7. Since geometric guidance cannot carry as much information as probabilistic guidance due to its analytical simplicity, the error floor Č' does not vanish 17 1, unlike in probabilistic guidance models; see further discussion in Remark D.1
This result suggests that increasing the guidance scale does not harm the generating performance, which may appear counterintuitive and inconsistent with empirical observations. In practice, however, ODE dynamics are typically approximated using the Euler discretization (or the Euler-Maruyama scheme for SDEs), which introduces additional discret ization error. In our setting, the Fuler discretization error for the geometric guidance model (=) is bounded by O(h)?), where h denotes the step size; see Appendix D.5 for details. Therefore, the performance dogradation observed at large guidance scales ariscs not from the model formu- lation itself, but from the discretization algorithm. For example, Wu et al. (2024 Figure 3) showed that the large guidance scale would harm the modality of the original data, but this problem can be mitigated by reducing the discretization step size.
6 Nonlinear Extension
In this section, our main objective construct nonlinear geometric guidance model suitable for real-world image datasets, and to evaluate its generation performance under varying guidance scales 7)-
13

---

Published in Transactions on Machine Learning Research (02/2026)
The first challenge is to construct the geometric guidance term for image datasets, which may not lie in linear subspace. To this end, we study the geometric structure of noisy data manifolds without assumiig linearity of the target data manifold, by extending the result of Proposition
[ to the nonlinear case (see
Section 6.1 Then, following the idea Ross et al., (2024), we train functions Fg. RD - R to model noisy data manilolds via M (0) so that Ver can replace (TD - AA AAT)a ) to be the nonlinear geometric guidance term (scc Section 6.2 Finally, we examine this nonlinear geometric guidance model on CIFAR-10
Krizhevsky 2009), and evaluate its performance under the different guidance scale (see Section 6.3
6.1 Noisy Data Manifolds for Nonlinear Case
The geometric guidance term (ID >AT)x is constructed based on the result in Proposition which assumes that the larget data manifold M Im A is linear. However, for real-world image datasets, it may unrealistic to assume that the data lie in linear subspace. Instead, it is more reasonable to assumee that the target image data lic on nonlincar manifold M C RD with intrinsic dimension d < D; see Appendix for basic knowledge of manifolds. This assumption is known as the manifold hypothesis (Bengio et al., 2013), and it has been supported by both theoretical analyses (Fefferman et al. 2016) and empirical studies (Brown et al. 2022
Coaiza-Ganeni et al. 2022).
To construct a new geometric guidance term, because of the nonlinearity of M, we must extend the result of Proposition 1 to uncover the geometric structure of noisy data manifolds. Although the d-dimensional additionally require that it is locally isometric to IRd.
manifold M C R is not assuned to be linear,
More precisely, we assume the existence of a C function o: R - R² such that Im ø = -M and & is an isometry; that is, J J = Ia- Then, by Lemma 12 we obtain an analogue of Proposition ! in Theorem 13 which shows that the noisy data manifolds M are hypersurfaces- -ic. (D 1DDD-1--dmmmon 1)-dimensional submanifolds of RR;; see the proofs in Appendix G.1
Lemma 12. Let Φ: Rdd +R RDD be a C isomelry such that M Im C RP is a d-dimensiondl submanifold. Then, there erists a C function 6² RD + IRd such that 6² o idga and
J ( JZ(2) VzERd.
Remark In fact, the isometry of & implies that Im Φ is a submanifold, because it is proper (i.e., the preimage of every compact sot is compact) by the Hopf-Rinow Rinow theorem (Jost) 2008
Theorem 13. Let M C DR. be d-dimensional submanifold as defined in Lemma 12 and let Px on R such that supp C M. Let X, be generated by DDPM ( initialized from Px. If d < D, then X, concentrates on hypersurface M CRD with high probability, where
M = (x: ((x)=r(1))
r(1) = V(D - -d)(1 an),
for some C function ft: R + R.
6.2 Learning Geometric Guidance
For an image dataset (X, Y)= Pxy with class label Y E (1.2....,K) K}. we adopt the union of manifold hypothesis (Brown al. 2022 that is,
supp P x Y y)
where My C. R is a dy dimensiiio submanifold. To apply Theorem
13 we further assume that, for each My, there exists an isometry dy IRdv + RD such that Im o, My Then, the noisy data manifolds generated by the forward process initialized from My are given by
M = fa ER. f(()) = r()), r(1) = V(D -(D- 1 d)(1 -(,, ax),
for some function f _ R.
By adopting the same idea as Scction 4.2 for My the guidance term Va log ( (x) is approximately normal to M at that is, it is approximately parallel to (ar)( Therefore, construct the nonlinear
14

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 1: Images generated by GeGM on CIFAR-10
geometric guidance term as (x) to replace the probabilistic guidance V Po | x) in the reverse process for conditional generation. The resulting nonlinear geometric guidance model (in deterministic form) is defined by
- x x = X (og) -1 VN
(16)
where V lo is the score function of the unconditional DDPM initialized from Px.
To implement the nonlinear geometric guidance model, must estimate both the score function and the nonlincar geometric guidance term. The score function Va log (ax) can be estimated using an unconditional
diffusion model specifically, by training a network (1 x) via the score matching method (Vincent 2011 on the unconditional data X. The main task, then, is to estimate (x,)
First, Thcorem 10 shows that MG = (ff)) so such function fy is called a manifold-defning function in Ross et al. 2024 Following a similar idea, we train a network Ft. RDD + R to estimate J - rt)), so y.0
F needs to satisty
(x) 0, and {
#0, YIEM,
where the first condition follows directly from the definition of M and the second condition, called the rank condition, ensures y.0 a manifold-definnng function, as guaranteed by the Constant Rank Theorom (Lemma I.2h. Therefore, the loss function for training y, is designed as
(0) == Ex~m((1a)
- - Va 0.0 X)) |3]
(17)
where K > 0 is chosen for controlling the strength of the rank condition. We simply set 1.
6.3 Experiments
Effectiveness of GeGM. We use the Fréchet Inception Distance (FID) (Heusel et al. 2017) the metric for evaluating generation performance, because it can be regarded as practical surrogate for the Wasserstein distance. We compare the FID of samples generated by the nonlinear geometric guidance model [66 (GeGM) with those generated by the classifier guidance model (CGM) c The results are reported in Tabl where we present results for selected classes; the remaining classes are provided in Appendix G.2 Note that the
guidance scales used for CGM and GeGM differ, since the norms of the probabilistic and geometric guidance terms are not comparable. For visualization, Figure [ displays images generated by the nonlinear GeGM. These results demonstrate the offectiveness of the nonlincar GeGM in generating worl-world images.
Performance vs. guidance scale. By applying the noulinear GeGM 16 we evaluate how generation performance varies with the guidance scale n on selected classes from CIFAR-10; results for the remaining classes are provided in Appendix G.2 As shown in Figure 2 performance improves with increasing n within
15

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1:
Comparison of FID on CIFAR-10
Automobile
Frog
Horse
Ship
Truck
CGM (1) = 1)
13.46
17.87
13.97
11.61
16.85
GeGM (n = 50)
9.70
16.28
12.65
13.84
11.02
100
sutamobile frag
2h10
80
FID
60
40
20
5
10
50
100
n
Figure 2: FID v.s. guidance scale of GeGM on selected classes of CIFAR-10
a reasonable range. Since FID serves as a practical approximation of the Wasserstein distance, this trend is consistent with Theorem even the nonlinear setting.
Remark 9. We emphasize that the observed trends are consistent with the spirit of Theorem in nonlinear regimes, but they are not derived from it. Establishing nonlinear analogues of Theorems 10 will require additional analysis and left for future work.
7 Conclusion
In this work, we studied the role of the guidance scale in conditional generation with diffusion models. To address the analytical intractability of the probabilistic guidance term, we introduced a geometric guidance model that enables theoretical analysis under the linear manifold hypothesis. To facilitate this analysis, we proposed mollification technique to ensure the regularity the score function in the presence of multi- modality. Our results showed that increasing the guidance scale within a reasonable range can enhance generation performance, in line with empirical observations reported in prior studies. We further extended the model to nonlinear scttings, and experiments real-world datasets demonstrated the effectiveness of the geometric guidance model and provided additional evidence consistent with our theoretical findings.
Limitations: While the geometric guidance offers a more tractable alternative to probabilistic guidance, it comes with certain limitations. Notably, our analysis showed that the upper bound of the Wasserstein distance between the generated and target conditional distributions bounded by a constant, regardless of the choice of the guidance scale. This implics that, unlike probabilistic guidance, which can approximato the target conditional distribution by setting the scale to 1, the geometric guidance does not guaranlee convergence to the target distribution. This is trade-off made for the sake of analytical tractability.
Although our experiments on the nonlincar extension partially supported the thcorctical results, our current theoretical analysis restricted to the linear manifold setting. In the nonlinear case, the geometric structure of the score function remains unclear. Regarding regularity the score function, while Lipschitz continuity can be ensured under compactness assumptions, extending this the non-compact setting remains an open problem. Furthermore, the log-concavity of the score function cannot be guaranteed, even in compact nonlinear cases.
16

---

Published in Transactions on Machine Learning Research (02/2026)
Acknowledgments
We thank Ming Li and Luheng Wang for the helpful discussions. ZZ was supported by Institute for AI and Beyond at the University of Tokyo. MS was supported by JST ASPIRE Grant Number JPMJAP25B1, The authors also thank the anonymous reviewers for their careful reviews and insightful comments, which have been invaluable in improving both the clarity and rigor of this work.
References
Brian D. O. Anderson. Reverso-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313 -26,, 1982.
Dominique Bakry, Ivan Gentil, and Michel Ledoux. Analysis and Geometry of Markov Diffusion Operators, volume 348. Springer Science & Business Media, 2013.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pallern Recognilion, pp. 843 852, 2023.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspec- tives. IEEE Transactions on Pattern Anolysis and Machine Intelligence, 35(8):1798- 1828, 2013.
Sergey G. Bobkov and Michel Ledoux. From brunn minnnnnki to brascamp- lieb and to logarithmic sobolev inequalities. Geometric and Functional Analysis, 10:1028- 1052, 2000.
Nicolas Boumal. An introduction to optimization on smooth manifolds. Cambridge University Press, 2023.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104, 1344778 2021.
Bradley C. A. Brown, Authony L. Caterini, Brendan Leigh Ross, Jesse C. Cresswell, and Gabriel Loaiza- Ganem. Verifying the union of manifolds hypothosis for image data. In International Conference Learning Represendations, 2022.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User- friendly bounds under minimal smoothness assumptions. International Conference on Machine Learn- ing, pp. 4735 4763. PMLR, 2023a.
Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation, and dis- tribution recovery of diffusion models on low-dimensional data. International Conference on Machine Learning, pp. 4672-4712. PMLR, 2023b.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is easy learning the score: Theory for diffusion models with minimal data assumptions. In International Conference on Learning Representations, 2023c.
Sinho Chowi, Jonathan Niles-Weod, Weod, and Philippe Rigollet. Statistical optimal transport. orXiv preprint arXiv:2407.11163, 2024.
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. What does guidance do? a fine- grained analysis in a simple setting. In Advances in Neural Information Processing Systems, 2024.
Hyungjn Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Newral Information Processing Systems, 2022.
Hyungjn Chung, Suhyeon Lee, and Jong Chul Ye. Decomposed diffusion sampler for accelerating largo-scale inverse problems. In International Conference on Learning Representations, 2024.
17

---

Published in Transactions on Machine Learning Research (02/2026)
Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. Transactions on Machine Learning Research, 2022.
Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications score-based generative modeling. Advances Neural Information Processing Systems, 34: 17695 17709, 2021.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems, pp. 8780-8794, 2021.
Anh-Dung Dinh, Daochang Liu, and Chang Xu. Rethinking conditional diffusion sampling with progressive guidance. Advances in Neurol Information Processing Systems, 36:42285 42297, 2023.
Lawrence C. Evans. Measure Theory and Fine Properties of Functions. Routledge, 2018.
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4):983-1049, 2016.
Gerald B. Folland. A Course in Abstract Harmonic Analysis. CRC Press, 2016.
Xuefeng Gao, Hoang M. Nguyen, and Lingjiong Zhu. Wasserstein convergence guarantoes for a general class score-based generative models. Journal of Machine Learning Research, 26(43):1-54, 2025.
David Francis Griffiths and Desmond J Higham. Numerical methods for ordinary differential equalions: initial value problems, volume 5. Springer, 2010.
UTrich G. Haussmann and Etienne Pardoux. Time reversal of diffusions. The Annals of Probabilily, pp- 1188 1205, 1986.
Yutong He, Naoki Murata, Chich-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion. In International Conference on Learning Representations, 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to local nash equilibrium, Advances in Neural Infor- mation Processing Systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint rrXiv22007.15566 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:68440-881,, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffussin models. Advances in Neural Information Processing Systems, 35:8633 86466, 2022.
Jürgen Jost. Riemannian Geometry and Geometric Analysis, volume 42005. Springer, 2008
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. DiffWave: A versatile diffusion model for audio synthesis. In International Conference Learning Representations, 2021.
Alex Krizhevsky. Learning multiple layers of features from tiny images. https://www. cs. toronto. edu/ +triz/learming-eaauue featinesseeeeue---eeeeee 2009---00--TRR.. 2009. Technical Report, University of Toronto
Beatrice Laurent and Pascal Massart. Adaptive estimation of quadratic functional by model sclection. Annals of Statistics, pp. 1038- 2000.
Jean-François Le Gall. Brownian Motion, Marlingales, and Stochaslic Calculus, volume 274. Springer, 2016.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-basod generative modeling with polynomial complexity. Advonces in Neural Informalion Processing Systems, 35:22870- 2-222870-22882, 2022.
18

---

Published in Transactions on Machine Learning Research (02/2026)
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data
distributions. In International Conference on Algorithmic Learning Theory, pp. 946-985. PMTR, 2023.
John M. Lee. Introduction to Smooth Manifolds, volume 218. Springer Science & Business Media, 2012.
John M. Lee. Introduction Riemannian Manifolds. Graduate Texts in Mathematics. Springer International Publishing, 2019.
Gen Li and Yuling Yan. Adapting to unknown low-dimensional structures in score-based diffussion models. Advances in Newral Information Processing Systems, 37:126297-126331, 2024.
Gabriel Loaiza- Ganem, Brendan Leigh Ross, Jesse C. Cresswell, and Anthony L. Caterini. Diagnosing and fixing manifold overfitting in deep generative models. Transactions on Machine Learning Research, 2022.
Gabriel Koaiza-Ganem. Brendan Leigh Ross, Rasa Hosseinzadeh, Anthony L. Caterini, and Jesse C. Cress- well. Deep generative models through the lens of the manifold hypothesis: A survey and new connections. Transactions on Machine Learning Research, 2024.
James Raymond Munkres. Topology. Pearson, New York, NY, second edition, reissue edition, 2018
Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. In International Conference on Machine Learning, pp. 26517-26582. PMLR, 2023.
Tyrrell Rockafellar. Conver Analysis, volume 28. Princeton University Press, 1997.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resohution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognation, pp. 10684-10695, 2022.
Brendan Leigh Ross, Gabriel Loaiza Loaiza-Ganem, Anthony L. Caterini, and Jesse C. Cresswell. Neural implicit manifold learning for topology-anen awwae density estimation. Transactions on Machine Learning Research, 2024.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M. Weber. CADS: Unleashing the diversity of diffusson models through condition-an annealed sampling. In International Con- ference on Learning Representations, 2024.
Seyedmorteza Sadat, Manuel Kansy, Otmar Hilliges, and Romann M. Weber. No training, no problem: Rethinking classifier-free guidance for diffusion models. In International Conference Learning Repre- sentations, 2025.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text- to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479 36494, 2022.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021a.
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning, pp. 32483 32498. PMLR, 2023.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Newral Information Processing Systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishck Kumar, Stefano Ermon, and Ben Poolc. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021b.
19

---

Published in Transactions on Machine Learning Research (02/2026)
Wenpin Tang and Hanyang Zhao. Score-based diffusion models via stochastic differential equations: A technical tutorial. Xix preprint or 00x::2402.077487 2024.
Pascal Vincent. A connection between score matching and denoising autoencoders, Neural Computation, 23 (7):1661 1674,, 2011.
Zhengchao Wan, Qingsong Wang, Gal Mishne, and Yusu Wang. Flucidating flow matching ODE dynamics via data geometry and denoisers. International Conference on Machine Learning, 2025.
Yuchen Wu, Minshuo Chen, Zihao Li, Mengdi Wang, and Yuting Wei. Theoretical insights for diffusion guidance: A case study for gaussian mixture models. In International Conference on Machine Learning, pp. 53291-52250 5-327. PMLR, 2024.
A
Notation
The symbols used throughout this paper are clarified below.
1, Letters: Unless otherwise specified, lowercase letters such as a and æ denote deterministic variables, while uppercase letters such as X and X denote random variables. Scalars are typically represented by non-bold symbols such as and Y, whereas vectors are denoted using bold symbols such as æ and X. In partieular,
I, IRxx to denote the identity matrix and RR" to denote the zero vector.
2. Linear Algebra:
(i) Let O c. gmxx (with m > n) denote the set of matrices whose columns are orthonormal, i.e., those satisfying A I,.
(ii) For vector RR, the notation ||a:|| refers to the f--norm. For matrix A R=xn, the operator norm is defined
| = sup |x V Amax /maax(A (ATA),
where )mmx denotes the maximum eigenvalue.
(iii) Let A, B E RRxa be symmetric matrices, i.e., A = AT and BT. We write 3 B (or equivalently, B A) if B- A is positive semi-definite, i.e.,
(B (B-A)x20 20, VEER".
3. Calculus:
(i) For a scalar-valued function f: R RR, the gradient with respect to æ is denoted by (=)( and the Hessian matrix by (x)(
(ii) For a vector-valued function F: R" RT JF denotes the Jacobian matrix of F, and the second- order derivative D2F is bilinear map Dd F(=): Th R R defined by
D²F(x)(U, w| dsot
F(x + tw) (v (z) D (z) T
where F = (F FT)). If each F has continuous derivative of order k, F is called C* (iii) For any sot U R", the characteristic function Xo: R IR defined by xu (x) (x:) 0 otherwise.
U, and
(iv) For integrable functions f.g: R" + IR their convolution is denoted by
I (*g(x)) f f(y)(( y)dy.
(v) For a function f: RD - T let I = f(R") denote the image of f. In particular, for a matrix E Rx Im A refers the image of the lincar map æ + Ax.
20

---

Published in Transactions on Machine Learning Research (02/2026)
4. Probability-relatted Symbols:
(i) We fix the base probability space FR, where Ω is the sample space, F is o-algebra, and IP is probability measure on F.
(ii) On R", we typically work with the Borel o-algebra B(R"), and let P(R") denote the set of all probability measures defined on B(RR). Symbols such as l and v represent elements of P(RR). The integral with respect to mcasure a denoted by f())du((z) or cquivalently by f())))(d)) (ii) For a measurable map f: Ω + R", the push-forward measure of IP under f is denoted by f#P, and is defined as
= -
Y ((R)).
(iv) A random variable (or vector) x: 0 RR is a measurable map. Its distribution, denoted by Px (or px), is a probability measure on R defined by Px = P For some μ E P(R"), we say X ~ p if μ = Px. Two random variables X and are said to equal in distribution, denoted by X $ Y, Px = Py.
(v) For X ~ Px, if Px is absolutely continuous with respect to the Lebesgue measure dæ, then by the Radon-Niin ModoonNikodyym Theorem, there is a function px (or denoted by p*) such that
Px(U)= = 1 px(x))d, V B(R"),
and px is said the density function of X. For a measurable function g: R" + R", if Px then g(X) gxPxx. When Px admits a density px, the density of g(X) denoted by J=Px. particular, if g(x) Ar for matrix A mx 9 x also denoted by AgPx for simplicity. (vi) For random variables X: + R and Y: Ω - R, the joint distribution of (X, Y): Ω + RR x R denoted by Pxy = (x, D), a probability measure on R" x R. The conditional distribution Pxpr(: Y) defined as
(U P(X EUEB(R")
which is a probability measure on IR.. (vii) For probability measure μ P(R"), the support of μ is denoted by
supp /r {ar R / (B)())) 0, v r 0}
where (x:)) C T denotes the open ball centered at x with radius r. When a admits a density function p;
supp μ fx E R" p(x) >
B More Details in Background
B.1 Analytic Solution for DDPMs
To solve the SDE
dx, -() + V V [0,
we multiply both sides by the integrating factor e So p(s)ds This gives
# f p(e)do dx, + 2 (())) So (())ds x,dt vB(tje S. p(s)de a
which leads to
( (loddo
3(o)do x1)dox)) /Bttee So glo)ds dW,,
When unambiguous, Px also oecasionally referred as the distritution.
21

---

Published in Transactions on Machine Learning Research (02/2026)
by applying Ito's formula. to e² Jo B(s)ds Xx. Therefore, obtain the solution
X, va +
where a, == exp (- Jo (s)ds)) and
S .. A(r)dr
{==
VB(s)dW..
Since (Wi)r20 is a standard Brownian motion on I it follows that & ~ N/ [ denote the quadratic variation. Then
oz In). To compute oz let
of E 116, El,l
fo St. A(ridr ((rrddrr(ssd ((s)
E
/
S s(r)dr B(s)ds= = - exp (- B(s)ds)
(see
Le Gall
2016 for details). As a result,
&. d VI- ais, {~\((0,
B.2 Density Functions in Conditional DDPMs
Proposition B.1. Consider joint data density function p(æ, y) and the process governed by the SDE:
dx, -
- ((t + V3(t)dW.
For the following two scenarios:
(a) Let X ~ p(æ Y = y), and run the SDE for X = X. Let vi (2xe) be the distribulion of X,, (b) Let (X,Y) ~ p(ar,) v), and run the SDE for X = X. Let p pe( u) be the distribution of (X,: Y),
Then, we have
(xx, = pr( v).
Proof. As shown in Equation 2
X. Va.Xa + v1- orE, &~ &~0,
where a; = exp (- (s)ds)) Therefore, in the first case, we have
(x.) = (Voz) y) + +N a.)
Moreover, by Lemma B.2 since & is independent of (Xo, Y). it follows that
(r v) p (Var /1 a, v) p(Var y) * (0 (1 az) Ip) = (Van)+p(æ v) * (0, (1 a, Ip).
Consequently, obtain:
py pe(ax v)-
22

---

Published in Transactions on Machine Learning Research (02/2026)
Lemma B.2. Consider three random variables, X, x.Y& R", and Z R. Let Y be independent of paired (X. 2), and W = X +YY Then, we have
Pwiz(w 2) = (px]z(] 2) pp (w)
Or informally,
Px + yl px|z pr (y).
Proof. Because Y is independent of (x, 2),
Pxy z xz, 2) pxz(2,
Let Dw = ((x,)): + S w) Then, we have
P(W I w, 2 P(X P 7 25
L' (JL (x, >)deddy da
L' (Jh, pxz(a, z)py 2) (y)dwdy dz
! ! (pxzl1,2 py (-)) (s)dsdz,
where W (Wi), (w:); means W S wy for all i I....,n, n, and Jo ds -o" " ds ... dsp. It follows that
Pwz(w,22 2) = py
Therefore,
Pw]z 22) Pwz wovz(w,2) (w,
(xx(1,2) * xpr()) (w) = (pxiz( 22) py pr))) (un). (z(2)
(z(2)
Remark B.1. In Lemma B.22 the existence of density functions is assumed, which also makes it necessary assume the existence the density for Xy in the proof of Proposition [.11 However, this condition is often not satisfied in practice. To address this limitation, consider the convolution of two probability measures p. E P(R"), defined by x ++v(U)== = f f(c (u + y))d(())dr(y)
Note that * is still a probability measure. Moreover, it follows that if X and Y ~ v with X independent of Y, then X + ~ u. Under this formulation, the conclusion of Lemma B.2 remains valid in the general case:
P P () Px Z) PP = PxIz( Z) PPy
where the first equality follows from the fact that independence of Y and (X, 2) implies that Y is inde- pendent of X conditional 2, and the second equality holds because Y is independent of Z due to its
independence from the pair (X,2), Therefore, by following a similar line of reasoning as in the proof of Proposition B.1 replacing statements about densitics with statements about distributions we can obtain the same result even when Xo does not admit a density function.
C More Details of Geometric Guidance
C.1 Omitted Poofs Section 4
Proof of Proposition Fix a time & 0. By Equation
X, 0z AZ v1 o
23

---

Published in Transactions on Machine Learning Research (02/2026)
for some (V, ID). It follows that
= vI- AA AAT)ElI
Note that AA is the orthogonal projection to Im A. Therefore, there exists U ODX such that
ID AA UT diag( 1, ,,.,, 1 ,,O
D-d
Morcover, the orthogonality of U implics that v = (2/,...,, vDD) = -UE (.( Ip). Hence,
f(X,) = = a: ai -A ) = v1 as (v} + ... v v
For any > O, by selling a = (D d)s in the Laurent-Massarl bound (Lemma H.1 we obtain
p 11001122555 ((x)) S $+00 28 2 1 - 2e- -DD-
Since d < D, we choose sufficiently small such that 5 2(D--d) is also sufficiently small. As result, P(f(X,) ~ r(t)) 21 - 6, i.c., X, concentrates on M (r((( with high probability.
Proof of Theorem First, by applying the orthogonal decomposition of the score function in Equation the deterministic reverse process 4 can be rewritten
1
x = X AV, log 7 (A
- ar (ID - AA )x;
(18)
dt
(a) Because A E ODxd we have A A Ia and A AT(D - AAT) = -O. Therefore, by multiplying A on the both sides 18
Z = zi
1o (Z,
dl
for Zi- = AT A X, Moreover, by the equivalence of the continuity equation of the Fokker-Planck equation (or by the statements in Appendix C.2 Z: - satisfies the forward process of DDPMs starting from p².
(b) Similarly, by multiplying ID AAT on the both sides of 18
0T X
x
dt
for X (LD - -AT)X, Note that aT = e - solution given by
Therefore, this equation has the analytical
e- (1o+6)))))
X =
2(T to.
When X -d -d e- -(("- it follows that
2(T-
[
| V(D d) (1 2-2
c- 2(T-10)
24

---

Published in Transactions on Machine Learning Research (02/2026)
C.2
Decomposition of Score Function
By Equation
2 and the assumption Xo AZ, have
X; = va + v1 agk
vaiXo + vI aiQE v - Q)
=1,,,1
for some ~ (V ID), where Q AA is the orthogonal projection onto Im A.
We compute the covariance:
Cov(QE, (TD Q)5) E [QE (IID Q)4) E [QE] E (ID Q)8)
QQ ((In- - Q) = QE E (ID - Q)
Q Q) =0,
which shows that QE and (ID - Q) are uncorrelated. Since both are Gaussian, they are independent. Henco, X independent of /T- ag Combined with the fact that independent of Xo, follows that X1 is independent of X By Lemma H.2 the density of X, admits the decomposition
pe(x) = (x) )P.. [x)
(19)
where Pt.ll and Pt,1 are the densities of X and X with respect to the canonical volume measures on Im A and (Im A)+, respectively. Here, # Qz and æ x Next, let us analyze Pt,ll and Pt.L, respectively.
(i) For the parallel part, first define Z, == ATX, Then, by multiplying A on the both sides of Equation we obtain
dz, -zdt + V2dB
where (Bt)220 = (ATW), Wi)oo is standard Brownian motion on Rd by Lemma H.3 Therefore, the process Z. ~ pt is governed by the DDPM dynamics initialized from p². Since
X1,I QX: AZi,
this shows that X evolves a diffusion process on the target data manifold M Im A.
Morcover, applying Lemma H.4 gives
Pc. (2 A4p = (A (A x).
(20)
(ii) For the orthogonal part, we have
X = - -a==---o/pt---0010-aaa
where P In Q is orthogonal projection with rank D-d. d. So P B B for some B E (D-()))) It follows that X., is Gaussian on Im B, i.e., X = BW for some W ~ N' (1 - a, p- Therefore, X,, is basically a (D - d) dimensionall Gaussian. When d < D, as shown in the proof in Proposition
[| 1x = | x
which implies that the orthogonal part X is responsible for the concentration of X, on M and endows X, with its geometric structure. Furthermore, by Lemma H.5 pr approximately uniform on the sphere ((D-d)-((r()). In other words, the density Pt, which concentrated on the cylindrical- like surface M remains constant along radial directions and varies only in the longitudinal direction governed by Pt. consequence of diffusion along the subspace Im A.
25

---

Published in Transactions on Machine Learning Research (02/2026)
Moreover, applying Lemma H.4 again, obtain
(..()1) = B&p W (x = pp"
x1 = =p (BT p
(21)
where
nd (w) = (2 at))
R=d
|| 200 2(( ovt)
exp
Finally, for the decomposition, by combining
20 and
with 19 we get
(z)) = log P 2 (A σ) + log p" (B"x). x))
from which the orthogonal decomposition formula immediately follows:
V lo) A V log (2) (2)l2-ATa at
(ID P,,,,
as originally derived via direct computation by Chen al. 2023b).
For the geometric property, the randomness of X, arises from the diffusion process the target data manifold M = Im A, while the geometric structure of X, results from the concentration behavior of the orthogonal part.
C.3
Construction of Geometric Guidance
To clarify our intuition about Ve log = a)) "almost normal" Mi, we will show that there exists a small 3e > such that
V Py = x) + m S Br, E
for some scalar ne O. But first, we need the following lemma.
Lemma C.1. Let M C. RRD smooth manifold with dimension 1. Let V C RD be a tubular neighborhood of M with the orthogonal projection w: M. Let f: V be a C² -ffucttionn satisfying the following two condilions.
(a) [122f())]lo L.
(b) flm is B-Lipschilz with the induced distance of R on M. Then for any e V,
[ - -..[((())) S B + dist((x,
where n: M + R is continuous unit normal vector field along M, 0nf = (Vf,n) n) the derivative along and dist(æ, = inl { yll: y E M} is the distance from i to M.
Proof. Let M be equipped with the induced Riemannian structure of R and V be the corresponding Levi-Civita connection. Because M C RRD is hypersurface, i.c., submanifold with dimension D 1-1, 1,
V M ()n.f)n,
(22)
see the details in
Lee 2019
Chapter 8). Fix x E V with = π (xx) E M. Note that
dist(x, || y
(23)
by Lee
2019
Proposition 5.26 (c)). Writing
[=)()) -.. (..(2))(())))
- +V a. Oz)()))()))
(24)
26

---

Published in Transactions on Machine Learning Research (02/2026)
I. For the first term, by
Val( - (n) = V (y + s(x - y)) - y)ds,
the fact that ( T, and Equation 23
have
[a) Vef(y)) _| - yll = L dist M).
(25)
11. For the second term, first, by 22
(=f(y) ( = | f(y)ll
By assumption, flM is B-Tipschitz with the induced distance of IR" M, i..,
(f()) f(y2)) S (dM 92),
where dm- It implies that see the details in Bounal
| f(=)) S 8, MEEM,
(26)
2023
Proposition 10.43).
Then combining the inequalities 25 and 26
with 24
/ (x) ()) 8 + da,
Let fx( (v 1 z). It is natural to assume that is C² tubular neighborhood V of Mi
that 2 I L, on V, and that ft is Be-Lipschitz contimuous Mi. Then by Lemma C.1
[V))((>) a, (r..((((r))))) s 3 + I dist(r,
In particular, for any E Mi and x(x) = a, we have
[V= - (. S 3.
Two questions remain: whether (z),(z) = Pi for some scalar > 0, and how to bound B.. For the first question, we can choose ny nx( = Piz//1
Pr /
by the definition of MM and Lemma L.1 So
0 (z) -1p Plar
for
|
Moreover, because Pı(y 1 x) is the classifier for (X,, y 1) and such X, concentrates Mi by Proposition [ fe(x) = log (ı = 1 æ)) decreases when æ moves away from Mf. So
0nf((x) <
7>> 0.
Next, to bound 3r, we introduce the following lemma.
Lemma C.2. Let p(y- = x)) be softmax classifier with logits (x( for 1: = 1,2, K, that is,
exp(gx(x))
plu = k EK exp(gx(x))
Assume (x)))l S for all k, Let Mr be the region where points with label y k. concentrate on.
Assume classifier confidence
p = k YoEMa.
Then
IV n logr(y k 2:)] s 2Te, VreM.
27

---

Published in Transactions on Machine Learning Research (02/2026)
Proof. Fix k. Let f()) logply k x))
K
[=f(x) Valk(z) - Σ
plu = x) Vagg(())
j=1
K
Σ
ply = x)) (v])(<)) Vz9x (x))
j=1
Σ ply = j x)) (Vag,(>) Vz9x(x))
By assumption,
|| (x) V *> - | (x)) (e)x((>) s 2T.
Therefore,
Vaf(z))1 & Eply j x>)[V=)}(()) - (x)x
jtk
S 2T, Σ ply = j ))) = 21(1 ply k x)).
8/k
It implies that
[aa[(>)|| - 2Le, V Mr.
by the assumption that classifier confidence > 1 & on M
Therefore, for all (1 æ;), we assume that they satisfy the conditions in Lemma C.2 Then if
Prly = x) Et. Væ Mi,
for small then
NVM f(())] < - +1. 2CE, VIEM
So 3e 2C
Combining above results, we have
V log = x) + p/ Si. Væ e Mi
for some 7/ > 0. Moreover, B, = O(a.) for p. (v x) > 1 - Mi.
D More Details related to Main Results
D.1 Omitted Proofs in Section 5.1
Proof of Proposilion First, the Hessian is
v logpx(#)) Vap((=) VaPx(c)VnPx(e))
Px(x)
Px (x)²
To express the above formula explicitly, by the definition, for any æ E R"
px(x)= = 1 (2))))2(2)dd2 K2(x)= = (2 + exp
aB2 |
2,32
and so
V (x) aBz x K (x:)
32
28

---

Published in Transactions on Machine Learning Research (02/2026)
1
(a - aBz) - aB22
(2)(()) - 8 (x) +
K2 K=
1r
Let
K2 (x)x)p²(22 (xx))
(2)
dz
be the posterior probability measure on T Then, for the first term
V)px(x)
JoN (2) 3 In + Ez-ua - (x--aBB)xxaald
(x)
(x
B²
B
and for the second term
V (a
1
Bz-ua (x: -aB2]Ez--p 1x aBZ
E2
px (x)
3
Moreover, note that
Ezzne [ (z aBB) -Ez-us [x aBZ E Zz (x aBZ
= CoV (x aB2) = a² Covp, (BZ) = =a2 Cov,,, (Z) BT
Therefore, we get
a²
V2 logpx(())
B (2)BB
32 I..
(27)
83
It follows that
a²A | uz (Z)llop
V2 log px (x)
I
+
(28)
up
32
8a
It sufficient to bound |Cov,,, (Z) llpp To do that, Mr is required to satisfy the Poincaré Incquality. Let (z) exp(-V(z)) for some V: R* IR and
Un(2): == 1 - aB2l 2
+ V(z),
2,32
which indicates that (z) (2) se Because V2V(2) =-22logp" 2 molk,
22)
(²Ux(2) = 3z B B+ V V2
+ mo I
32
Then, by Lemma H.6 Na satisfies the Poincaré Inequality with constant a21/32 mo- Thus, for any C function f: E. R.
1 _ |
Var,, (a)p,())= S En,
m
For any u E RR, let fu(z) = (u, 2) with (z) = u. The above inequality implics that
u Covp, (Z)u= Var (fa) S Ep, [[V=full?
1
1
1 =|ull?
s
m
m
for any u E RR Therefore,
1
||Cov (2) llp < m
(29)
Finally, by plugging inequality 29 into Equation
28
we get the result
a 8 +
log
(l)
g
32)
29

---

Published in Transactions on Machine Learning Research (02/2026)
Proof of Corollary By Equation
29
A
A
B p. B
+
B
Cov pz (2)B
1 In.
op
By combining this with Equation
27
have
a²A
1
V logpx(x)) 3
In
Ba(a²> + x
}2
Proof of Proposition 5 By Lemma
H.7
Px AgpP w1 A + W2
Moreover, because 11, O) + +oo P.C with &~ NO Ia):
AZ A +AA p (: Y 1).
Note that 2 P ( = =). Therefore,
Wi(P (:1 Y = 1), Y 1)) S E| A, = oE [||I||] S ovd, where the final inequality is because AÇ N Ia) and Lemma H.8
Similarly, it can get
M(P (: Y = 2), P YY= =)) sovd.
Combining these two incquality and by Lemma
H.9
we have
WIPP3, Px) w Wi (P Y = 1), Y = =1))
+ (P (-1 2), | 2))
<ovd.
Proof of Lemma 6
Let
(0) == wp)(x)
r2(x)= == 1-rr (1 - w p2))p(()) x)) pl (z)
p(xx)
We have
w WPp + (1
w) V)
V logp=
ri Vog P1 + +2 V logp2.
p
and
2 logp= V2 22 log + r2 log Vri (V V log
For ri wpr/ /p,
Vri = pV - Vp
p²
= w (urp + (1 w w))
(W/Vp +(1 Vp2)
1
wr(1 w)
12 (pz Vp1 P1 Vp2)
1172 (V log
V llgg22).
Therefore,
2 log = 2
Volog log 2 log p2 r172 (V log V log
(V log
V log
30

---

Published in Transactions on Machine Learning Research (02/2026)
For the first two terms, by the assumption,
lo + + V2 log rrLL + r2222,, 3 max {L1, L2) In
For the third term, because sup
logpi (x) V logp2(>)) _ M,
v log - log V Vogg2)) "' 11, S SM?
which implics that
(V logg1 V Vlgg2) (V logp1
V 1og M
For the coefficients r172, because T1, 12 (0, rir2 S 1/4. Combining these results, we have
V logp 3 max [L,, L27 + =4" 12²
I
Proof of Corollary is decreasing in o,
Because
mi = mī(o) = m
M
+ mo²
M
mg mm(0)
4
With the Assumption ||I we have
M
> 1
4
Therefore, by choosing small o, we can also have m > 1. It follows that
m
m. mî + (1
m8) 2-
is decreasing in I. So
m
mi inf rE(0,T]
m4 mr
mn
(1
mn
2T
1.
D.2 Proof of Theorem 10
Proof of Theorem
By differentiating
Y from 14).
1 d
Y, 2
Y,, dt
Y.
2 dt
=(Y, Y, + P. v log log n/)i)
n)) Yall Y, P.V log ,
(1 7)) Yi1l Y logrr,,
Therefore,
a ا <1 (1- (1- Vl 1o.
Taking the expectation on the both sides yields
d m, < (1 m/m, + [[[ log logpf-((X.)]]]
(30)
de
where m, E ( Therefore, the next step is to bound E [[[V~ logry (X.)]]]
31

---

Published in Transactions on Machine Learning Research (02/2026)
Let
I. = sup Ltt C sup eE[o,T]
V (ogr((()) < co;
(31)
(E|6,T)
where Le is defined in Theorem
8 By the Ls-Lipschitz of V logg (Theorem 8
V log (X,)
2 V log pX - V log logrl- + V (0
(32)
S Ls| Xel +C
For X, in Equation
+ we have
X 1| = 2 x11 122 + 22 Vo log log - 2n/ (X,, PiX.)
dt
22 |1| +2 X Vx log - (X.))
2 X 2 + 2 V log pT (x+)l
where the second inequality is because (X P1 x 0. Combining this with (32)
X S (1 + Ls) +C.
By taking the expectation on the both sides of above inequality, Grönwall's Inequality (Lemma H.11 implies
E X E [o]x c
C
(ca+L2)--1)
(33)
I+ Ls
Because In), E
S DD (Lemma H.S It follows that
X || S (L
+ (ell+Es)T -1) =: M
C
sup xE(o,T-a) A]
1 L1
and 32
implies
Ve log logpp (
v sup Ls || X +C
sup -E
1
(34)
Ls =: M2.
Then by substituting this into 30
d
(17 1)m, M2.
dt
Because mo = E Yo [ & /D di by Lemma H.8 by applying Grönwall's Inequality again, we obtain
E Y, m, S V D die-
1)-1) 1)t + 17 7-1
M2
(1 (1-6-(9-1)) =: (t),
(35)
which implies that
6 s /D- - die (1-1)((0)) + M2
E Yr
For any E >> 0,
M₂
8
2M2
<
+
2
+1,
17
2
and
D die - ()
<
8
+
2 T
log
2/D di
+1.
2
Therefore, for any g > 0, by choosing
2M2
2vD di
" 2 max
log
1,
T
we have
E
< E.
32

---

Published in Transactions on Machine Learning Research (02/2026)
D.3 Theoretical Analysis for Universal Guidance
Consider the universal guidance model
dx; x + v lo
-NV - X ~N(,
(36)
dt
for t E T), where Pt is the density function in DDPMs.
Theorem D.1. For the dynamics 36 assume that log Pt is L-smoothness, f p-strongly conver, and E [[(x < cc. Then
- (
where v. is the unique minimizer of J.
Proof. By differentiating ((
f(x)) (va((x)) d X;- dt
d
(X,
V (
$ | V |1
V
Let C= = suPte(6,1] V ())] 00. Then, the L-smoothness of log Pt implies that
| (X
Therefore, by ab < (a² - -22)/2, we have
d dt
< (1 + T.) /007:/101100000000///00000000001
< 1 [1x==""++++,,,,x>>""">"
( +0 - ((
(
2
2 (n [>
1
1+
X 12
C2
2
2
Because f is p-strongly convex, by Lemma H.12 it satisfies the P-PI. inequality,
[x -
f(x.)),
For L, we obtain
d - +
1+ L
C
X +
1
+
di
2
2
Taking the expectation on the both sides yields that
(B(()- ((x)))
1 1 L E
|x:112 112
C2
(37)
T
The next step to bound ||] Let R = X; E.. Then
R (R, X + V log (X: - -
(38)
RR x R4, V log ( - (R,, vxf(X.))
To obtain the desired inequality, we consider these three terms respectively. For the first term,
RR, x,, |R (R, 1 R2 2 ||.
(39)
33

---

Published in Transactions on Machine Learning Research (02/2026)
Let Va Pog-)- By the L-smoothness of log Pt. we have progpr- )| < Pr v log (x..
R +c.
Therefore, for the second term,
(R.. V Po - | P I
(40)
- R. || + c R
For the third term, because f is p-strongly convex, (x)(xt)) = 0 and
Re. Vaf(X;
RR, V v=f(X; vzf(xx)) 2 Rel
(41)
Then, by combining 38 with
39
40
41
we have
d
R
< 2(L + - m1p) R2 26 R.
dt
S (2L 2npp Re| + c
where ē implies that
||==|| +
By taking the expectation on the both sides, Grönwall's Incquality (Lemma H.11
E
R |?]
E
Roll']
(2np-22-33) +
6²
(1-
(2np (2n0-21- 110-22--30 22n0-21-3)1
2np 2L
By taking sufficiently large 7 such that 2mp
22L -3 δ > have
E
Re [
S |oll +1
Note that X ~ ~N(O,, which implics that E
| !|
D. Therefore,
E
s
|| ||
+ || 2| s D + 12=11
and
E
|
1|2
<
E
||x.|| ~ S 2/ 112 +1 MM.
By substituting M2 into
37 we obtain
f(x)) + M4
di
for M4 = ((1 L)M3 C2) Then, by Grönwall's Inequality,
M4
[[ f(x.) (B[ [(x)) --1-2-1.jJ + p(n - T.)
which means that
f(cc)) ( ( - -11)
D.4 Proof of Theorem
Proof of Theorem 17
The proof consists two main steps:
34

---

Published in Transactions on Machine Learning Research (02/2026)
(i) Let Q1 ALAI. For any coupling (X, X) (fr-6,Pxiy |Y= 1)), we have
Wi
Y== = 1)) S <E X
[[QQ Q1 || + [|] - - PiX|]
= E [[Q] X|] + |
where the final equality holds because X ~ Pxlr (: | = 1) implies that Q1X = X, and X ~ PT-6 implies that Pi Yr--- And by 35
E
Y < MT - = =0 T(n
Let (QiX; X) be chosen the optimal coupling for ((Q)) ( -o xxpr (- YY= 1)), i.e.,
Wi ((0))=p==sPxx((((
= E (1 1))
Y
-X]]]
Therefore, we have
Wi (iT Y==11 = 1)) < Wi (Q)))Tr-a,Pxx) xY=1) + + +n--)
(42)
(ii) For Wi ((Q))#PTo ( (1-1-1) 1)), by the triangular incquality,
wi ((Q0)))77-mPP = 1)) S wi
+ + (x
(43)
S Wi (pr-s,Px) + (Q1))4PxPPxpe (Y=
where the final incquality is because Q1 an orthogonal projection (Lemma H.13
By Lemma D.3. the second term in above inequality is bounded by
Wi ((Qi) Pxir Y 1)) s č
(44)
for some constant Öj For the first term, it can be divided into
Wi (ĀT-6, S Wi (pT-8, - Wi (fs.PT) + Wi (0g. PP + Wi PP),
(45)
where P. is defined in dynamics 51 pf is the density evolving in the DDPM initialized from P see
13 and P& is defined in Proposition 5 For the four terms in 45
(a) By Proposition
D.5 and m2 > 1 (Corollary
Wi (ØT P8) < O + +
(46)
for some constant O2.
(b) By Proposition D.4
Wi (ps.P%) S O(e
(47)
(c) Note that
X JazAZ VT- ~ pg,
0g =
for Z ~ vz. Morcover,
AZ P Therefore,
Wi (08, Px) < X AZ
< Xg Var A Z
/08 E || A 2
v as E [||||]] (1 (a8) Ezzp? [|| |]
< 228 + 5m
where m = EZzpp
||| < oo by Lemna D.2 It follows that
Wi (rg, PPx) (
(48)
35

---

Published in Transactions on Machine Learning Research (02/2026)
(d) By Proposition 5
Wi (PP x, Px) S O(o).
(49)
Then, combining
46
47
48
49
with
45 we have
Wi (fT Px) 50) + 5 +0+ + C2.
(50)
Combining
50
44
with
43
it follows
Wi ((Q1) (x = 50 +01/3 + +
where Ö = Õ1 + C2. Therefore, substituting this in 42 we obtain
Wi (iT-a,PxIx( - =))
( + 12 +o 17 +C.
Remark D.1. For the error floor Č we provide two further discussions.
(i) First, it follows from the above proof that Ö = O1 + C22 where
C1 determined by 44 and Lemmna D.3
Ō = w2m?, m² = m² E Ez-p? [|1
which is independent of the parameters T,6,0.
C2 is given by (46) and Proposition D.5
0 M2
where M2 is defined in 34 and depends Ls supie(o,T- La and T, while my inf me Since Lg and my are specified by Theorem through pf, Ö₂ depends implic- itly on T, 6, and a.
(ii) We believe the error floor is inherent to the geometric guidance model. Because of the analytical simplicity of the geometric guidance, it cannot provide as much information as the probability guidance term did. More preciscly, in Appendix C.3 we show that
V log x) + P S 8c, V
= (= x) > 1 - Er for all E MM. This shows that the probabilistic guidance V log pr = æ²) is "almost parallel" to the geometric guidance Plæ, but the norm of the probabilistic guidance carries additional information that the geometric term cannot capture. This is trado-off made for the sake of analytical tractability.
for some scalar 7/2 O, and 3, - O when pr(y
Lemma D.2. Let Zi pt for 1,2. If m [[] < x0, then for vz defined in
12
m Eznpy [|| xx.
Proof. By the definition of 12
Z $ E|| Z,
oE [| S m + ovd < 00
for Zioo ~ p,, where the second inequality is by Lemma H.S Then, by
E Ez [|| = /
zpz ==²(x)x (x)dz
W1 JRJ zp (z)dz 002 / 2 (2)dz
Rd
wy E [|] Z 1 || + w= E. (2,0= < co.
36

---

Published in Transactions on Machine Learning Research (02/2026)
Lemma D.3. For
Px = wxPxir Y == =1) + u2 +=22xxr(-111 [==Y/ 2)
under Assumption !
Wi((Q)))Px,,xxx) (-1--(1 w2mf,
where Q1 = A and m = E 2|1]
Proof. First, by Lemma H.7
(Q1)=Px = (Q1)) (11 1) P 2)
For the two terms, if X Px 1Y=1) then Qi = X, which implics that
(Q Y = P Y =1
On the other hand, X ~ Px Y = 2) implics that Qı = 0 so that
(Q1)*Pxfr( (-YY- = 2) = 60.
the Dirichlet measure at O. Therefore, by Lemma H.9
(i((Q)) Px, Y 1)) S (oo X
For any coupling (D, X) ~ (60,Px|y 1 YY= 1),
(6,6 Y= =1)) < E [|| XII
E || [|] = Ez ||A| = Ez~py [|]
where the last two equalities are because Px (: = 1) (A1) Fa and A E Oxds by Assumption !
In the following, unless otherwise specified, we assume that Assumptions ! and hold. Proposition D.4. Let pl be defined in 15 Consider the following two dynamics:
dx.
X. + Va log (8)))
~o0N(0,1n Ip)
(51)
dt
with the notation X, p8 and
dX,
X₁ Va (Xi)) Xo 8o~pft pt.
dl
where note that X, -- For 8 > 0, we
(i( S E mi((88 (m2 VD)
where m = Ez-p3 [ and my = inf my defined in Theorem
Proof. First, by the Theorem p is my-strong log-concavity for 1 E [0, - 6), which follows that
( X,, Va lo V log = (X, X,, v2 loggp (x, - x.))
mm 1x,, x.11.
Therefore, we have
dt 1 x,12 (xx; X,, d (x, x.))
dt
37

---

Published in Transactions on Machine Learning Research (02/2026)
2|| - X.112 +2 2 (x Xx,, v (X.) - V 17
- 2(ms 1)||X; x.11?
which indicates that
d - -xll < - (m - X.1l.
dt
Then, by Grönwall's Inequality(Lenv (Lemma H.11
8 - Xo- Se (ma 1)(0 1::0 - Xoll.
Therefore, by the definition of Wasserstein distance,
Wi P3) E [| Xr-oll
S (mss- (ms--)((T 66) |1xo Xoll
By choosing (Xo, the optimal coupling, obtain that
Wi (1% P8) S (mm - 1)( () ID).PTT)
(52)
For the right band side of 52 by the definition of pi in Equation 13 i.e.,
X = Va,AZ + VI aig p4
for Z ~ p/ E ~ No0, ID), and o4 = 20 we have
S ||AZI = = 21]
Moreover,
(1- (L ID) (1- B se-rVD.
Therefore,
WOO ID)) S
$ E T m + VD)
Substituting this in the inequality
52 implies that
Wi (8% p.) S e m. (1-)- (m3 + vD) < (T((5) (m² + +VD)
Proposition D.5. Consider the geometric guidance model and the dynamics generoted distribution ot and Pi, we have
51 for the corresponding
M/ Wi (pr- S WVD 10-d dj - 1X(T-8) + (m1 1) (n -1)
11 my
where M2 is the constant defined in 32
Proof. By the my-strong log-concavity of pt (Theorem g
we have
1 d
X - xp112
x, Xx dt (x: - x.)
2 dt
(x, X,, X; X; + V. pf_, Va talogpT- (X.) + 1)p;
-(mi 1)(IX, - x2112 + n|| X, xx||||
38

---

Published in Transactions on Machine Learning Research (02/2026)
Note that P,X, Y. It follows that
d
1) X (ml1))
X.1l + 7 E/|l
dt
Moreover, by 35p. taking the expectation on the both sides yields
[1 | xx1l + (1)) x(1| s -m 1)B
R
dt
Then, Grönwall's inequality implies that
Wi (6T--.19
S E [ X
T-b
My(s)e (mns 1 -6 ds =: I(1),
(mmy
S1 1)
0
when the initial coupling is chosen as Xo = Xo ~ NNO,, For I(n), by the definition of (t in (35 we have
T-8
VDD die- 1)-1)) + M2
1(1) S 11
--1 & (ms
nVD di
(mi (1)--) (1 e ((-mi)(((00)
11 m111
(m. 1) (1) (1 - (m1--)
n112
7V d1 (my 1)(T
nM4
17 - m2
(ml - 1) (17 - 1)
D.5 Discretization Error
To clarify why performance degrades in practice when y) becomes too large, analyze the discretization error of the geometric guidance model ( In practice, ODEs are typically solved using the Euler method, while SDEs are solved using the Maruyana (EM) scheme. Since our model is formulated as a deterministic ODE in we focus on the Euler approximation; the analysis for the corresponding SDE and the EM scheme is analogous.
More specifically, we partition the interval T', 5] into N subintervals with step size = (T - 8)/N, and define & kh for k = 0, I. N. The Euler scheme then constructs the sequence {
h J k=o
via
X k+1 = x +h + V po (x P)
X ~ (O)
Let X ph Our goal is bound the Wasserstein error Wi ( Under the Lipschitz continuity of Va logyf, standard results yield Wi ( O(he") (Grilliths Higham 2010 Theorem 2.4). Because of Theorem S not only have the Ls-smoothness
2 log (z)llop S Ls, Ls sup Ln, tE(6.7]
but also the strongg log-ciicavity
v2 log 2 milp, my inf rE(6.T)
The additional strong log-concavity yields the improved bound
Wi p O(hm)))
39

---

Published in Transactions on Machine Learning Research (02/2026)
Theorem D.6. Assume that t t V lo (x) is C' for each æ, and there crist A, B 2 0 such that
1 logp & + B||x||-
For 1j 2, if h(n 1) 1, we have
Wi(pr-sPPM $ Ohrr²)
Proof. Let
((1, 0) Va log (0g79))(22)) P/ax
Let (x(x) =: 21ss1 where is the solution of ODE
dæ
E tx+1) ((tt
(53)
dt
with initial value Itz x. By
we can see
X X = (Xn,)
Moreover, define the Fuler one-step map
Vx(x)= = x + -hb(lxx22 x),
so that Euler scheme
X k+1 = V/a (.)
Therefore,
Ck+1 == Xim.i X/ k41
(x(x,.) - vx(
(Xx(X 2n (9x - (x()))
Next, analyze these two terms respectively. (i) By the my-strong log-concavity, we have
V log V logr)- - 4) S 5m, ||: y11
Moreover, since P1 is orthogonal projection,
(ID Pa) - y). - y) = -| - y1 -
Pr - y)1l s 1| -
Therefore,
(b(t,z)) - b(t, y), x 8) S - -(m1 1|| - yll
Let It, Yr be the solution of with the initial value St. = x and yr. = y. So we have
d 1, y 2 2 (b(1, m.) - bil, v.). x. - y.) < 2(m 1)1la, - yel1?
dt
Then by the Grönwall's Inequality (Lemma H.11
| Vtuiall |l (mi
1 -1m--1)M1la
y/1.
which implies that
( +b(1))l < (mm ||: y/
(54)
40

---

Published in Transactions on Machine Learning Research (02/2026)
(ii) Fix x E R and let æ, be the solution of (33 with the initial value ai.
Note that by definition
de+1
(x) a + b/
b(1,0x))dl
Therefore,
/
A (b(t, -b(tx, x)) dt
(x(x) Vx(x)) =
xx-1
(b(t,zs)) b(t, dt ( (b(t,2)) b(tk. dt.
For above two terms, we analyze them respectively.
(a) Since 1 - nPillop = - (n > 2) and Va log Lg-Lipschitz continuous, b(l,)) :) K(n)- Lipschitz continuous for (()) = L.s + 1. So
16(t, - a(t, S K(n) 1/m | x||ds
Note that
I b(s, x. )ss s [ [iib(s, x1bss S (t (t-th)
| x
sup sEltu,t411
[b(s,
Therefore,
dxt1 [bb(t,,,) b(t, |s 1 h K(n)
sup
|ib(s, xz)Il.
(55)
For the right hand side, using same notation
31
let
sup TE(A.T
V log p. (0)l < cc.
It implies that [b(t, 0)1 < and so
[bbss xx)) || < (b(s, b(s 0) |bb(t. 0) s C K(n)lx,.1.
Let sup supsetan. [x,|| < oc. Using the similar idea as in the proof of Theorem in Appendix D.2
S = sup
||>.
C1 |0>|| C2
sElte At+1l
where C₁ exp Ls)h) and C2 are independent of 17. So
1)/ (1+ Ls) as shown in 33 and they
sup +1)
(bs S C + K(n)
sup (ttt
C2K(n) | S C] C +
(56)
Combining 55
and 56
we have
[b x;) b(t, ldt < h2
(GXK())\l]|+<CXXK) CK(())
(57)
(b) Since b(t, b(l,, x²) Va log 1o (.)) Va logrā (2)),
(b((t x(11, < / 110. log po - (t (x) (A+ Bl(+())
It implics that
1R12
[|b(t, x) - b(tR, x)||dt < h²
(A+ B|xx
(58)
2
t/
41

---

Published in Transactions on Machine Learning Research (02/2026)
Therefore, combining (57 and 688.
dx+1
(--1 [bb(t, - m11l
f/ne [1((t,)) b((e,, x))ldt
[1xx(=)) x= S
(59)
h²
(A C2K(1)² + (K())) +B (1111)11111))
2
Combining (54]
and 55 by setting x = x and = X
|extill ill s 1x (x)
Dx (x || + (x (x(x))
S (ms 1e- + h
2 (A+ C2K(n) + (k(()) + (B+ (B= x1I)
Let dg = [|]e+]]]- By the following Lemma expectation of above inequality, we have
D.7
bocause - 1) < 1, E x & M Taking the
h (A +CK()) + (B BB+ CKK()))M.)
a&+1 e (m) a)
ak + 2
Therefore, by coupling Xo =x0; i.e., an 0, we have
h²
N
aN =
2 (B
(myy-1)k
k=
( 1}h
s AA + C2K(1)) +CC]))*CK(+)+00) + (B+CK)))))
m.
It follows that as + and 17 + xc,
Wi (pT pkx) S EE [ xx1l1 s O(mmt).
Lemma D.7. For 1j > 2, if h(7 - 1) < 1, then
sup [|| s Me.
where Me is independent of n7. Proof. First, by construction,
x = (LD + +(ID - np()) --1+-(1DP))
x + hV log 1
Let M,-- In + MTD p)). Then because P1 is an orthogonal projection, there are only two eigenvalues
of MN: for E ker P1, M, (1 + h)a, and for E Im P1, M, (1 h(1 n))æ.. Because h(T 1) < 1,
+ h(1 - v) E [0, 1.. So
M = 1 + h.
Similarly, shown in
32
V a log (x < -
Therefore,
x + || S Ma X + h V logpt (xx)
< (1 + h(1 Ls)) X + Ch.
Taking expectations on the both sides, we have
R-1 E x s (1 (1++A11-0 Ls))* E |||x + Ch
Σ (1 h(1 + Ls))
0=
42

---

Published in Transactions on Machine Learning Research (02/2026)
e
E x Cta).
Because X ~ ~((,,D)) B X
/ | VD. Therefore, if let
Mc = e
+ CVT -6)
which is independent of then
E x & Me.
E Analysis for Assumptions
E.1 More Details for Orthogonality Assumption
For Assumption consider the case where A A2 + O, ic., Mi is not orthogonal to M2. In this case, A (A1, A2) Dxx, meaning that A * Ia and AA is no longer an orthogonal projection. We claim that this relaxation does not affect analysis regarding the guidance scale 7) Based necessary to examine its influence from three perspective: the smoothness and concavity of log (Section 5.1. the estimation of the target manifold M₁ (Section 5.2 and the distance between generated and target distributions (Section 5.3
results, it is
(a) Smoothness and Convexity: First, the results on the strong log-concavity of the latent density in Theorem are independent of the orthogonality A. Therefore, to analyze the smoothness and concavity of pf suffices to revisit the proof of Theorem Note that
X Va,AZ + v1 og ~ P..
By Proposition 3 Corollary 4 and the my -strong log-concavity of the latent density pe (Theorem obtain the following bounds:
s L L, = a: dA) + ar)mg V2 log (2) (o
(1 - -.)) + mm]1 ax))
and
(1 - amjm a,(AA Ax)
22 log (x) 2 m Ip. m =
(1 - an))(ada + mi ax))
where
Aa = || Amax (ATA), A), Xa = Amin (A (ATA) A).
Because A A2) and A, are orthogonal,
A =
Idj
) lat
C C.
-
(00 (o.). c= AT
O CT
C
It
( Let mmax (C) be the maximal singular value of C. Then, have
1 Fmax (C) < d S AA S 1 (C)
Moreover, because ||Cllop < $|AA1llop 11 A2llop 1, Fmax (C) < which implies that 0 AAAASAA
For smoothness, is clear that < I.A. < 00, so the non-orthogonality of A does not affect the L-smoothness of logrf, except that the constant changes from L, to L However, for strong log- concavity, requires m. > (Corollary D). which holds if
log m AA m XA
(60)
2
under the condition m 2 AA- This requires modification of Assumption III M 2 2, for the same reason discussed in the proof of Corollary
43

---

Published in Transactions on Machine Learning Research (02/2026)
(b) Estimating Target Manifold: Since Theorem 10 depends only on the L-smoothness of log and the geometric guidance model
(+ does not involve A, the result of Theorem 10 remains valid even when A
is not orthogonal.
(c) Distance to Target Distribution: Because the condition mf requires inequality 660 one can set
mi AA XA
log
m
and consider the geometric guidance model on interval [0, T - 6]. With this adjustment, the results in Theorem [T still hold, up to changes in certain constants. For instance, the non-orthogonality of A changes the bound on (I( Px 1)), specifically the constant C in Theorem
E.2 More details of Assumption
In the following, we demonstrate a family of distributions that satisfy both Assumption and Assumption Consider the density function pf of the distribution P?, given by the form:
(z) = V.(2)) (x)
where K; C Rdd is convex and compact set, and
(i((2)) 2 mla,-
In other words, p belongs to the class strongly log-concave densitics supported on convex and compact subsets of Rdd,
First, for such pf strong log-concavity on convex set does not perfectly align with Assumption which induces question of whether this property can substitute for Assumption in deriving the strong log- concavity of the mixture latent density p defined in Equation 11
In the proof of Theorem the strong log-concavity of p5 is inherited from that of the component densities p defined in Equation 12 which are shown to be strongly log-concave via Corollary under Assumption II In other words, the key question is whether strong log-concavity on a convex set suffices to replace the strong log-concavity condition in Proposition and thereby still allow us deduce the conclusion of
Corollary 4
Proposition E.1. Let Z be a random variable on R with the densily function x² given by
c
where K is conver set. Let B E Bank Assume there are mo A > 0 such that
VV(2) 2 mo
|B < A,
and X := Amin (BB)) 2 0. For E R and 3 0, let
X BZ 3E, & I,,,
with the density function px on Ra. We have
a²A
1
2 logpx(()
I
g2
Proof. By the same calculation, we have
a² B Cov n (2)B
1
2 log
I
34
g
44

---

Published in Transactions on Machine Learning Research (02/2026)
Note in this case,
duz(z) (2) Xx (x)=)dz
JK (y)dy
where
1
U2(z)
| Bz + ((z).
20²
Tt. follows that
a² g2
a² X
(2)(2)= B + V2) - mlk m + mg.
Instead of applying Lemma H.6 by using the Brascamp- Lieb Inoquality on a convex set [Bokkov Lodoux 2000. Proposition 2.1), we still have
Varp,(()s 1 Enn |1
m.
for any C function f: R** + R, which also indicates that
1
||Cov (Z) S
m
Then, the following proof as same as the proof Proposition same result
3 and in Corollary
so that we have the
a²A (2(022 + 32
v logpx(<)) 3
32 In.
Therefore, Proposition E.1 shows that, in our settings, Assumption I can be replaced Assumption
Assumption II'. For i- Pp admits the densily function of that has the form pf (z) eVVi(=) xx, such that K2 C TRd is conver and compact sel, and
V2)((z) 2 mla.
Next, verify if p3 in such class can satisfy Assumption Proposition E.2. For i 1,2, let of salisfy Assumplion o > 0, we have
If and let vÊ. defined by Equation 12 Fix
|K + [K21
sup V loggr (x) - Va log logr2 20 (a))
0²
x
where [K|] sup (1|2|| K,).
Proof. First, by the definition 12
pf.. (z) = (2mT²)-3) f exp
|| (-2221=
(=1.0)]11²) pf (z1)dz1. (=20
K3
Therefore,
1 JK, (z (21, exp 1-2 (21, 0) (2
V (xp(
(z)
V log
(2)
JK, cxp
20 112 - 112z-222,00)) ()T13) p? (z1)dz)
1
It follows that
V log (). σ² ((m1(2),0) - -z).
where
(- 2 - (z1, (0,,0)Ti) T)T of (z (2)
JK, 21 exe
m1(x)
JK, exp
|| - (z1, T)T 1f (21)dZ)
45

---

Published in Transactions on Machine Learning Research (02/2026)
Similarly,
1 G2 ((0m m2(2)) -z),
(2))) (z) V log logg2.0
for
JK, 22 exp
(-2a- || (z2, 0) T]T1() pz (22)
m2(z)=
JK. exp
2 ||= (z2, 0)T n2 (22)dzz
Therefore,
1
V log loggf, (2)
(z) - V- V=logp3. (z)]1 (=) m2(2))
02 v Il (z)]l + [
Note that
-2 2 - (z,, () pe (2) (z,)
exp
m²(x)= = Ezzz; [Z], (z.) ==
JK, exp
2 | (z,, 0) T)T pf (zi)dz;
By the convexity of K2 and Lemma H.10 m;(=) E Kü. Then, the boundedness of K₁ implies
Kg VKG |
sup
V (x) V log 12 (x)
r²
Sufficient conditions for Assumption
If p belongs to the class of distributions
{ (=-V=))x(( V 2 mI, K is compact and convex.
(61)
then pf.. given Equation 112 is strongly log-concave by Proposition E.1 Moreover, we choose a such that
V [Kg 2 + K2 M <
K1l + |22 4(m
22mm- -1
(62)
02
Proposition E.2 shows that Assumption is satisfied. Then the mixture latent distribution p given by Equation 11 is mo-strongly log-concave provided by Theorem which further implics that pf in the geometric guidance model satisfies:
V2 log (x) up < Ln
x logpl (x)) 2 mI
F Lipschitz Continuity of Score Function
If we only locus on the Lipschitz continuity the score function Va log where pt is obtained by a DDPM initialized from a distribution whose latent distribution admits smooth density function p², then the conditions in Proposition g can be relaxed. We consider two cases below.
The first case aligns with the setting considered in De Bortoli (2022). where suppp is assumed to be compact. We provide an alternative proof for this case, motivated by the argument used in the proof of Proposition 3
Proposition F.1. Let be a random voriable on g² with the density function p², and let d: R - R be continuous. Assume supp x compact. For
and B > 0, let
X = =a00220 (0)(2)++) + BE, $ ~ N( In),
with the density function px on B We have
R2
2 logxx(x))ll)))
<
g2 gr
for some conslant R>0 0.
46

---

Published in Transactions on Machine Learning Research (02/2026)
Proof. By the similar calculation as in the proof of Proposition 3
a² 3
1
V log
Cov pa (Ø(Z))
3 I..
which follows that
1 a² logpx( 3 32 B1
+
[[ (b()))llop)
To bound the second term, first by the definition of Pa; supp supppe. Because supp p is compact and Φ is continuous, d(supp px) is compact, which means that there exists a R > 0 such that
sup [1](())]1: z E suppyat S R.
Then, we obtain that for any u E R",
u (. (Z))u Varn, (u 2(Z)) Varn, (1lu|llbb() s R2||a]
which indicates that
IC (o(Z)) S
Therefore, we have
a R²
V2 log
S
g2
84
Remark F.1. This proposition shows that when the latent density function has compact support, no addi- tional conditions such log-concavity or L-smoothness are required for the latent distribution. Moreover, under the compactness assumption, the results of Proposition can be extended to the nonlinear case, shown in De Bortoli (2022).
Next, in the non-compact case, Proposition 3 requires the strong log-concavity of the latent density p² it is used to establish not only the L-smoothness but also the concavity of log Px (see Corollary E.. However, if we are only interested in the L-Lipschitz continuity of the score function, the assumption of concavity can be relaxed to the Lo-smoothness of logp", i.e., logp" < Lo, or even to the weaker condition V2 log p² (z) 3 L: see Proposition F.2 below.
Proposition F.2. Let a random variable on g² with the density function p² and B gank Assume there are Lo, such that
logpp LoIk.
|BB| SA,
and x Amia (B 0, the minimum of all eigenvalues of B B. For a and B > 0, let
= aB + BE, E~ I,,
with the density function px on R". a x Lo32 > 0, we have
a² A S g2 B2aa2> 2 - Ln532
22 lx)
+
and
a²A
1
2 log)x(#) 3
In.
Ba Ln,
132
Proof. The main difference of this proof to the proof in Proposition 3 is how to bound |[Cov (2) Mlop Note that
a² o 3
a²²> B2
Lo) Lo)
v²Un(2)-
BB 2
lx
because -V2logp²(2) [ZV(2)> -l When a² X Lo > 0, we similarly obtain
32
||Cov ns (2)1 op < 2 x Lnnz
47

---

Published in Transactions on Machine Learning Research (02/2026)
Therefore,
1 S
a²A
V2 log (x)
+
32
8²
Lob3)
On the other hand,
Ag
B Bov 12 (Z)B²
Lo Am
a²)
which follows that
a²A
1
2 logpx(x)) 3
In.
(3(a2x- - Ln,82)
/82
G More Details for Nonlinear Extension
G.1 Omitted Proofs in Section
Proof of Lemma 12 The proof consists of the following three steps. First, let z E Idd be arbitrary.
(i) Local construction: Because Φ: Rd M IRD an isometry, the columns of JØ(z) form an or- thonormal basis for the tangent space Mo These vectors can be extended to orthonormal basis of RD by adjoining
nD-a(=))
where each n, is smooth normal vector fields along M. For such n,, one can define the Fermi coordinates map as
D-d
F: x RP, F(z,0)= 6(z) + Σ (2)
Then, by the Tubular Neighborhood Theorem (Theorem for the set
L.3 there exists e: Rdd + (0, oo) such that
V = ((z v) E M M P-4,
||||| < {(2))
F: V ((=FF() is a diffeomorphism, where the open set U c RD is neighborhood of M, called a tubular neighborhood. Let m: Rd x D-d + Rd be the projection, ie. π(2,0) = z. Then, one can construct
か.: U Rd. (z(2) =
iii Check conditions: First, because M C U, and F is diffeomorphic from V to U with F(z,0) d(z),
(o(2)) (o(=)) T(2,0) =, Z
For the derivative condition, by the definition of F, we have
JF(2,0) 0) =
0)) = - n()),
where (n1(z),... ... (z)). By JΦ
J J Ia. JF 0) is orthogonal, which follows that
= (F(, JF(2 ((1o(()) )
[ō(z)
n(z)]
On the other hand, F-1 can be written as Fz) = (Fi (Fi F2(x)), where = = 3 on U. It implies that
J(F (F = (ō(()(z))
(F2()(z)) )
Therefore, (d()) T
48

---

Published in Transactions on Machine Learning Research (02/2026)
(iii) Global construction: By the Urysohn Lemma (Munkres 2018 there exists smooth function X: RR [0, such that xle = and xlæaAu O, where U C is open neighborhood of M. Let h: IR + Rd be any smooth function for instance, a constant function = c. Define
-=))= + (11xx(()))(())
x(x)))(x),
then the desired identities hold:
0 Φ idga, (
Proof of Theorem 13 Let Φ: R R be the isometry for defining M Im Φ. Because supp 1eppPxx Pxx C M,
there exists a pd defined on R' such that X = 2Z ~ Px when Z ~ Pz. Let t be fixed in (0, T7. By 2
X, va +v--aa
Define F²; PR + D by
F² (()) = Vans o 0=
at =e 21
where o² is defined in Lemma
12
Then we have
F'(X,) o Φ + V {)
04 0ig
X6 = (Z))
Now consider the Taylor expansion of (p = 60 00 at Xu = (Z), with integral remainder. We obtain
= F + vI
where R(E) denotes the remainder term.
Next, we analyze the three terms on the right-hame hand side one by one. For the first term, because X,- e Z) E M, Z = (Xo) by the definition of 6"; see the proof of Lemma 12 It implies that
F')))) Va ( = Va,Xo
For the second term, by Lemma 12
Jup(Xn) = (2)(Z)) =
Morcover, because JØ = Ia, P = J(p(X) is an orthogonal projection with rank d. For the third term,
1 - (' (1 s)D& (Xo + (1(11 ox)/oi)) [g, Elds.
R(E)
0z
By the proof of Lemma 12 Ø² = c on R U for tubular neighborhood U of M, which means Jø = 0 and
g² = on P
U. It follows that
D²p(=))u,
(1t el) =0
for R'U For chosen 8, can choose tubular neighborhood U sufficiently thin such that Xo + 8 / an)/ar & U for > 8. Therefore, we have
R(E) = 1-at
Vor ( (1 (((1-)) s)D2 Xo + 3o+ss(( (1/11aa)//// an)/art) Elds.
Assume D2vp bounded on U. Then, for any small é > O, one can choose 8 sufficiently small such that [R(E]]] E'.
49

---

Published in Transactions on Machine Learning Research (02/2026)
Combining these analyses, obtain
vI orl (ID P)Ell - 5 X; Fd (xx)) S vT an (ID PIE +
(63)
Let
f @ = | F (x)). Similarly as the proof of Proposition by the Laurent-Massart bound (Lerma
63
implies that
P (((t)) 1-00 ( S r(t)
2VE 2VE 21 2e -2(D-d))) 21-2--20--4
where r(1) V(D d) at). Because d < D, one can choose small such that 8= -2(D is also small enough. As result, (X)) r(t) 2 8, i.e., X, concentrates M ((ft)=(((()) with high probability.
G.2
More Results of Experiments
Comparison of FID.
Table 2
serves complement to Table 1
Table 2:
Comparison of FID on CIFAR-10
Airplane
Bird
Cat
Deer
Dog
Overall
CGM (17 = 1)
17.95
21.69
20.34
19.24
23.62
4.07
GeGM (5)
18.98
18.39
17.35
17.38
18.45
5.15
FID v.s. guidance scale on CIFAR-10. By sampling with the nonlinear GeGM 16 Figure 3 shows
how the FID varies with the guidance scale across all classes from CIFAR-10, which is consistent with the result of Theorem
100
airplane
+0dog
automobile
--frog
hird
+ *choss
cat
<hhhip
80
deer
--uucc
FID
60
40
20
5
10
100
Figure 3: FID guidance scale n of GeGM on all classes of CIFAR-10
H
Technical Lemmas
Lemma H.1 (Laurent & Massart (2000))- ). Let. be a -random variable with n degrees of freedom, i.e.,
X Σi ? with iid. 1). Then, for any O, we have
P(X 2 2 + 20x) -
PPX S no) S -a
50

---

Published in Transactions on Machine Learning Research (02/2026)
Lemma H.2. Let == be orthogonal decomposition of R", where V is linear subspace and VL is ils orthogonal complement. Let X, Y E I be random variables such that X E EV, Y E V+,, and X independent of Y. Suppose that X and Y admit densities and py on V and V+, respectively, with respect to the canonical volume measures on and L Then the density function of Z = + is given by
pz(z)= = ppx)==xx(x)pp@))00 (Q+=x)
where Q is the orthogonal projection onto V, and Q² = In - Q is the orthogonal projection onto V+,
Proof. Let my and my be the canonical volume measure on V and V+, respectively. Define Φ: V V- R by p(x, y) x + y. Clearly, & an orthogonal lincar map, which indicates |dct. J so
+ (my @ my+) = mn:
where my is the Lebesgue mcasure on R
Let Px and Py be the distributions of X and Y, respectively. Then dFx = pxdmy and dPy = pydmyy. By the independence of X and Y, we have
d (Px @ Py) = px px (m & my (y)).
Since Z X Y ((X, Y), it follows that Z ~ Pz + (Fxx & Py), and thus
[ (u(2))PZ(z))
P2((U)
& my- (y)
I xu(=)xx 1Q 2) (mv(w* @ my (y))
(Q
Therefore, we have
(x
0
Lemma H.3. Let (Wi)12a be slandard Brounian molion on I and A E Omxn Leb
B. = AT W.
Then (Br)22 slandard Brownian motion on R",
Proof. The path continuity of B, ATW follows directly from the path continuity of L + W., does the independence of increments. The initial condition Bo A Wo 0 is immediate. Moreover, since A E Omxx, we have.
B. B, AT - B,) NO (t Im), Vt>s.
Lemma H.4 Jost 2008 For function g: R" R" if R" Im g is a diffeomorphism, that is, both and its inverse g Im + Rn are continuously differentiable, then I#Px, the density function of 9 Txx on Im with respect to the canonical volume measure on Im g, satisfies
9+Px (v) -xx(x) det (Jo(z)Jg(x)) ))}²_ x x= -y)
Moreover, when g(x) Ax. for E Omxn Aupx(v) (AT
51

---

Published in Transactions on Machine Learning Research (02/2026)
Remark H.1. This result is essentially general form of the chango-of-variables formula, which has been widely used in the context of generative models on manifolds (see, e-g-, Loniza-Ganem et al. (2024)). To rigorously justify this result, some basic knowledge of Riemannian geometry is required. Since R" + Img is a diffeomorphism, the image Im g C R" is a submanifold. When Im g is equipped with the canonical Riemannian structure induced from the ambient Euclidean space IR", the canonical volume measure on Im g coincides with the Riemannian volume measure. Therefore, the relevant results from Jost (2008) Section 1.4) can be applied to establish the desired formula rigorously.
Lemma H.5. Let X ~ (N(O. I,) with large n. Then, with high probability, X approzimately uniformly distributed on the sphere (V-)(Vn); i.e., X (niT()*-((TT)).
Proof. First, consider Y homogeneous space:
X X
We first show that Y Unif(s Note that g is a compact
s--1 = SO(n)/ SO(n 1),
where (n(n) C Rax= denoles the special orthogonal group. Consider the natural action of SO(n) on g"-1 given by R: gn-1 7 g--1 + Ræ for all R E SO(n). Then by the existence and uniqueness of Haar measure Folland 2016 Theorem 2.49), Unif(S is the unique rotation-invariant probability mcasure on
gm-1, Therefore, it sufficient to prove that the distribution of Y rotation- inatiioniivvaiant, i..,, Y _ RY for R SOnn).
Since X ~ ~N, I,) and R E SO(n), have RX
NO, I.) and RX | Hence,
X
RX
Y
RY.
X
RX
which implies that Y Unif(S"-!), Similarly, by the uniqueness of the invariant measure,
vn x
VnY
U
X
Moreover, shown in the proof in Proposition high probability when n is large. Therefore,
the
Bound implies that |X|| 2 v with
X
X Unif(S"-I( (Vr)).
Lemma H.6 (Corollary 4.8.2 of Bakry et al. > 0. Then the probability measure
2013
Let U: R" + R be C2 such that V²U 2 pI, for some
c ((x)
dp(x)) fe- dæ
on Ra satisfies the Poincaré Inequality with the constant 1/p.
Lemma H.7. Let n. P(RR) be two probubilily measures, and lel. J RR - R be measurable. Then
fu wzy) = wi wwffH + w2
for any wi, wa E [0, 1] with wi w2 1.
Proof. For any A E B(R"),
f ( (A) (iiM + wzy) (A))
WIH (A)) + uzy (A-)
wi1 A) W2 w (A))
52

---

Published in Transactions on Machine Learning Research (02/2026)
Lemma H.8. Let j be probability measure R Then, we have
Ex~ ||| S V Ex~~ 1
In porticular, p= (O, I.);
E [ Vn.
Proof. Because y is probability measure, by the Hölder's Inequality,
| idp(x) & |1el| [Ill(dpptt))) ) # (/ 1du((z)) 4
3m
that is, Ex~μ ||] $ V (x-mxlll | In particular, if u= NO (YOO.,.). In), |||||| n.
Lemma H.9. Let ia, 12, Mi, be probability measures on R,, and let
wpI (1 -w) v w (1 w)22. E 10.
Then, we have
Wi v) S wii( (1 (H2,
Proof. By the existence of optimal coupling g. such that
(Chewi et al.
2024h, there is 7 E L'(uss for 1,2
Wi(us, = f xR"
|| yl
Let
π == w + (1 - w)72-
Clearly, is probability measure R x R Moreover, by definition,
#(A x Rd) w// (A x R") w))2((22 A RR" wyyh (y)(1) (A) (1 () u(A),
TR" B) w/y (R" B) + (1 - (R" x B) wv (B) (1- w)- v(B),
which means π E v). Therefore,
W((mv) - [ JRAXRN
| yl/dT(0, y)
| yyld71 (x. 3) + (1 w) / B=
||x 211d72(xx y)
XR"
wWi(mn 11) + w) (12 v2).
Lemma H.10. Let μ E R" a probability measure such that its support K is closed and convex. Then
E
Proof. Suppose that m = Ex X & K. By the convexity and closedness of K, the strong separation
thcorem Rockafellar 1997) implics that thore are u E R² {0) and c E such that (u, m) > c and
(u. x)) S c, V x E K. Let X m. X K for almost everywhere and
X) < c, a.e.
Then taking the expectation on the both sides, we have
(u, m) - c,
which induces contradiction.
53

---

Published in Transactions on Machine Learning Research (02/2026)
Lemma H.11 (Grönwall's Inequality). u: [0, + R satisfies the linear ODE inequality
d di m(1) ((1))(((++b)) b(1):
then
u(t) s v0)ee a(r)dr + S b(s)edi a(t)drr ds.
Proof. Let (tt)= exp (- So a(s)ds). Then, ('()) (t(t)((tt) and u(t) - ((t)a(t)u(() + ((t)b(t) = dt
d ( ((t)b(().
(tt) dt
By integrating on the both sides of above inequality, have
u(t) - v0)eeo a(r)dr
a(r)ar
+ M(ssee a())dr ds.
Lemma H.12. If C function f: R R is p-strongly conver, then it satisfies p-Polyak- Lojasiewicz (PL) inequality:
[V 2p( f(x.)),
where x, is the unique minimizer of f.
Proof. Because f is p-strongly convex,
f(y) 2 f(x) (V=f(x), x) Gily x|12
Minimizing the both sides with respect y, we obtain
f(x.) 2 f(x) - 2p
VV (x)))
which is precisely the p-PL inequality.
Lemma H.13. Let f: R E= be L-Lipschitz continuous. For two probability measures μt, v E P(RR),
Wi(S#u, S LWi(U,
Proof. Let (X,Y) be an optimal coupling for (H, that is, X ni Y u, and Wi = =E[[]x X YII. Besides, f(X) ~ f# and f(Y) ~ fav. Then, by the Lipschitz continuity of f,
(1 < (I) - f(X)
- LE [1IX - YII
= (H,
Preliminaries for Manifold
We provide only the minimal background on smooth manifolds necessary for this work. For a comprehensive treatment, we refer the reader to Lee (2012).
Definition I.I. A subset M C R is called a m-dimensional (embedded) (sub)manifold of RD if there are a family open sets {Uabaer R", a family of open sets {VaJaer in RT, and a family of smooth (Co)) maps (do (orra such that
MC U Un, and d.: V. Un NM aer
is diffeomorphism, i.e., Φ-ト: Ua N.
Va also smooth.
54

---

Published in Transactions on Machine Learning Research (02/2026)
Each pair (da, Va) is called chart, and {(oa: Va) is called an atlas of M. In general, single chart cannot cover the entire manifold M. However, if M is closed, then there exists chart o: V + M that can almost cover M, in the sense that the volume measure of the set M \ (VV) is zero; see Lee (2019) for more details.
Definition 1.2. Let M C I be a m-dimensional manifold. For any a E M, the langent space, denoted M,,, a vector space defined as
T:M: = (7(0): = 0, (-E,5) M smooth, 7(0) = =
Lemma L.1. Let M C R" be smooth submanifold. If C function g: R + RR is constant on M, then for any x E M, Vg(x) is normal to M; that is, Vg(æ)) 1 T.M.
Proof. For any v E M, let 7: 10, + M be smooth curve such that 7(0) æ and 7'(0) - v. Then, because g(>(t)) = c,
d
s()(t)) = (Vo()(0). (Vg())))))))))))))))))>>))))))>) (V)()),))
dt 1=
Therefore, Vg(x) M.
Theorem I.2 (Constant Rank Theorem Lee 2012 Let f: IR" F be smooth map and E R', Let
M ( E R f(x) = c)-
If rank (F(x)= = for any æ E M, then M is (n r) dimensionall manifold.
Theorem I.3 (Tubular Neighborhood Theorem Lee 2012 LetM C TRD be a d-dimensional submanifold. There smooth e: M + (0, 20) such that for
||||| < <<(z)}
F: V.V-U= F(V) is a diffeomorphism and U C TRD is a neighhorhood of M.
Remark I.1. For a given tubular neighborhood V of M, we also call U = F(V) C RD is its tubular neighborhood in D Moreover, we can define the corresponding orthogonal projection x: U + M as
(Φx) = (F
where mi V + MM is TI (2,
55


Reviews:
[{"review_id": "jSFCIIaB5l", "reviewer": "Reviewer_bkAk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Classifier guidance and classifier-free guidance are central techniques in diffusion-based generative modeling. In practice, the conditional score term $\\nabla_x \\log p_t(y\\mid x)$ is not analytically tractable and must be approximated (e.g., via an auxiliary classifier or an implicit CFG construction). Despite its widespread use, the role of the guidance scale $\\eta$ is strong empirically but remains theoretically under-explained. Motivated by the fact that practitioners routinely tune $\\eta$, this paper aims to provide a geometrically interpretable perspective on guidance and to derive bounds that clarify the effect of $\\eta$.\n\n\n**Geometric reinterpretation of probabilistic guidance**\n\nTo pursue this goal, the paper adopts a linear manifold hypothesis. Specifically, it assumes that the data for each class live in a linear subspace spanned by the columns of $A_i$ for the $i$-th class. Under this hypothesis, the paper argues that the conditional guidance term $\\nabla_x \\log p_t(y\\mid x)$ acts (approximately) in the normal direction of the noisy class manifold, and can therefore be replaced by a simple geometric vector field $-\\eta P_i x$ with $P_i = I - A_iA_i^\\top$.\n\nUsing this replacement, the authors define a “geometric guidance model” and prove that as $\\eta$ increases, samples become closer to the target class manifold (Theorem 10). They also provide a coarse $W_1$ upper bound to the target conditional distribution containing an $O(\\eta^{-1})$ term (Theorem 11).\n\nNote that the proposed geometric-guidance ODE analysis requires regularity of the score field $\\nabla_x \\log p_t(\\cdot)$ (existence as a function, Lipschitz-type control, etc.). In particular, for a purely manifold-supported distribution, densities can be singular; and in multimodal settings, scores can behave poorly. In this regard, the authors explicitly introduce mollification to “ensure existence of density,” avoid non-smoothness issues, and obtain strong log-concavity-type properties needed for the analysis.\n\nFor real data, they propose learning implicit functions $F_\\theta^t$ so that noisy manifolds are approximated as level sets $M_t = (F_\\theta^t)^{-1}(0)$, and then using $\\nabla_x F_\\theta^t$ as the geometric guidance signal.\n\n**Nonlinear extension via learned level sets.**\n\nBeyond the linear subspace setting, the paper assumes a union-of-manifolds hypothesis: each class $y$ lies on a nonlinear manifold $M_y \\subset \\mathbb{R}^D$. At a fixed diffusion time $t$, the class-conditional noisy distribution is argued to concentrate near a $(D-1)$-dimensional hypersurface that can be represented as a level set $M_y^t = ( x : f_y^t(x)=r(t) )$. Since $\\nabla f_y^t(x)$ is normal to this surface, the paper proposes replacing probabilistic guidance by a normal-direction term and obtains a nonlinear geometric-guidance ODE with $-\\eta \\nabla_x f_y^{T-t}(x)$. Because $f_y^t$ is unknown, they learn an implicit function $F_{y,\\theta}^t$ from forward-noised class-$y$ samples such that $M_y^t \\approx (F_{y,\\theta}^t)^{-1}(0)$, and then use $\\nabla_x F_{y,\\theta}^t$ as the guidance signal.", "claims_and_evidence_yn": "Yes", "claims_explanation": "**Strengths (well-supported):**\n\n- The paper’s *internal* theoretical claims about the geometric guidance surrogate are supported by formal statements and proofs under explicit assumptions: e.g., Theorem 10 gives an $\\eta$-dependent convergence rate of the off-manifold component, and Theorem 11 gives a corresponding Wasserstein bound.\n- The need for a surrogate / mollification is explained clearly (lack of density for manifold-supported distributions; multi-modality causing irregularities), and the mollified distribution’s closeness is quantified.\n\n**Concerns (claims may be overstated relative to evidence):**\n\n- The abstract and narrative sometimes read as if the theory explains practical guidance scale in real conditional diffusion, but the core proofs are for a modified dynamics where $\\nabla_x \\log p_t(y\\mid x)$ is replaced by $-\\eta P_i x$. This is acknowledged as a replacement made “to avoid the difficulty” of analyzing $\\nabla_x \\log p_t(y\\mid x)$, but it weakens the directness of the conclusions for classifier/CFG guidance.\n- Theorem 11 includes an additive constant $\\tilde{C}$ in the $W_1$ bound, meaning the bound does not go to zero as $\\eta\\to\\infty$; this limits how strongly one can interpret the result as “increasing $\\eta$ improves generation performance”.\n- The well-posedness results with mollification rely on assumptions that are both strong and somewhat hard to validate for realistic diffusion latents.\n- The paper suggests that large-$\\eta$ degradation is likely due to discretization stiffness rather than the continuous-time formulation; this is presented as a hypothesis and supported by citing prior empirical observations, but it remains not fully tested within the paper via a targeted numerical study (e.g., step-size sweeps at fixed $\\eta$).\n- The nonlinear extension is conceptually appealing but still largely heuristic: it introduces an additional learned object $F_\\theta^t$ and uses $\\nabla F$ for guidance, but there is limited analysis connecting level-set learning error to sampling error.\n\nOverall: the evidence convincingly supports the paper’s claims about the proposed geometric surrogate model; it is less convincing if interpreted as a theory of standard classifier/CFG guidance without additional bridging arguments or experiments.", "audience_yn": "Yes", "audience_explanation": "Some TMLR readers would still be interested, but the appeal is likely narrower and more conditional than in a “direct CFG theory” paper.\n\nOn the positive side, guidance (classifier/CFG) and the tuning of (\\eta) are ubiquitous in practice, and the paper offers a clean, geometry-driven lens (“guidance as normal-direction manifold contraction”) with formal $\\eta$-dependent statements—so researchers who care about *analyzable surrogate models* for conditional sampling, manifold hypotheses, or Wasserstein-style stability bounds may find the framework and proofs informative.\n\nHowever, the interest may be substantially reduced for a large portion of TMLR’s applied diffusion audience because the main theorems are not about the actual guidance mechanisms people use: the paper replaces $\\nabla_x\\log p_t(y\\mid x)$ with a hand-crafted proxy (projector-based regularizer), introduces mollification (changing the target), and still yields a bound with a non-vanishing constant floor. In addition, its optimistic framing of large (\\eta) can be seen as incomplete or potentially at odds with other analyses emphasizing failure modes under score error. In that sense, readers seeking operational insight comparable to more practice-facing work (e.g., Autoguidance-style explanations/interventions) may find the contribution less direct, whereas theory-oriented readers may still value it as a principled but idealized proxy study of guidance scaling.", "requested_changes": "- Tighten/clarify claims and scope.\n    - Rephrase statements like “increasing (\\eta) improves generation performance” to explicitly mean “improves manifold alignment under the geometric surrogate,” and discuss the role/interpretation of the constant $\\tilde{C}$ in Theorem 11.\n- Stronger bridge to real guidance (CFG / classifier guidance).\n    - Add a section that more carefully maps the geometric proxy $-\\eta P_i x$ to properties of $\\nabla_x \\log p_t(y\\mid x)$ beyond the “approximately constant on manifold” heuristic (e.g., conditions under which the proxy is a controlled approximation).\n- Empirical diagnostics of the “large-(\\eta) degradation is numerical” hypothesis.\n    - Provide an explicit experiment varying discretization step size at fixed large $\\eta$, showing whether quality/diversity degradation is mitigated as hypothesized.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "T0ukGrJX42", "reviewer": "Reviewer_vsZJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the problem of controlling diffusing models to generate conditional distributions, where, importantly, a guidance term appears. The authors propose a new linear geometric guidance term, assuming a linear manifold hypothesis, which requires certain regularity conditions. To ensure these conditions are fulfilled, the authors use mollification technique to construct a surrogate score function. Finally, the method is extended to nonlinear settings. \n\nOverall, the paper is well written and really easy to follow. Although the paper is sometimes math-dense, the authors did a good job of introducing each term with enough details. Moreover, if more information is needed, the appendix is well structured and includes substantial extra details about the results. \n\nHowever, I think the experiment section felt short; it lacks details and discussion about the results, and it is not convincing.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in the paper are generally supported by theoretical results", "audience_yn": "Yes", "audience_explanation": "Yes, studying the problem of controlling diffusing models to generate conditional distributions is of interest to the community.", "requested_changes": "I am not an expert in this topic, so the following points should be viewed more as suggestions to improve the work rather than critical adjustments:\n\n1. I think it would be helpful to add a discussion about the choice of $\\sigma$ in the mollification; there is no analysis about the impact of this hyperparameter in the method.\n2. I am wondering if there is any scenario where considering the linear geometric term would be useful? If so, I think it would be interesting to do an experiment to showcase this, since most of the paper focuses on the linear manifold assumption.\n3. In my opinion, since it's a novel assumption, we authors should provide more intuition about assumption 3. In particular, it is important to discuss how restrictive this assumption is and to at least give a simple example in the main text when this assumption is fulfilled.\n4. I think more details in the experiments are necessary. For instance, it is not clear which dataset is used in the first experiment; I suppose it's the CIFAR-10, but there is no information about it in the main paper. Moreover, in Table 1, why is the guidance scale selected as $\\eta=5$ and $\\eta=50$?.  Figure 2: Is there any intuition why the FID increases for $\\eta=100$? Overall, I think the experiment section needs more details and discussion of the results. \n5. Why do the authors set $\\kappa=1$ for eq (16)? Could give more intuition about it?\n6. One minor comment, I think the proofs in the main text are not necessary. They make the paper longer and are not particularly insightful. Therefore, I would suggest moving them to the appendix.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mdLMfPEhZW", "reviewer": "Reviewer_xZ9X", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\n\nThe paper provides a geometric viewpoint on conditional diffusion guidance by modeling class-conditional data as lying near low-dimensional manifolds. Under this interpretation, classifier and classifier-free guidance correspond to pushing samples in the normal direction of the target class manifold to offer an intuitive explanation of how the guidance scale $η$ influences sample alignment and fidelity. The authors introduce a geometric guidance ODE that replaces probabilistic guidance with an explicit normal-direction projection toward the target manifold. In the linear case, this reduces to the analytically tractable form $-η(I - AA^T)x$. \n\nTwo key theoretical results are presented:\n\n1. Theorem 10: Increasing η contracts samples exponentially toward the target manifold:  $E[\\text{dist}(X, M_y)] \\leq e^{-\\eta} + \\frac{1}{\\eta}$\n\n2. Theorem 11: The Wasserstein-1 distance between the geometric guidance model (GeGM) and the true conditional distribution satisfies\n   $W_1 \\leq O(e^{-T} + \\sqrt{\\delta} + \\sigma + \\frac{1}{\\eta}) + \\tilde{C}$\n   implying that larger η does not worsen the theoretical error bound.\n\n### Strengths\n\n1. The geometric interpretation provides a clear and intuitive explanation of the role of guidance in diffusion models.\n\n2. The theoretical analysis is rigorous and shows that increasing η moves samples closer to the target manifold without worsening the distributional error.\n\n3. The proposed geometric ODE gives a principled alternative to probabilistic guidance.\n\n### Weaknesses\n\n1. As shown in Theorem 11, the bound includes a non-zero constant $C̃$, implying that the nonlinear geometric guidance model cannot exactly match the true conditional distribution.\n2. Transition from probabilistic to geometric guidance lacks mathematical rigor. The claim that $\\nabla_x \\log p_t(y|x) \\approx -\\eta P_1 x$ relies on informal arguments about approximate constancy without explicit error bounds.\n\n3. Experiments are limited to CIFAR-10 and do not provide direct comparisons to standard probabilistic guidance, making it difficult to assess relative effectiveness.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The submission provides strong theoretical support for its core linear-manifold claims, including score decomposition for manifold recovery (as discussed in Theorem 2) and the stability of mollification under Wasserstein and log-concavity arguments. \nOverall, while the foundational linear theory is solid, the paper still contains specific gaps, particularly in justifying geometric guidance rigorously and extending theory to the nonlinear case.\n\n### Identified Gaps in the Submission\n\n1) **Geometric Guidance Justification**  \nThe claim that $\\nabla_x \\log p_t(y=1 \\mid x)$ is “almost normal’’ to $M_1^t$ (implying $-\\eta P_1 x$) is only supported by Lemma I.1, which assumes exact manifold constancy. The paper does not quantify $\\|\\log p_t(y \\mid x) - c\\|$ on $M_1^t$, so the accuracy of this approximation is unclear.\n\n2) **Assumption III Validation**  \nAssumption III ($M < 2\\sqrt{m-1}$) is validated only for a synthetic log-concave family, with no empirical check on CIFAR-10 or method to estimate $M$ from data. Thus, it is uncertain whether real image distributions satisfy the conditions required for the theoretical guarantees.\n\n3) **Limited Empirical Validation of Nonlinear Theory**  \nThe nonlinear guidance method (Section 6) is evaluated only on CIFAR-10, which is relatively simple and low-dimensional. There is no experimentation on more complex or highly nonlinear datasets (e.g., ImageNet subsets, CelebA-HQ, or natural image manifolds with nontrivial curvature), making it difficult to assess whether the nonlinear formulation reflects the intended geometric principles or generalizes beyond a single dataset.", "audience_yn": "Yes", "audience_explanation": "TMLR focuses on machine learning theory, and this work touches several active directions:\n- **Diffusion Theory:** Extends prior 1D or GMM-based analyses with a more general geometric framework.  \n- **Generative Modeling:** Offers theoretical insight into conditional diffusion guidance.  \n- **Manifold Geometry:** Provides a geometric interpretation linking score fields and class manifolds.  \n- **Analytical Tools:** The mollification construction is technically interesting and potentially reusable.\n\nThese concepts are of great interest to researchers studying diffusion model theory, conditional guidance, or geometric generative modeling.", "requested_changes": "**Formalize the Geometric Guidance Derivation**  \n   Provide a quantitative approximation guarantee such as  \n   $\\|\\nabla_x \\log p_t(y=1 \\mid x) + \\eta P_1 x\\| \\le \\epsilon(t)$  \n   where $\\epsilon(t)$ depends on manifold curvature, classifier smoothness, and deviation of $\\log p_t(y=1\\mid x)$ from constancy on $M_1^t$.\n\n**Characterize and Validate Assumption III**  \n   - Give sufficient conditions for when $M < 2\\sqrt{m-1}$ holds.  \n   - Provide concrete examples (e.g., Gaussian mixtures, truncated Gaussians).  \n   - Propose a practical estimator for $M$ using observed data.  \n   - Empirically verify the assumption on CIFAR-10.\n\n**Quantify the Error Floor in Theorem 11**  \n   - Report empirical values of $\\tilde{C}$ on CIFAR-10.  \n   - Compare $\\tilde{C}$ to $e^{-T}$, $\\delta^{1/2}$, $\\sigma$, and $\\eta^{-1}$ to evaluate practical relevance.  \n   - Discuss whether the error floor is fundamental to geometric guidance.\n\n**Provide Theory for the Nonlinear Case**  \n   Either introduce convergence or stability guarantees for the nonlinear flow, or clearly state that it is intended as an empirical contribution only.  \n\n**Expanded and Fair Experiments:**  \n  Ensure comparable guidance strength between CGM and GeGM.  \n  Add ablations on $\\eta, \\sigma, \\delta, \\kappa, M$.  \n  Critically, evaluate nonlinear guidance on harder datasets (e.g., ImageNet subsets, CelebA-HQ).", "broader_impact_concerns": "The broader impact of this work centers on both positive and negative implications of improving theoretical understanding of conditional diffusion models. On the positive side, stronger guidance mechanisms may enable better synthetic data for addressing class imbalance or improving data efficiency. However, the same improvements could also facilitate the generation of sensitive or undesirable content if misused. Additionally, the reliance on linear manifold assumptions may not hold for highly structured datasets, potentially leading to biased or unpredictable behavior in real-world scenarios.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)