The Geometry of Phase Transitions in Diffusion Models: Tubular Neighbourhoods and Singularities
Authors: Manato Yaguchi
Kotaro Sakamoto
Ryosuke Sakamoto
Masato Tanabe
Masatomo Akagawa
Yusuke Hayashi
Masahiro Suzuki
Yutaka Matsuo
Venue: Accepted by TMLR ()

Abstract: Diffusion models undergo phase transitions during the generative process where data features suddenly emerge in the final stages. 
The current study aims to elucidate this critical phenomenon from the geometrical perspective. We employ the concept of ``injectivity radius'', a quantity that characterises the structure of the data manifold. Through theoretical and empirical evidence, we demonstrate that phase transitions in the generative process of diffusion models are closely related to the injectivity radius. Our findings offer a novel perspective on phase transitions in diffusion models, with potential implications for improving performance and sampling efficiency.

Published in Transactions on Machine Learning Research (07/2025)
The Geometry of Phase Transitions in Diffusion Models: Tubular Neighbourhoods and Singularities
Manato Yaguchi"
monato. hwtp://oooooodddoooo.o.oo.o.....
Graduate School of Engineering, The University of Tokyo
Tokyo, Japan
Kotaro Sakamoto'
kotaro. kotaro. htt.:r/..cccmoo.....bbb....oo...
Graduate School of Engineering, The University of Tokyo Tokyo, Jopan
Ryosuke Sakamoto"
Department of Mathematics, Hokkaido University Sapporo, Japan
Masato Tanabe'
tanabe. ac.jp
Deportment of Mathematics, Hokkaido University Sapporo, Japan
Masatomo Akagawa"
akagguur..assatomo... 1 wwwelmms.ooodaai.co ac.jp
Department of Mathematics, Hokkaido University Sapporo, Japan
Yusuke Hayashi
hayashi@uuialign.ne
AI Alignment Network Tokyo, Japan
Humanity Brain
Tokyo, Japan
Masahiro Suzuki
masa@@webaab....utoooo....
Graduate School of Engincering, The Universily of Tokyo Tokyo, Japan
Yutaka Matsuo
matssu@@ucchabb..ootoooo..ooo
Graduate School of Engineering, The University of Tokyo Tokyo, Jopan
Reviewed on OpenReview: https: opt.reveeppr.vee....m net/ forum? id= Ph
Abstract
Diffusion models undergo phase transitions during the generative process where data features suddenly emerge in the final stages. The current study aims to elucidate this critical phenomenon from the geometrical perspective. We employ the concept of "injectivity radius", a quantity that characterises the structure of the data manifold. Through theoretical and empirical evidence, we demonstrate that phase transitions in the generative process of diffusion models are closely related to the injectivity radius. Our findings offer a novel perspective phase transitions in diffusion models, with potential implications for improving performance and sampling efficiency.
Equal contribution
1

---

Published in Transactions on Machine Learning Research (07/ (07/2025)
1 Introduction
Generative models BBond-Taylor et al., 2022 address the fundamental challenge of approximating and
sampling from complex probability distributions. Diffusion models (Sohl-Dickstein et al. 2015 Ho et al.,
2020: Song et al. 2021 J. a prominent class of generative models, incorporate two primary processes: a forward (diffusion) process, wherein data points are perturbed by incrementally adding noise to the data, mapping a complex distribution into an analytically tractable prior distribution, and a backward (reverse diffusion) process, where noise is denoised back into sample from the data distribution by reversing the noise perturbation. The reverse process involves estimating the "score vector", the gradient of the log-density of the perturbed data distribution, typically using neural networks. Despite their remarkable empirical success, a comprehensive theoretical understanding of the geometric and topological mechanisms underlying these models remains elusive.
Recent findings reveal that diffusion models undergo a sudden emergence of distinctive data features
often
described as "critical phenomenon" -during the generative process (Ho et al. 2020 Meng et al.
2022
Choi
ct al.
2022
Zheng et al. 2023
Raya & Ambrogioni 2023 Georgiev al. 2023; Sclocchi et al.]
2024
Biroli
ct al.
2024 Li & Chen
2024 Li et al. 2025). We refer to this abrupt change
"phase transition" (scc
Figure for CIFAR-10 example at around step 400). Elucidating these phase transitions helps to distinguish
between irrelevant noise and the distinctive features that emerge from the model's learned distribution, offering valuable insights into optimizing the sampling process and guiding the design of conditional diffusion models (e.g., language-conditioued image generation). Although multiple studies have empirically reported such critical behaviours, theoretical frameworks for explaining the underlying geometry remain limited (Raya) & Ambrogioni 2023). In particular, Raya & Ambrogioni (2023) proposed a local energy analysis restricted to simpler data manifolds (e.g., hyperspheres). In this work, we extend these ideas to more complex geometries via the concept of "tubular neighbourhoods," aiming to shed light on phase transitions in broader settings (see Section
Building on this observations, we propose a novel geometric interpretation of phase transitions in diffusion models, grounded in the behaviour of the score vectors. As demonstrated prior research (Stanezuk et al. 2024p, the score vectors at the final time step of the generative process are orthogonal to the tangent plane of the underlying data manifold. This orthogonality implies that the diffusion dynamics effectively push noisy data points toward their nearest locations on the manifold at the final stage of the generative process. However, such nearest points need not be unique if the manifold has complex curvature, and the uncertainty of generative trajectories generally increases as one moves farther from the manifold. To formalize this geometric view, we leverage the concept of the "injectivity radius" - the supremum distance within which the nearest. point on the data distribution is always uniquely determined. We define the region within the injectivity radius as the tubular neighbourhood (Fig. D. We hypothesise that the boundary of this neighbourhood plays a crucial role in the generative dynamics and is intimately connected to phase transitions. In the next. sections, we test this hypothesis through synthetic experiments and thcoretical analysis.
To test our hypothesis, we conduct experiments using synthetic data and demonstrate that, under conditions of constant curvature, the hypothesis holds true. In contrast, in scenarios where the curvature of the data manifold is non-constant, singularitics corresponding to varying curvatures can emerge, leading to the possibility of multiple phase transitions. Moreover, we show that the concept of the tubular neighbourhood corresponds to the final phase transition in the generative process. Finally, we experimentally demonstrate that by embedding the original data distribution into hypersurface, the theory of the tubular neighbourhood can be leveraged to achieve more efficient sampling. Our code can be found at
Contributions
As a core part of our geometric approach, we propose an algorithm to estimate the injectivity radius of the data manifold (Section 3) This allows us to define and quantify the tubular neighbourhood, providing practical tool for characterizing the manifold's geometric properties in diffusion models.
We analyse the diffusion dynamics through the theory of tubular neighbourhoods and empirically demonstrate that phase transitions occur around these regions (Sections and 5 This combined
2

---

Published in Transactions on Machine Learning Research (07/202) 2025)
injectivity radius
the projection of tubular neighbourhood
Vx, In (xre)
the generation process
Figure Conceptual diagram of our perspective. The orange path represents the generative process from Gaussian noise to the data manifold M. A singularity occurs at the endpoint of this path. The grey region represents the tubular neighbourhood of the data manifold M. We hypothosise that transitions of particles within the grey region play crucial role in the generative process.
Figure 2: Example of phase transitions in CIFAR-10 late-Initialisation gener- ation.: Around step 400, the generated images undergo marked change in qual- ity-an an event we associate with the critical phenomena of phase tramsitions or symmetry breaking where distinctive data features begin to emerge from the initially random patterns.
theoretical and experimental approach strengthens geometric interpretation of diffusion models and offers a potential method for optimising sampling efficiency by identifying critical points in the generative process.
2
Preliminaries
In this section, we briefly introduce some basic mathematical concepts related to the paper.
2.1
Diffusion models
In Song et al. 2021 score-matching (Hyvärinen 2005) and cirtusion-bascee generative models (Sohl- Dickstic et al.) 2015 Ho et al. 2020) have been unified into a single continuous- time score-based framework where the diffusion driven by a stochastic differential equation (SDE) or Langevin dynamics. this context, I2 E Rd represents the data at time 6, which evolves through time E [0,T], T]. This framework relies on Anderson's Theorem [Anderson] 1982), which states that under certain Lipschitz conditions on the drift coefficient f Rd x Rd T and the diffusion coefficient, Rdd R Rd Rdd and an integrability condition the target distribution po(xo), forward diffusion process governed by the SDE
dr, ft + gi (xe)dw.
(1)
has reverse diffusion process governed by the SDE
dx. -- -([((te)) (x(2))2
V.. In Inpe(are) dt go (x.))
(2)
2
3

---

Published in Transactions on Machine Learning Research (07/2025)
where Wa is standard Wiener process in reverse time. The probability distribution pz(x) of the SDE satisfies the Fokker-Planch equation
p((x)) = Vv (p(()))f((za)) + 1 مa (=)(
(3)
Diffusion models are trained by approximating the score function Va, papr( with a neural network sy sø(2,, (1,,)) parametrised by 0.
2.2 From the Manifold Hypothesis to Tubular Neighbourhoods
Empirical studics show that data obscrved in high-dimensional spaces often concentrate near a manifold of much lower intrinsic dimensionality, a claim commonly referred to as the manifold hypothesis. Early support came from non-linear dimensionality-rccuctoo techniques such as Locally Linear Embedding and Tenenbaum al., 2000). Subsequent theoretical analyses (e.g. Fefferman et al. 2013 Siniza-Ganem al. (2024)) have formalised testable conditions for the existence of such manifold structure.
Isomap (Roweis & Saul 2000
Assumption 2.1 (Compact smooth data manifold). Throughout this paper assume that any data manifold M is compact C submanifold of the Euclidean space I& with the standard metric.
Why embrace the hypothesis? Accepting manifold regularity provides powerful geometric guarantee: any sufficiently small neighbourhood of the manifold becomes a tubular neighbourhood, that is, every point in the neighbourhood admits unique saarest-point projection onto the manifold. the data support fails to satisfy the hypothesis one may still appeal to the theory of stratified sets and their associated tube systems; however, such objects possess corners or singular strata that render the normal-projection map far more complicated and, in practice, computationally unusable.
Scope. Assumption 2.1 is indispensable for the differential-geometzii machinery developed below. Specif- ically, it guarantees that the manifold admits a well-defined tubular neighbourhood and unique normal projection map within radius R(M). Real-world datasets, however, may violate these regularity conditions because their underlying distributions often exhibit sharp edges, occlusions and other non-manifold artefacts. We therefore regard the forthcoming analysis as most reliable for data manifolds that satisly (or can be embedded into spaces that satisfy) the above conditions, such as the synthetic examples in Section
Tubular neighbourhoods in nutshell. A tubular neighbourhood of manifold is roughly speaking set of points near the manifold and every point of the set has unique projection onto it (see Appendix C.3 for the formal definition). It is theoretically known that every manifold embedded in IR has tubular neighbourhood. In fact if we take sufficiently small neighbourhood of a manifold, we may find a tubular neighbourhood. On the other hand, it is easy to imagine that we cannot. take a too large neighbourhood as a tubular neighbourhood. See also Appendix A for previous studies which inspired our perspective.
3 Injectivity radius of a data manifold
In this section, we present how to estimate the supremum of possible radii of tubular neighbourhoods - the injectivily radius- a given data manifold. Based on the theoretical argument below, we establish the algorithm for the estimation (see Algorithm I in Appendix Fp. Throughout this section, let M denote an n-dimensional manifold (data manifold) in the Euclidcan space Rd For the terminologics of Manifold Theory, see Appendices
C.2 and
We refer to Litherland et al. 1999 for some notions and the case where (n, = (1,3), iee. the manifold M is a knot. The first crucial claim of this section is that many theoretical facts proven in their paper work for general dimensions as well. The second claim is that the quantities appearing in their paper can be estimated from a given data cloud and its data manifold. For simplicity, we will explain the former briefly and focus on the latter.
4

---

Published in Transactions on Machine Learning Research (07 (025)
3.1 Endpoint maps and Tubular neighbourhoods
We explain how to realise tubular neighbourhood of a manifold embedded in the Euclidean space. Definition 3.1. The e-meighbouuthood of M in Rd is the set
M(c) = U (VER*
2 <...
ΦEM
Definition 3.2. The normal bundle to M in Rdd is the set
NM (xx v) E& R M, MTM)
where T&M denotes the tangent space M æ.
Notice that the set NM forms a d-dimensional manifold. (The dimensions in the direction to M and its normal are n and d n, respectively.)
Definition 3.3. Let Eg: Rd Rd ,R4, (xr, v) + x be the addition map. We call restriction
E = EoNNM: NM - Rd, (2+
the endpoint map (or the exponential map).
Proposition 3.4 (ef. Theorem C.1I Let >> O and consider the subset
NM, = ((( E NM ||e|| < NMM.
Then the image E(NM.) coincides with the e-neighbourhood M(t) of M in Rd. Farthermore, this image forms a tubulor neighbourhood of M if and only if the map E NM. is an embedding.
3.2 Injectivity radius and its estimation
Definition 3.5. The injectivity radius R(M) of M is the supremum of numbers 0 such that the e-ncighbourhood of M in R is also a tubular neighbourhood. If such does not exist, define R(M) 0. We also define the following two quantities.
(1) The first injectivity radius Rí (M) of M is the infimum of the set
|| (0, v) NM critical point of the map E. for some point E M }
(2) The second injectivity radius R2(M) of M the infimum of the set
æ1. 32 E M, œ1 + T2, æi - 02 1 Tny M and a 22 Tag
{2 x1
x2ll
Roughly saying, (M(M) is the radius that the endpoint map fails to be regular at some point; R2(M) is the radius at which two separated tubes touch each other (see Figure 3). Thanks to the following assertion, these quantitics preciscly measure the injoctivity radius, i.c., where the first singularity for the c-noighbourhoods occurs.
Theorem 3.6 (62 of Litherland et al. 1999 ). It holds that R(M) =
In this paper, the estimation of (2) is performed following the definition. See Appendix D.3 for some ideas which may make the estimation easier. Therefore we here argue how to estimate R2 (M). This quantity is closely related to the curvature of M (ef. [efferman (2020)) ). Also, is simple we consider the case that the manifold M is curve in E² (see Appendix D.2.. In general case, seems to be difficult. However show the following (sce also Theorem C.7
5

---

Published in Transactions on Machine Learning Research (07/202) 2025)
M
Rg(m)
Ra(s)
(a) Ri(M)
(b) R2(M)
Figure 3: Sketches of the first and second injectivity radii
Theorem 3.7. Assume that the manifold M C. Bd is expressed by M= where F: Rd - Ted- is a differentiable map of which O E Rd-n is a regular value. In addition, assume that we have vector fields t1,4, 11.22...,.y L.. (n S N) defined near M such that for every æ E M the vectors ti ta(x), t2))), (x(x))...x.,) are longent to M and span the langent space T.MM Then the first injectivityy radius R1 (M) coincides with the infimum of the Euclidean norms ||l|| of vectors v E IRd such that v 1 T&M and the d (d+ N n))-mtrii
=
{o R' F(z)= 0},
OF dx (x)
(041 (x, v) Zo
0:1 Oo
[8pN (x, v) Zx
dPN do
LM(Ø, v)
(x, v)
(x, v)
(4)
is degenerate for some point æ E M, where
8p: Rd x Rd
RR,
RR (0( (t.(2:), m)
for 1,2, N.
This assertion is proven by application of the Method of Lagrange Multiplier. See Appendix D.1 for its precise proof. We here note some remarks.
Remark 3.8. The condition that the matrix L(x, v) degenerates at (x, v) NM is equivalent to that the determinant of the d x d-minor
OF
T (x) Ox
(x, v) (D00) 0x
Oein (x) (c,
(0gin (x, (x, v) Ix
Opin (x, ve 7
(5)
do
do
of L(x, v) vanishes for every n-tuple (iI,..., in) satisfying that 1 is < ... OF On (z) is of full-rank for every point æ E F-
... < in S N. Indeed, the matrix
Remark 3.9. It crucial to find vector fields t, satisfying the above condition. For example, (small extensions of) the gradient vector fields t; grad x; (i = 1....,d) satisfies the condition, where _;; M - R denotes the projection to the i-th axis in Rdd In general, have to take the number N greater than
3.3 Example (unit circle 5!)
Let verify Theorem 3.7 through the most typical manifold the unit circle $1. Define a function F: R + R by
F(x,y) = x² + + 1
Then we have 8 = (-) One of the normal vector field on S is given as grad(F) = =(9 (FF 8F = (2x, 2y), By
(-y, 2) is tangent vector field which spans the tangent space to S at each point (z, v) S1. Applying
Theorem 3.7 the first injectivity radius Rí (St) is calculated as follows. For point (z, y) E S1, the matrix
12x
+
L ((x, y). (w1, o2)) =
2y
- U1
x

---

Published in Transactions on Machine Learning Research (07/2025)
is degenerate (i.e., its determinant is zero) if and only if (01, v2) (-z, -t). Thus, obtain
Ri(S') v(-2))+ =1.
By definition, R2(S is also equal to the injectivity radius R(S!) is equal to 1.
This result is utilised in Section 5.2 for the experimental validation.
3.4 A pilot numerical experiment to validate the proposed algorithm
We perform pilot experiment to verify the algorithm. The detailed setting and the results are present in the Appendix F.1 The estimated R for S is 0.999 = 0.006.
4 Tubular neighbourhoods and diffusion dynamics
In this section, we investigate the relation between tubular neighbourhoods and diffusion dynamics.
4.1 The proportion of particles within the tubular neighbourhood
Let >> O. Let M(e) be the e-neighbourhood of compact oriented manifold M in the Fuclidean space Rd as defined in Definition 3.1 Suppose (x)) is smooth solution to the Fokker-Planck equation (3) with an initial condition po(x) = 6m(x) here (x) is Dirac's density function with its support M. We define a
function (t) as follows:
(t) = [ (x(d))
Remark 4.1. The readers may understand this function represents the proportion of particles within the tubular neighbourhood (see also Section 5.2 for the specific cases in numerical experiments).
Proposition 4.2. Assume ((1) R20 R is smooth function and fa(x) ((()(())) (()))))))))))))>>))))))))))))))) /B(1) in
3 (f) is some smooth vector field). We have:
@ lim-++ df (t) 0 and lim 0t 82
(t) = 0. Thus there crists at least one te in (0, +20) such that
I (te) 0. Moreover if B(t) 0 and
0t2
(V=lnp(()) <
(7)
for any E aM(() and any E R,o then (t) is strictly monotonically decreasing. Here is unit outward pointing normal veclor field along OM(e).
Remark 4.3. In other words, the first term of the SDE 2 at the boundary of the tubular neighbourhood is closely related to the behaviour of I (t). We can write I ('are) (t) in terms of free energies on the boundary of the tubular neighbourhood. Refer to Appendix H for a comprehensive analysis and additional details.
4.2 The score vector fields and tubular neighbourhoods
The marginal distribution (x) arising from Variancc-Preservvii stochastic differential equation (VP-SDE; also known as DDPM) can be written as
/ M
(x)
N(y10,2, (1 1 ())))add)d.
where 0, - d( and Po(y) is the distribution at time t 0. As a score vector field characterises the diffusion dynamics, one should analyse its behaviour. In this soction, we investigate how three quantities (dimension, injectivity radius, and time step) affect the behaviour of the score vector fields V, (z). Let first consider the case of spheres.

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Proposition 4.4. Suppose = 5th is n-sphere of radius R in RR. Let e be as R O. Let be unit outward pointing normal vector to the boundary of c-meighhourhood AM(e). Assume
6 + (1 02)(R - Vd.
(9)
V1 0f
dM(() and pu(y) is constant C greater than 0 M. Then the inner product of the score vector and n (a unit outward pointing normal vector to the boundary) at the boundary of c-neighbourhood is less than zero:
Vnln((() 5.
Example 4.5. Let M 5t in R² with radius and |x| = 0.99 (i.e., = 0.99). Compute (9) and we understand that V, (a)) points toward S if 0, > 0.712. Similar things can be observed for S in R3 Therefore this explains the Figure 21 and Figure 22
Remark 4.6. the previous study (Theorem D.1 by (Stanczuk et al. 2024 ), it is claimed that when the step time 9 is large enough, the score vector points the normal direction. In this work, we claim that when an inequality (9) involving the number of time steps, the dimension of the ambient space, and the injectivity radius, is satisfied, the score vector points toward the manifold side. This is kind of generalisation of Theorem D.1 in (Stanczuk et al. 2024). This indicates that geometry of data governs the behaviour of the score vector.
We generalise the statement as follows:
Proposition 4.7. Suppose M is compact oriented manáfold embedded in RRd. Let eo be an injectivity radius. Let ED > O. Let n be a unit outward pointing normal vector to the boundary of e-neighbourhood θM(e). Assume
+ [ -0,) 2 Vd.
(10)
0z
E 0M(() and Py( is constant C greater than 0 on M. Finally assume line segment with and the origin as its vertices do not intersect M. Then the inmer product of the score vector and n at the boundary of e-neighhourhood is less Ihan zero:
V Inp((()) n n S. O.
Remark 4.8. This is simplified statement, see Appendix 1.2 for details. The smaller the injectivity radius slower time of the turning of the score vector field becomes. This explains phenomena. in Section 5.3
5 Experiments
In this section, empirically demonstrate the presence of phase transitions at the boundary of the tubular neighbourhood during the generative process of diffusion models. Specifically, use two main indicators: (1) the proportion of particles outside the tubular neighbourhood at cach time step, and (2) the Wasserstein distance between the training data distribution and the generated distribution. We also adopt the late initialization scheme (Raya & Ambrogioni 2023). in which generation starts from different initial time steps, allowing us to capture abrupt changes more clearly. After describing our general methodology (Section 5.1). we present experimental results on various synthetic manifolds (hyperspheres, ellipses, tori, and disjoint ares), and finally apply our approach to real datasets by embedding them into hyperspheres.
Remark 5.1. Throughout this section, given manifold M embedded in Rd the term tubular neighbourhood of M refers to the set of points within the injectivity radius R(M) of M.
5.1 General Methodology
Diffusion Model Setup. We employ the DDPM framework (oo al. 2020) with T 1000 diffusion steps in the forward and reverse processes. Let. (Xp, (xn,X1.... Xi...., X7] denote trajectory under Gaussian noise

---

Published in Transactions on Machine Learning Research (07 (07/2025)
perturbation; follow the standard parametrisation where xy N(O, I) and each intermediate x is obtained by adding Gaussian noise according to fixed variance schedule. All experiments share the same neural network architecture for score estimation and use the Adam optimizer. The batch size, learning rates, and number of training iterations are kept consistent across experiments unless stated otherwise. Additional hyperparameter details are provided in Appendix J
Late Initialisation.
a standard DDPM sampling procedure, we typically begin with Xy No I) T 1000 and run the reverse diffusion steps down to ( 0. By contrast, late inilializalion modifies this procodure such that we start the reverse diffusion from chosen time Tii (ranging from 1 to 1000), while still sampling the initial state as XTma ~ (N I). In other words, rather than varying the "noise level" per se, we effectively shorten the reverse diffusion process and thus shift which segment of the generative process is observed. By examining how the model behaves for different values of Tinit, we can reveal abrupt changes in the generated samples (i.e. phase transitions) that occur at particular stages of the reverse diffusion.
Evaluation Metrics, We track two main quantities in our experiments:
Proportion of Particles Outside Tubular Neighbourhood. At each diffusion time step t, we record the fraction of sample points x, whose distance to .M exccods the injoctivity radius R(M).
Wasserstcin Distance. We also compute the 1-Wasserstein distance between the generated distribution and the training distribution. We use 1- 1-Wasserslein distance as a qualitative, parameter- free indicator of how the generated distribution changes under late initialisation. Due to space constraints, we only provide a brief overview here; the formal definition (including cost matrices and the optimal transport formulation) can be found in Appendix J.2
5.2 Relationship between tubular neighbourhood and phase transition in unit hypersphere
In this subsection, we investigate how phase transitions occur when the data manifold is a unit hypersphere. Since the injectivity radius of a unit hypersphere is 1, the boundary of the tubular neighbourhood is simply all points at distance from the manifold surface.
Setup. We consider three cases: S 52, and 520, each embedded in a suitable ambient space Rd. For each manifold, we measure (1) the proportion of particles outside the tubular neighbourhood (red line) at each time step, and (2) the Wasserstein distance (blue line) under late initialization. We use Timit from to 1000, following the DDPM setup described in Section 5.1
Results. As shown in Figure ! a sharp increase in the Wasserstein distance consistently appears after the majority of particles enter the tubular neighbourhood (the purple dashed line indicates the time when 99% of particles have crossed the boundary). This observation supports our hypothesis that crossing the injecuivity-radius boundary triggers or coincides with the onset of phase transitions. Table [ presents the Wasserstein distances at selected proportions of particles outside the tubular neighbourhood; we see that Pproportion decreases (i.c., more particles move inside), the Wasserstein distance tends to increase sharply.
Table 1: Wasserstein distances W for different late initialisation times. Pproportion represents the proportion of particles outside the tubular neighbourhood.
Pproportion
0.1
0.5
0.9
0.95
0.99
0.999
1.0
Dataset
S' embedded in R 0.283
0.073
0.020
0.019
0.018
0.019
0.018
2 embedded in R16
1.344
0.343
0.058
0.038
0.030
0.030
0.031
$20 embedded in 224
3.895
1.858
0.970
0.882
0.807
0.781
0.759
9

---

Published in Transactions on Machine Learning Research (07 (025)
0.98 Thewerrld
53hh red
2o Thoesaatd
2.99 Thaesholc
Waveenthar Diberre
307
10P
30"
-0"
1000
800
500
400
200
1000
800
650
400
200
1020
500
600
400
200
(a) 8 embedded IR:4
(b) g2 embedded in IR:1
(c) $20 embedded in R
Figure 4: The blue line (left axis) depicts the Wasserstein distance between the training data distribution and the generated distribution, measured as function the shifted initial diffusion time T. The red line (right axis) indicates the proportion of particles outside the tubular neighbourhood standard diffusion model. The purple dashed line marks the diffusion time when 99% of particles exit the tubular neighbourhood. Notably, sharp increase in the Wasserstein distance is observed particles enter the tubular neighbourhood.
each diffusion time in
Discussion. These results indicate that, for manifolds with constant curvature (such as unit hypersphere),
the tubular boundary serves as a clear geometric marker for the transition. In Section 5.3 we explore whether
a similar relationship holds for manifolds with varying curvature, such as ellipses and tori. For more detailed descriptions of the hypersphere experimental setup, please see Appendices J.3 and J.4
5.3 Phase Transitions in High-Dimensional and Mixed-Curvature Manifolds
In the previous section, we showed that when the data manifold is a unit hypersphere (with constant curvature), the tubular boundary closely aligns with the onset phase transitions. However, natural data often have more complex geometry, and high-dimensional embeddings can introduce additional challenges. In this subsection, we investigate two scenarios that deviate from the constant-curvature setting:
5.3.1 High-Dimensions Hypersphere. We first consider 820 embedded in R48 thus increasing the ambient dimension compared to Section 5.2 As shown in Figure 5al the hypothesis breaks down: the Wasserstein distance begins to rise well before most particles enter the tubular neighbourhood. This discrepancy can be attributed to the fact that a higher-dimensional Gaussian is typically concentrated farther from the origin, while the data manifold (a unit hypersphere) remains a fixed radius of 1. Consequently, under late initialization, the sampled points deviate substantially from where the model expects them to be at that time step, making accurate reconstruction difficult. Those observations indicate that high-dimensional embeddings can create significant mismatch between the data manifold and the initial Gaussian distribution.
5.3.2 Ellipses and Tori (Mixed Curvatures). Next, we consider ellipses and tori, which exhibit non- constant curvature. Specifically, we examine an ellipse with major axis 2R, minor axis 2r, and injectivity radius r2/R, as well as a torus with major radius R, minor radius r, and injectivity radius min R r} (see Appendix J.5). Figure illustrates that the Wasserstein distance undergoes multiple rises under late initialization. By systematically varying Tinit from 1000 down to lower values, we observe several partial transitions, culminating in final pronounced increase when particles cross into the region of highest curvature. This suggests that multiple phase transitions occur for manifolds with varying curvature, and the last transition consistently aligus with the boundary crossing of the region of highest curvature.
Discussion. Table 2 summarizes how the Wasserstein distance changes as the proportion of particles outside the tubular neighbourhood decreases. For the high-dimensional sphere (520 in (R)), the mismateh is pronounced, indicating that late initialization may not reliably capture the true transition point. For ellipses and tori, multiple riscs the Wasscrstein distance suggest multiple partial transitions, culminating in a final transition where particles cross into the region of greatest curvature. These findings imply that the injectivity radius is tightly linked to the largest curvature the manifold, and thus serves as key geometric quantity
10

---

Published in Transactions on Machine Learning Research (07 (07/2025)
for understanding phase transitions more general shapes. In the next subsection (Section 5.4). we delve further into the link between curvature and transitions using disjoint ares with different radii.
Table 2: Wasserstein distances W for different late initialisation times. Pproportion represents the proportion of particles outside the tubular neighbourhood.
Peroportion
0.1
0.5
0.9
0.95
0.99
0.999
1.0
Dataset
500 embedded in R48
21.754
13.380
7.964
6.978
5.366
3.900
0.752
Ellipse (R 3, 1) embedded in R
7.762
5.893
4.016
3.601
2.936
2.184
0.479
Torus (R 3, 1) embedded in IR 16
2.597
1.888
1.433
1.335
1.167
0.872
0.272
3.1
Uiozersen Diztnce
Wegse Disance
301
10
1000
000
Difusioon Time
200
Difuutoo Time
Difuston Time
(b) Ellipse with R = 3, embeed- ded in 16
(c) Torus with I = 3, embedded 16
(a) 500
embedded in R
Figure 5: (Blue, left axis) and (Red, right axis) show the Wasserstein distance and the proportion of particles outside the tubular neighbourhood, respectively, as described in Figure 4. The purple dashed line marks the 99% threshold. Unlike Figure 4, the Wasserstein distance increases before the particles enter the tubular neighbourhood.
5.4 Disjoint Arcs with Multiple Curvatures
In the previous sections (e.g., Section 5.3 we observed that when a manifold has regions of distinct curvature, each region exhibit its own timing for phase transitions. To verify this hypothesis more directly, we construct a simplified manifold consisting of two disjoint arcs of different curvatures in R6
Setup. As illustrated in Figure G one are has radius 1 (higher curvature) and the other has radius 2 (lower curvature). The injectivity radius each individual are is its own radius; however, when these aros are combined into single manifold, the overall injectivity radius is determined by the smaller radius (i.e., the higher curvature). We apply the samne late-initialization scheme described in Section 5.1 tracking both the proportion of particles outside each arc's tubular neighbourhood and the Wasserstein distance over time.
Results. Figure shows that the Wasserstein distance undergoes two distinct rises: the first corresponds to the are of radius 2, and the second occurs when particles begin entering the tubular neighbourhood of the are with radius 1. Table quantifies these two "phase transitions." demonstrating that each arc's curvature indeed induces separate transition timing. Although the manifold's global injectivity radius ultimately determined by the smaller are (radius 1), this experiment confirms that multiple curvatures can lead to multiple transition events within the same manifold.
Discussion. These results support our hypothesis that each distinct curvature region in manifold can trigger ils own phase transition. While many real-world datasets may only exhibit one dominant curvature scale (leading to a single noticeable transition). this disjoint-arcs example demonstrates how multiple abrupt transitions can emerge when different parts of the data manifold have substantially different curvature. the next section (Section 5.5) we explore how these insights can be extended real-world data embedded into hypersphere.
11

---

Published in Transactions on Machine Learning Research (07/2025)
Table 3: Wasserstein distances for different late initialisation times. Pproportion represents the proportion of particles outside the tubular neighbourhood.
P Pproportion
0.1
0.5
0.9
0.95
0.99
0.999
1.0
Datasel
Disjoint ares
0.283
0.073
0.020
0.019
0.018
0.019
0.018
101
Threshold
1.0
0.30
Threardd
0.0
20
1.0
1000
5400
400
1000
800
600
Diffuiion Timle
Dritision Time
Figure 7: A second rise in the
Figure 8: Fashion MNIST 520 bedded in R224.
Figure 6: Projection the 1st and 2nd dimensions of data em- bedded in 16
Wasserstein distance occurs, indicat- ing phase transitions.
5.5 Exploring Tubular Neighbourhoods in Real Datasets
As shown in Section 5.4 the injectivity radius of manifold is determined by the region of highest curvature. However, directly applying this concept to natural datasets like MNIST or Fashion MNIST can be problematic, because their local geometry can vary siguificantly across different samples. Such diverse curvature often leads to an extremely small injectivity radius when computed in the raw data space, making it difficult to capture meaningful phase transitions.
Hyperspherical Embedding. To address this limitation, we embed each dataset into a unil hypersphere using Hyperspherical VAE (Davidson et al. 2018). This procedure maps all data points onto 520 in 224 thereby constraining their global geometry to fixed radius. While this embedding does not yield uniform distribution the hypersphere, it simplifies the analysis of phase transitions by ensuring that large-scale curvature effects are more tractable. The detailed setup of the Hyperspherical VAE is provided in Appendix J.7
Experimental Results. After embedding, we apply the same late-initialisation scheme (Section 5.1 measuring the proportion of particles outside the tubular neighbourhood at each diffusion time slep and the corresponding Wasserstein distance to the embedded training distribution. Figure g shows that for Fashion MNIST, the Wasserstein distance starts to rise after particles begin to enter the tubular neighbourhood in the latent space. Table g presents the quantitative results for both MNIST and Fashion MNIST, indicating an increase in the Wasserstein distance once T (the reverse diffusion start time) falls below a certain threshold.
Table 4: Measured Wasserstein distances W for different late initialisation times. The variable Pproportion represents the proportion of particles outside the tubular neighbourhood, and the corresponding W at the respective time points are shown in the table.
Pproportion
0.1
0.5
0.9
0.95
0.99
0.999
1.0
Dataset
MNIST 520 embedded 24
2.254
1.060
0.668
0.632
0.592
0.576
0.559
Fashion MNIST 200 embedded in 24
2.697
1.295
0.673
0.590
0.509
0.472
0.453
12

---

Published in Transactions on Machine Learning Research (07 (025)
Figure 9: Fashion MNIST images decoded using SVAE after diffusion times of 1000, 500, 200, 100, and (from left to right) in the latent space.
Discussion. These findings highlight that while real-world datasets exhibit diverse local curvature, embed- ding them in a hyperspherical manifold enables us to observe and interpret phase transitions more cleanly. The mapping aligns with our earlier analysis of synthetic manifolds, where entering the tubular neighbourhood correlates with notable rise in the Wasserstein distance. Further exploration of this approach -eet--iincinnggg faster sampling, multi-modal embeddings, and applications to other data modalities-b is left to future work.
6 Discussion and conclusion
In this study, employed the concept of the injectivity radius to understand the generative process of diffusion models and analysed it theoretically and experimentally from the perspective of the geometric structure of data manifolds. Specifically, we provided a new perspective on phenomenon where certain features emorge rapidly over a short time interval, referred to as a phose transition.
However, our theoretical results rely on the assumption that the data manifold M is a compact smooth submanifold of ıd While this assumption is satisfied by the synthetic datasets in Section 5 real-world image distributions often contain edges, occlusions, and other singular structures that violate smoothness. Extending the injectivity-radius analysis to stratified spaces, developing numerical schemnes that remain stable in the presence of such singularities, therefore remains an open research direction..
In addition, the concept of the injectivity radius, which may correspond to the region the manifold with the largest curvature, only partially explains the phase transition phenomenon. To address this limitation, defining mathematical quantity that corresponds the smallest curvature of the manifold may remove the assumption of constant curvature, potentially leading to more general theory applicable datasets. This would allow for more efficient sampling methods that better reflect the geometric properties of the data manifold, thereby enhancing the overall performance of diffusion models. From the perspective of nonequilibrium thermodynamics, the system's 's free energy, defined by weighting the energy at cach point with the probability distribution function, can provide a more comprehensive macroscopic understanding of phase transitions (for details, see Appendix G This approach could enable a more accurate discussion of phase transitions as macroscopic phenomenon, complementing the microscopic geometric analysis.
wider range of
By exploring the relationship between the data's geometric structure and the score vectors, as discussed in Section 4.2 it may also be possible to design optimal noise scheduling strategies that are tailored to the geometry of the data manifold. Clarifying how score vectors interact with the manifold's geometric properties could further optimise the generative process. In addition, our experiments with real-world datasets suggest that modifying the architecture or objective used for VAE embedding into hyperspheres could extend the applicability of the tubular neighbourhood concept to more complex datasets. This modification is expected to improve sampling efficiency and increase the flexibility of diffusion models in handling a wider variety of real-world data.
These discussions lie at the intersection of concepts from differential geometry, particularly singularities, statistical physics, especially phase transitions in non-equiibrrum thermodynamics, and computer science, specifically diffusion models. This interdisciplinary approach represents an important step forward, and further thcoretical development and practical applications in this direction hold promising potential.
13

---

Published in Transactions on Machine Learning Research (07 (02))
Author Contributions
Manato Yaguchi led the numerical experiments and, together with Kotaro Sakamoto, co-designed the experimental methodology (Section 5); he also coordinated reviewer communication and consolidated the final manuscript. Kotaro Sakamoto co-led the conceptualization with Ryosuke Sakamoto, co-designed the experimental methodology with Manato Yaguchi, and implemented the AIER algorithm (Appendix F he contributed to manuscript revisions. Ryosuke Sakamoto originated the research idea, co-led the conceptualization, and mainly drafted Section 4 Masato Tanabe and Masatomo Akagawa proved the main theorem and jointly authored Section 3 Yusuke Hayashi contributed non-equilirrium- tereuullbrruun-tteroodynmamn thermodynamies perspective and drafted Appendix G Masahiro Suzuki and Yutaka Matsuo supervised the project and secured the research resources. All authors discussed the results, revised the manuscript, and approved the final version.
Acknowledgments
The authors express special thanks to the anonymous reviewers whose comments led to valuable improvements of this paper. Part of this work is supported by projects commissioned by JSPS KAKENHI Grant Number 23H04974 as well JST SPRING, Grant Number JPMJSP2119.
14

---

Published in Transactions on Machine Learning Research (07/ 2225))
References
J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, S. W. Bodenstein, D. A. Evans, C. C. Hung. M. O'Neill, D. Reiman, K. Tunyasuvunakool,
Z. Wu, A. è, E. Arvaniti, C. Beattie, O. Bertolli, A. Bridgland, A. Cherepanov, M. Congreve, A. Nowen-Rivers, A. Cowie, M. Figurnov, F. B. Fuchs, H. Gladman, R. Jain, Y. A. Khan, C. M. R. Low, K. Perlin, A. Potapenko, P. Savy, S. Singh, A. Stecula, A. Thilaasundaaam, C. Tong, S. Yaknoen, D. Zhong, M. Zielinski, A. dek, V. Bapst, P. Kohli, M. Jaderberg, D. Hassabis, and J. M. Jumper. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, May 2024. ISSN 1476-4687. doi: 111108//11556--02--07477------51 URL /ttp://////////////////////////0
Brian D.O. Anderson. Reversc-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-83) 1982. ISSN 0304-4149. doi: http:://bol.org//00000///// 00000000000000001100009990000000 URL https:
// //www htt.://wwcaaattte.ccooc/acaaaaaa
Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Linear convergence bounds for diffusion models via stochastic localization. CoRR, als/2308.03305 2023a. doi: 10.48550/ ARXTV .3008.
03686. URL httpps///ol..rrg// 1111111..4588888 (8550/arXiv... 0368.
Joe Benton, George Deligiannidis, and Arnaud Doucet. Error bounds for flow matching methods. CoRR, abs 01280.166600, 2023b. doi: 10.48550/ARXIV 2305.16860. URL http:://lbo..org/1...5555/
2305. 16860
Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, and Marc Mézard. Dynamical regimes of diffusion models. CoRR, abs aabs/2402.1844 2024. doi: 10.48550/ ARXIV 1849. URL https://6oi.orr//0..o
arXiv. 1849.
Adam Block, Youssef Mroueh, and Alexander Rakhlin. Generative modeling with denoising auto-encoders and langevin sampling. CoRR, abs/2002.00107, 2020.
Sam Baylor, Adam Leach, Yang Long, and Chris G. Willcocks. Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models. IEEE Trans. Pattern Anal. Mach. Intell., 44(11):7327-7341 2022. doi: 10.1109 R..119/TPAAI..O attps://dols www.ooo//////.. 111.119/TPAMIII1 2021.311666
URL
Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. Trans. Mach. Learn. Res., 2022, 2022. URL http:://ppaareeeeeeeeeeerrrrrrrp
Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications to score-based generative modeling. In Marc' Aurclio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jemnifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annuol Conference on Neural Information Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual, pp. 17695-17709, 2021. URL http:: httptpp://proccednngg...o http:///w...rrrpppscccppp//////p 22062155222740111227777778888882 Abstract...............c........ html
Defang Chen, Zhenyu Zhou, Jian-Ping Mei, Chunhua Shen, Chun Chen, and Can Wang. geometric perspective on diffusion models. CoRR, abs/2305. 19947, 2023a. doi: 10.48550/ARXIIV. 1994. URL https://doi. 48150..885a/aa... 2305 19947
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User- friendly bounds under minimal smoothness assumptions. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML, 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Reseorch, pp. 4733-4763. PMLR, 2023b. URL UTLLhttps://prrceednng mlr wwwppppwsseeeee/pppp press//202/chee22g html
Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. In Andreas Krause, Emma Brunskill,
15

---

Published in Transactions on Machine Learning Research (07/2025)
Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, TCML, 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 4672 4712. PMLR, 2023c. URL https://proceedding mlr, p v202/chen23o.. chen23o... html
Minshuo Chen, Song Mei, Jianqing Fan, and Mengdi Wang. An overview of diffusion models: Applications, guided generation, statistical rates and optimization. CoRR, abs/2404.077711, 2024. doi: 10.48550/ARD ARXIV.
2404.07771. URL lattps://aat org/10. 48550/ arxiv. 2474.07771
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. In 9th International Conference Learning Represcntations, ICLR 2021, Virlual Event, Austrin, May 3-7, 2021. OpenReview.nel, 2021. URL https: attps://openrevien
net miiimmmmdddiMMii NdNNNMM
Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probabil- ity flow ODE is provably fast. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: An- nual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023d. URL. htttp://ppperrrs. www..ic...aaaaaaee paepaperffles///eeeeeeeeessssses (ttlle//paper/223//aaaa html
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023e. URL attpps://operrevier.r net/ 7tdp///Wzzddz2y
Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 11462 11471. IEEE, 2022. doi: 2022.01118. URL https:://dot http:////oooorrr/11// 10. CVV.0//6/222288.... 2022.01118
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for in- verse problems using manifold constraints. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Bel- grave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NewrIPS 2022 New Orleans, LA, USA, November 28- December 9, 2022, 2022. URL http::t:://paaee www..nii.c... httppc///ppprrrrllllspps//p///p/ /tless/paper//2////// hash// 64666777c71600111115422226000000 -owaaa--tttttattttoo-reeeee
html
Giannis Daras, Hyungjin Chung, Chieh- Lai, Yuki Mitsufuji, Peyman Milanfar, Alexandros G. Dimakis, Chul Ye, and Mauricio Delbracio. A survey on diffusion models for inverse problems. 2024. URL
htttp:://gianiiddrree www.aa..itttbbbbb... /ettho/..//pub..ccctttoo//f/fefs wwwronssuuvvrrue pdf
Tim R. Davidson, Luca Falorsi, Nicola De Cao, Thomas Kipf, and Jakub M. Tomezak. Hyperspherical variational auto-cncoders. In Amir Globerson and Ricardo Silva (eds.), Proceedings of the Chirty-Fourth Conference on Uncertainly in Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6- 10, 2018, pp. 856-886 865.866.444 AUAI Press, 2018. URL http:://aaii org/ htt.aa///////oeeegg///eeeeeeeeee 330 pdf
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusson models beat gans on image synthesis. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual, pp. 8780-8794, 2021. URL https: Abstracttbtttt....... html
//proceediggs.. neur iourrps.. ips cc/ wwcc//aaer////e (aaperr/0221.0 htt.ha/h///aaaaaeeaaaaa///000000
Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, and Jie Tang. Cogview: Mastering texl-lo-image generation via transformers. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 94: Annual Conference on Neural Information Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual, pp. 19822-19888 2021. URL (tLhstts html
16

---

Published in Transactions on Machine Learning Research (07/2025)
Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, and Kaidi Xu. Are diffusion models vulnerable to membership inference attacks? Tn Andreas Krause, Em Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, 1CML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Leorning Research, pp. 8717 8730. PMLR, 2023. URL https::/prooceedings mlr. pta....rees//2222 hwtp//ddnnnnbbbmaccem
Jan Dubinski, Antoni Kowalezuk, Stanislaw Pawlak, Przemyslaw Rokila, Tomasz Trzcinski, and Pawel Morawiecki, Towards more realistic membership inference attacks on large diffusion models. In LEEE/CVE Winter Conference on Applications of Computer Vision, WACV 2024, Waikoloa, HI, USA, January 9-8, 2024, pp. 4848 4857. IEEE, 2024. doi:
URL hittpa://dos org/10
1109/HACV57701. 2024. 00479
M. Esposito and C. Van den Broeck. Second law and landaner principle far from equilibrium. Europhysics Letters, (5(4):40004 aug 2011. doi: 0295-5075/95/4000
0200000055995500000000000 URL https://dx..... doi. http:///tt..i..o/.......... 1209/
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis, 2013.
Ivanov S. Kurylev Y. Lassas M. Narayanan H. Fefferman, C. Reconstruction and interpolation of manifolds. i: The geometric whitney problem. Found. Comput. Math., 20:1035 1133, 2020. doi: 10.0007/110208-099-093333
Giulio Franzese, Simone Rossi, Lixuan Yang, Alessandro Finamore, Dario Rossi, Maurizio Filippone, and Pietro Michiardi. How much enough? A study on diffusion times in score-based generative models. doi:
Entropy, 24(4):633, 2023.
URL http:://///ii../11....33333
Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. A probabilistie fluctuation basod membership inference attack for diffusion models. CoRR, /t////338..1111 2023. doi: 10.48550/ TRXXVV2330 12143. URL htt::://oo..ogg/// hrt.:/...orrg/....///////r../ 2308 12143
48550/arXiv
Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not 2023. doi: 10.48550/ARX0
the destination: How data guides diffusion models. CoRR, abs/
2312.06205. URL /tttt:tt////tt org//..48550///////...
48550/ arXiv. 2312.06200
Sandesh Ghimire, Jinyang Liu, Armand Comas Massague, Davin Hill, Aria Masoomi, Octavia I. Camps, and Jennifer G. Dy. Geometry of score based generative models. CoRR, abs/ 22000022.44411 2023. doi: 10.48550/ARXIV, 2302.04411. URL https://doi. http///....rr/.00..066666//// 2302. 04411
/0/////1110 48550/8rXixy
Davide Ghio, Yatin Dandi, Florent Krzakala, and Lenka Zdeborová. Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective. CoRR, /t///2308.1088. 2023. doi: 10.48550/
ARXIV 22..2220...800 URL https:://oo/ /ttt::://doi.or/10000 www/o0..//00 48550/85Xiii 2308 14085
Yuji Hirono, Akinori Tanaka, and Kenji Fukushima. Understanding diffusion models by feynman's path integral. In Forty-first International Conference on Machine Learning, 1CML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.nett 2024. URL https::/ http://wwww...ttttoooo/raaaaaaaa
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusson probabilistic models. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020. URL httts::/ wttp://sgs..euur..ccccccoeooee Abstract .ttll
Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. J. Moch. Learn. Res., 23:47:1-47:: 2022a. URL htttss://jmtr..... org/ www.oo/pgp.///////////
221.///2/2222.2.. html
Jonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave,
17

---

Sosuke Ito. Geometric thermodynamics for the fokker fokker-planck equation: stochastic thermodynamic links between information geometry and optimal transport. Information Geometry, 7(S1):441-483 March 2023. ISSN 2511-249X. doi: 10.1007/s41884--223--8 Published in Transactions on Machine Learning Research (07 (07/2025)
K. Cho, and A. Oh (eds.), Advances in Newral Information Processing Systems 35:
Annual Confer- ence. on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, Novem- ber 28- December 9, 2022, 2022b. URL http://papers nips. cctpaper fwwes/paper//eeeeeeeeep 20////22222hhaaaaaa/ Awwtract- Conferenceeeeceeeoeeoeeece html
Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res., 709, 2005. ISSN 1532-1435,1533-7995
Kotaro Ikoda, Tomoya Uda, Daisuke Okanohara, and Sosuke Ito. Speed-accuracy trade-off for the diffusion models: Wisdom from nonequilibrium thermodynamics and optimal transport. CoRR, abs /2407 /445000040495 2024. doi:
AR URL https:://oi..org:
URL 48550/arXiv. www..2407.04440
841884-023-00102-
Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In Kamalika Chaudhuri, Stefanie Jegelka, Te Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. PMLR, 2022. URL https:///rooceedings seapeeddiinnmmm......mmmmmem mlr pra.s/.rees/v122///// jatpa///2a12/amee222aa... html
Zahra Kadkhodaie, Florentin Guth, Fero Simoncelli, and Stéphane Mallat. Generalization in diffusion models arises from geometry-adaptive harmonic representations. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL
Fei Kong, Jinhao Duan, Ruipeng Ma, Henglao Shen, Xiaofeng Zhu, Xiaoshuang Shi, and Kaidi Xu, An efficient. membership inference attack for the diffusion model by proximal initialization. CoRR, abs 2200 18355, 2023. doi: 10.48550/ 001/50//AXXV.2332 18355. URL https://doi. org///.......... 48550/arX111 wwt.aawwwww........ee.. 18355
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 7, 2021. OpenReview.net, 2021. URL
XF88mm255
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-based generative modeling with polynomial complexity. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novem-
ber 28- December 9, 2022, 2022. URL http:://pppprss nips. http:pppprrrippppppppappppppppee https///pap.tttel/ppppppe//////p htt.://lles/apee//22////aaa//
http://papers. .ngerrs..nn www..cc/pppprr html
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of scoro-basod generative modeling for general data distributions. In Shipra Agrawal and Francesco Orabona (eds.), International Conference on Algorithmic Learning Theory, February 20-23, 2023, Singapore, volume 201 of Proceedings of Machine Learning Research, pp. 946 985. PMLR, 2023. URL httpps:/pproceeeiings. mlr, press//201/lee222. html
John M. Lee. Introduction to Smooth Manifolds, volume 218 of Graduate Terts Mothematics. Springer, New York, second edition, 2013. ISBN 076114419--999225(5000)))))) doi: 11.100779978-14449-990 URL
Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards faster non-asymptotic convergence for diffusion- based generative models. CoRR, abs//2006 2306.09251, 2023a. doi: 10.48550/ 10.48550/ARXTV. ARXIV 2306.09251. URL
18

---

Published in Transactions on Machine Learning Research (07/2025)
Gen Li, Zhihan Huang, and Yuting Wei. Towards mathematical theory for consistency training in diffusion models. CoRR, abs/2402.07802. 2024. doi: 10.48550/ 00555//ARRXV.2402200070 URL htttp:://doi.org// 48550/arXiv.
Marvin Li and Sitan Chen. Critical windows: non-asymptotic theory for feature emergence in diffusion models. In Forty-first International Conference Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.nel, 2024. CRL httts:://openrevier: http://wwww....oooooorraaaaaaaaa
Marvin Li, Aayush Karan, and Sitan Chen. Blink of an eye: a simple theory for feature localization in generative models. CoRR, abs/2502.0002/ 2025. doi: aoooi.org/10 www.o////....//////////////...// www..wwwww.......0000 2502.
/ttp//AAA.......2.200002 URL https:
Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. Diffusion- Im improves controllable text. generation. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Bel- grave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, nips http:/ppprr..tlsesppppep/e/////e 2aapperr//22//aahh. November 28- December 9, 2022, 2022. URL Conference hhtml
Abstract ContrractCConfreee
Yifan Li, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. Diffusion models for non-autoregreesive text generation: A survey. In Proceedings of the Thirty-Second International Joint Conference Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China, pp. 6692 6701. ijcai.org, 2023b. doi: 10.24963 10.24963/IJCAI. 222 750. URL https: //do:.:///oorr/r www.oo/gg///.......o.o 2.0..2996/.aca.... /000://///aa2222/75550 2023/750
Yaron Lipman, Ricky T. Q. Chen, Heli Ben- Ham-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL PqvMRDCJT9t]
R.A. Litherland, J. Simon, O. Durumeric, and E. Rawdon. Thickness of knots. Topology and its Applications, (1): 244, 1999. ISSN 0166-8641. doi: http:///ooooog///000/////0000011
URL https
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo P. Mandic, Wenwu Wang, and Mark D. Plumbley. Audioldm: Text-to-audia generation with latent diffusion models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 21450 21474. PMLR, 2023. URL httpps://proeeedinge mlr. mlr.press/ve2 Titees//222/1iuu233 html
Xingchao Liu, Lemeng Wu, Mao Ye, and Qiang Liu. Let build bridges: Understanding and extending diffusion generative models. CoRR, abs abs/2208.1 14699, 2022. doi: 10:46550/ARXIIV22208..18 URL
https://dbi..or//0...e 40t.0//a...055//..v..... 2208. 14699
Gabricl Loaiza-Ganem, Brendan Leigh Ross, Rasa Hosscinzadch, Anthony L. Caterini, and Jesso C. Cresswell. Deep generative models through the lens of the manifold hypothesis: A survey and new connections. CoRR, 0bs/2204402955, 2024. doi: 00.4855/ARXTVV..0 AA0.0//0AAR....000055..... URL http::////o....gg0...0.50////// 02954
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. CoRR, abs/2310. 2023. doi: 008555//AXXV223110
URL
lttts::/ddo.orgg// wttg/10...55///2iiv wtw.aa.www.............aa....1ea
2310 16834
Tomoya Matsumoto, Takayuki Miura, and Naoto Yanai. Membership inference attacks against diffusion models. In 2023 IEFE Security and Privacy Workshops (SPW), San Francisco, CA, USA, May 25, 2023, pp. 77 83. IEEE, 2023. doi: 1111009/SPW593332222...000 URL Inttps://dot http//0..109////599333
2023. 00013
19

---

Published in Transactions on Machine Learning Research (07/2025)
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.nrg .ee, 2022. URL htttps://pperreviee/ wwwp://wwwww.en.oonnnnnuama aireffooruniddBBcccFFF tE
Alexander Quinn Nichol and Prafulla Dhariwal. Improved demoising diffuusio probabilistic models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp- 8162 8171. PMLR, 2021. URL htttp:/prroceedingg mmlr wwwpp (ttes//w13//ncho222...ht
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: towards photorealistic image generation and editing with text- guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 16784 16804. html
PMLR, 2022. URL https://proceedinga httpp//eedaaaamm..pppsee///////2 wttppp/ewwweee..oooooeepo
Schuyler B. Nicholson, Luis Pedro Garcia-Pintoss Adolfo del Campo, and Jason R. Green. Time information uncertainty relations in thermodynamics. Not. Phys., 16, 2020. doi: 10.1038 00008///11567000009888888 URL
httpp::/www..aaturreecc//////eet httpa//ree..oeee..cee//////////0
Maya Okawa, Ekdeep Singh Lubana, Robert P.
Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffuusion models on a synthetic task. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advonces in Neural Information Processing Systems 36: Annual Conference on Neurol Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http::p:: mips. wwtp:nccppppppp.pp.peeepppppp wwccpaper_fiieee files/paper/ 2023/hash/9d0f /ttp:///000000111100000000000010 ottp:///.-..aaa-aaaaaaa---------
Abstract-
html
Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. PMLR, 2023. URL
.mlr. mwr.m.rreeeeeeeee ptwss/v202 oko23a. html
Yan Pang and Tianhao Wang. Black-box membership inference attacks against fine-tuned diffusion models. CoRR, abs/2312.0822 008207, 2023. doi: 10.48550/ 10.48550/ARXIV ARXIV 2312.08207. URL https://doi.orrg org/10../40000....... 485500
arXiv 2312 08207
Yan Pang, Tianhao Wang, Xuhui Kang, Mengdi Husi, and Yang Zhang. White-box membership inference attacks against diffusion models. CoRR, bbss/2308.0644 2023. doi: 10. 10.48350/ARXIV 2308.066405 URL
https://doi.. org/10. www.o////..../////////////////// (rtp50/arxiv..20 2300.00005000000
Yong- Hyun Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. Understanding the latent space of diffusion models through the lens of ricmannian geometry. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annuul Conference on Nearal Information Processing Systems 2023, NewrIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http::/ppaper.....t www.aaperssnipp....eee.p wttppccrpppppiiiieec files/ paper/2023/nasse hash//
-twp://etoeo..----ataaat-------e Abstract Con.ettoooooooeeneeeeeeeeee html
Jakiw Pidstrigach. Score-based generative models detect manifolds. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, L,A, USA, November 28- December 9, 2022, 2022. URL URL http::t/ppppsrr...ptetm www.:iiiss....peeee wttppppcppppppee..eeeee/a/aeeeee filees//aper//222// haap://eeeeaaaaaaet0000006666110
html
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text-conditional image generation with CLIP latents. CoRR, abs/ abs/2204.06125, 2022. doi: 00.48550/ARXI 22044.661...
URL httpps:///bi.oor//11 /t8p://00...0/11...85/55/////// 1.85500arXii 2204.06125
20

---

Published in Transactions on Machine Learning Research (07/2025)
Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL htt.:ttpppppppps..iii... cc/ paper
Robin Rombach, Andreas Blattmamm, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 10674-10685. IEEE, 2022. doi: 10.1109/CVPR52668 2022.01042. URL https://doi. org/10. 1109///VP9/6688 2022. .00042
Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding. science, 290(5500):2323-2255 2326, 2000.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kam- yar Seyed Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novem- ber 28 December 9, 2022, 2022. URL http::/papper: .ips. cc/ppperr files/paper/2022) h o Awweaett--tttttttt---aatea Cow.rreeceeco.ereece......om.... html
Antonio Sclocchi, A Alessandro Favero, and Matthieu Wyart. A phase transition in diffusion models reveals the hierarchical nature of data. CoRR, abs/ has//2402.1999. 2024. doi: 10.48550/ARXI/ 2402.16991. URL
0085://0000/00/////////// https://doi.oom org////0..or////.. 48540/8555/tit 240.. 16991
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.neg net,, 2023. URT ltttts:// http:/rwwwieeeeeeddeeeeeeeeeeeee net/pdf? 1d=nJfylDvgzlq
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis R. Bach and David M. Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, Frunce, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. 9256-2266 JMLR.org, 2015. URL http://proceedings..u mlr. Sae...pees/vvv v37 wtw.lliiicssteii1...ccm
Yang Song, Jascha Sohl-Dickstcin, Diedcrik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, TCLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.nen net, 2021.
URL httpps///pparrvveeeeeee//// wwt..//oooomiiiiIIII.....
Jan Stanczuk, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieb. Diffusion models encode the intrinsic dimension of data manifolds. In Forty-first International Conference on Machine Learning, 1CML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.ned net, 2024. URL htttps://ppereviiet htw.pppnrrvviieeeeefrr.... net/forum?
id=a0XiA6v256
Shuai Tang, Zhiwei Steven Wu, Sergül Aydöre, Michael Kearns, and Aaron Roth. Membership inference al tacks on diffusson models via quantile regression. CoRR, abs/ abs//2312.05140, 2023. doi: 10.48550/ARRM ARXIV 2312.05114.. URL https::/doi.org: 10. 48550/arXiv. 2312.05140
Joshua B Tenenbaum, Vin de Silva, and John C &Lanfford. A global geometric framework for nonlincar dimensionality reduction. science, 290(5500):2319 2323, 2000.
Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlci Zhang, Jarrid Rector-Brooks Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch
21

---

Published in Transactions on Machine Learning Research (07/2025)
optimal transport. Transactions Machine Learning Research, 2024. ISSN 2835-8856. URL https
//openreview netf/forummld--DSSa77All Expert Certification.
Roberto Verdecchia, June Sallou, and Luis Cruz. systematic review of green AL. WIREs Data. Mining. 1507
Knowl. Discov., 13(4), 2023. doi: 10.1002/WIDM.15000 URL htttps://doi.or////
Li Kevin Wenliang and Ben Moran. Score-based generative models learn manifold-like structures with constrained mixing. CoRR, 0bs/2311.09952, 2023. doi: 005550/ARXIV23111.0000 URL https ;
18850//rxxyy 2311. 2311.09952 09952
Andre Wibisono and Kaylee Yingxi Yang. Convergence in KL divergence of the inexact langevin algorithm with application to score-based generative models. Coffi,
2022. doi: 10.48550/ ARXIV.
2211.01512. URL htttps://doi.org111 48550/ arxiv. 01512
Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. A survey on video diffusion models. CoRR, 2ttp//22100000.0100 2023. doi: 10.48550/ARXIV. ARI.V.RXIIIOO www.22311..114...1. 10647. URL
https://doi. org/oo/////
wtwp:/aww.............. 10647
Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diflusion models: A comprebensive survey of methods and applications. ACM Compul. Sure., 56(44:106:1110::3, 2024. doi: 10.1145/3626233 URL otg:://ooooooo//////o/ /t.gg//...0
1ttLL:ppp.///......0000011111/// 1145/362635
Melike Nur Yegin and Mehmet Fatih Amasyah. Generative diffusion models: survey of current theoretical developments. Neurocomputingg 608: 2024. ISSN 0925-2312. doi: httpss//bbo..ogg//..000//..
2024.1 URL
Kohei Yoshimura and Sosuke Ito. Information geometric inequalities of chemical thermodynamics. Physical Review Research, 3(1), February 2021. ISSN 2643-1564. doi: 10.1103 htt.///pppsrrrrsscrch....111111 URL
http:/ /ttp::/dxx.ee doi org/10..go.o... w1w.///yyyyeeeeeserrc.h..mm 3.033175
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Bureu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregrossive models for content-rich text-to-image generation. Trans. Mach. Learn. Res., 2022, 2022. URL
Zhendong Yu and Haiping Huang. Nonequilbrium physics of generative diffusion models. CoRR, abs/2405.1199 11932, 2024. doi: 10.48550/ARXTT/ 24t.:/AAAAX......1113333 URL 11932
Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu The emer- gence of reproducibility and consistency in diffusion models. In Forty-first International Conference on Muchine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview. net, 2024. URL httppp/wwwieeeeeetoorortoeeeeeee
Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Truncated diffusion probabilistic models and diffusion-based adversarial auto-encoders. In The Fleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL
https://openrevieet
Zhengbang Zhu, Hanye Zhao, Haoran He, Yichao Zhong, Shenyu Zhang, Yong Yu, and Weinan Zhang. Diffusion models for reinforcement learning: A survey. CoRR, abs 2bs/2311. 2311.01223, URL https: //do:///ooii ott.oi..rg/......0 /8555//
1222111112000 .01223, 2023. doi: 10.48550/A ARXIV. 48550/ arXiv. 2311.01223
22

---

Published in Transactions on Machine Learning Research (07/ 2205)
A Related work
Diffusion models (Sohl-Dickstein et al. 2015 Ho et al. 2020 Song ct al
2021) have emerged as a powerful
class of generative models (Bond-Taylor Taylor et al.) 2022), demonstrating remarkable performance in various
domains including text-to-image synthesis (Nichol & Dhariwal
2021:
Dhariwal & Nichol, 2021: Ding et al.
2021
Ramesh et al. 2022 Nichol et al.) 2022 Rombach et al.
2022
Saharia et al.) 2002 Yu et al.) 2022 Ho
ct al.
2022a), text-to-speech (Chen ct al. 2021 Kong et al.
2021
Liu et al. 2023), vidco generation Ho
et al.
b022b
Singer et al. 2023 Xing et al.; 2023), natural languag processing (Li et al.
2022: Lou et al.
2023
Li ct al. 2023b), robot manipulations (Janerr et al. 2022 Zhu et al.
2023 inverse problems
DDras
et al.
2024). and protein interactions modelling (Abramson et al. 2024).
Motivations and related works. Our work is motivated by several recent theoretical advancements Yegii
& Amasyal 2024 and practical challenges (Chen et al. 2024 Yang ct al. 2024
Optimisation of Diffusion Time: Some empirical studies report existence of an optimal diffusion time that enhances model efficiency (Franzese et al. 2023).
There are some empirical and theoretical studies report desterogeneityynoo-nniiiie htt///ooonuuuuui uniformity, critical phenomena during gen-
Critical Phenomena and Statistical Thermodynanies of Diffusion Models:
eration (Ho et al.) 2020;
Meng et al. 2022: Choi et al. 2022 Zheng et al. 2023 Raya & Ambrogioni
2023 Georgiev et
2023 Sclocchi et al. 2024 Biroli et al.) 2024 & Chen 2024 Yu & Huang 2024
(2023) reveals spontancous symmetry breaking in diffusion models, dividing the generative dynamics into two phases: a linear steady-state around central fixed point and an attractor dynamics towards the data manifold. They linked the fixed points of the Bokker-Plancce equations to moments of spontaneous symmetry breaking in the Hessian of the potential functions and demonstrated an end-to-end asymptotic analysis in a simple discrete distribution supported on two points and some other toy examples. The authors also propose a Gaussian late initialisation scheme which improves model performance, generation efficiency, and increases sample diversity. The concurrent work (Li & Chen 2024) introduces thcoretical framework to understand phase transitions (they coined the term "critical windows" to describe the narrow time intervals in the generation during which specific features of the final image sample emerge, such as image class or background colour). The authors propose a formal non-asymptotic framework to study these windows, focusing on data from a mixture of strongly log-concave densitics. They show that these windows can be provably bounded in terms of certain measures of inter- and intra-group separation. [Biooli et al. (2024) employs statistical physics to identify three distinct dynamical regimes: initial noise, "speciation" transition, and "collapse" transition. SSloochi et al. (0024) examines the hierarchical structure of data in diffusion models and identifies phase transitions in the generative process with sudden drops in high-level feature reconstruction probability whereas the smooth evolution of low-level feature reconstruction. Georgiev et al. (2023) focuses on data attribution to provide a framework for identifying specific training examples that influence generated images. These previous approaches, ranging from empirical studies lo theoretical frameworks, provide valuable insights into phase transitions in generative dynamics; however, many of these methods face the challenge of requiring assumptions about the data. Our method offers new insights into phase transitions derived uniquely from the geometric structure of arbitrary data manifolds. Furthermore, our framework is essentially parallel to prior approaches, and therefore we expect to advance our understanding of phase transitions by deepening the relationship between the findings of previous research and our theoretical framework. A yet another recent work şIkeda et al. 2024), while not explicitly addressing phase transitions, outlines connection between diffusion models and non-equilibrium thermodynamics, featuring interesting discussions on the relationships between noise scheduling, generation quality, entropy generation rates, and optimal transport. Other intriguing studies from physics perspective include path integral interpretation of stochastic trajectories (Hirono et al. 2024 and Bayes-optimal denoising interpretation incorporating spin-glass perspective (Ghio et al.
Kadkhodaie et al.
2024 Zhang et al.) 2024 Raya & Ambrrgioni
2023
Geometrical approaches: There are some geometrical perspectives on diffusion models inspired our
work CChung et al. 2022 Wenliang & Moran
2023 Chen et al. 2023a Park et al. 2023 Ghimire et al.
2023
Chen et al. 2023c
Okawa et al. 2023
Oko ct al. 2023
23

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Other theories to understand diffusion and generation processes: A deeper understanding of these processes is essential for advancing theoretical research and practical applications, such generation control through prompting and interpolation. Recent studies have delved into the underlying mechanisms of diffusion and generation trajectories to identify optimal intervention points during the generation process, which can help achieve desired data outputs. While flo--macching algorithms have shown promise, in the practical user cases, diffusion models surprisingly sometimes outporform the llo-matching, underscoring the need to understand the factors contributing to this superior performance.
There several works on convergence guarantees for diffusion models (Bortoli et al.) 2021 Bortoli
2022
Block et al. 2020 Chen et al. 2023b Lee et al. 2022
Liu et al. 2022 Pidstrigach
2022 Wibisono
& ang 2022 Chen et al. 2023e
Lee et al. 2023
Li et a.. 2023a
Benton et al. 2023a b Chen et al.
2023d
Li el al.
2024
Flow matching techniques: Flow matching algorithms Lipman et al. 2023 Tong et al. 2024
are yet another prominent techniques in generative modelling. They are closely related to diffusion models flow matching often leverages diffuusio paths for training, in which optimal transport via ordinary differential equations (ODEs) yields straighter trajectories. It is very interesting to consider the influence the quality and diversity of generated samples critical dynamics such spontaneous symmetry breaking. Our method may have the potential to analyse these aspects. Such generative models considering a transport from one distribution to another are expected to contimue to develop, and geometric interpretations will further contribute to improving interpretability, efficiency, and control to ensure safety.
B Social Impacts
Green AI (Environmental Impact): Reducing the high energy consumption of diffusson models during both training and generation is crucial. The exponential increase in computational demands due to the growing use of diffusion models in industry poses significant environmental concerns. Optimising these models can lead to more sustainable Al practices, addressing the urgent need for eco-friendly AI technologies. Recent studies emphasise the need for environmental sustainability in AI, focusing on reducing the energy consumption and carbon footprint of AI models Verdecchia et al. (2023). Fairness, AI Safety and A Alignment: Ensuring AT safety and alignment is critical. This includes improving the mechanistic interpretability of diffusion models, optimising control prevent undesirable behaviours, and mitigating risks such hallucinations and adversarial attacks. Effective control mechanisms and interprelability can enhance trust and safety in AT applications. MMassunoto el. al. 22023
report that the diffusion time is the crucial for mitigating the membership inference attacks (MIAs) on
diffusion models (Pang et al. 2023 Pang & Wang] 2023; Duan et al. 2023 Tang et al.) 2023 Fu et al.
2023 Dubinski et al. 2024 Kong et al. 2023). Raya & Ambrogioni (0223) and Li Chen 20024 show
that phase transitions help understanding and controlling diversity in generation and Li & Chen 2024
also examines some relationship between phase transitions and MIAs.
C Mathematical Supplementaries
In this appendix, we quickly recall basic mathematical concepts and facts concerned with Linear Algebra and Manifold Theory. See, e.g., Lee (2013) for detail of Manifold Theory.
C.1 Formal operations in Linear Algebra
For the Euclidean space Rd and its linear subspace C Rdd let V denote the orthogonal complement of in Rdd
Proposition C.1. Let V and W be subspaces of R", Then the following hold:
(1) V C W if and only V 5 w
(2) V- v-n WV-
24

---

Published in Transactions on Machine Learning Research (07 (025)
C.2
Differentiable manifolds
In this paper, as manifolds, we treat only 'submanilolds of the Euclidean space Rdd So we adapt the following definition.
Definition C.2. A subset M of Id is called an n-dimensional manifold, if for each point I E M, there is an open neighbourhood U of a in Rd an open subset of V in Rd =R" x Rd-n, and a diffcomorphism Φ: U + V such that 6(M nU) V n (R" (0)). We call the map Φ a chart on M around æ.
Definition C.3. Let M C Rd be a manifold and æ E M be a point. Then the langent space TrM to M æ is defined as the set consisting of all velocity vectors of curves on M through æ, that is,
d7 (0) dt
T
7: (-e,e) M, 7(0) = x}
Notice that the tangent space forms linear subspace of Rd
Definition C.4. Let M C Rd and M' Rdd be manifolds, and let F: M M' be a differentiable map (i.e., there is extension F: U + Rd of F which is differentiable map open set U of R Then the differential dF& of F at is defined the linear map
dF M + Tr(a) dF d7 (0)
d(F 07) d(Fo7) (0).
dt
Remark C.5. Take charts Ø: U + V and th U + V on M and M' respectively. Also let (Z1...., .n) and (yn....,yre) yur) denote the coordinate on V C IR" and V c R"", respectively. Then the differential d.F.. is represented by the Jacobi matrix
a FoO dr
O( o Fo o 0000))) )) 0x,
(x)
(x)
of the map sb o F Foo-l: - V at the point æ E M.
Definition C.6. Let F: M + M' be a disferentiable map between manifolds. A point æ E M is called a regular point (resp. a critical point) if the differential oF T&M + Tr'm)-M surjective (resp. not surjective). A point y E M' is called regular value (resp. critical value) if every point æ e M satisfying that F(x) = y is a regular point of F (resp. or not).
The following is essentially consequence of Implicit. Function Theorem.
Theorem C.7 (cf. Lee (2013) (oroolayy 5.14]). Let F: Rd + Rd be differentiable map and y E Rd regular value of F. Then the level set.
(y)= = E Rd F(x) = y) C Rd
(y)
forms d manifold.
Remark C.8 (explicit description of the tangent spaces to manifold). Consider the same setup of Theorem
C.7 and denote F = (Fi.... Then the normal to the tangent space T.M coincides with
(F (a)
oF (x))T)
0x
dx
which is spanned by the gradient vectors of components of F. Therefore the tangent space itsclf is noting but its orthogonal complement, i.e.,
OF dx
OF
T.M =
(x)
(x)
dx
Definition C.9. A differentiable map F: M M' is called an embedding if differential dF: T. T' is injective for every point M and the restriction F: M (MM) 1) topological homcomorphism (i.e. there is the inverse map F-1, and both F and F-1 are continuous).
25

---

Published in Transactions on Machine Learning Research (07/202) 2025)
y
T
Figure 10: Image under E and tubular neighbour- hood of the cosine curve R²
Figure 11: Tubular neighbourhood of S embed- ded in RR
C.3 Tubular neighbourhoods
Let M RRd be a manifold. Recall the normal bundle
NM {(xx, u)
Rd Rd
2 T.MM
to M and the endpoint map
E: NM Rd F(x, = x
which are defined in $3 (Definitions 3.2 and 3.3
Definition C.10 (Tubular neighbourhood). A tubular neighbourhood of M noighbourhood of M in Rd that is the diffeomorphic image under of open subset V NM of the form
= (xx v) NM ||a|| < 8(x)}
for some positive continuous function 8 > IR.
Theorem C.11 (Theorem 6.24 in Lee (011
Every manifold embedded in IRd has tubular neighbourhood.
Proof. Let Mo denote the subset {(x, 0) æ E M} c NM. Fix a point a E M. Since both dif- ferentials df T Mo > T&M and d'IN.M NuM - N.M are isomorphisms, we have that dE: To Rd is also an isomorphism. By Inverse Function Theorem, the map E is dif- feomorphism on neighbourhood of (xr, 0) NM. We can take the neighbourhood to be of the form Va( ( o') E NM || æ'|l < 6, ||'l| < 8} for some d > O. Let p(æ) denote the supremum of all such 1. We can prove that the function p: M + R is positive and continuous.
Now consider the open subset = ((x, v) E NM ||인||| p(x)) of NM. Then the map E injective on V, and hence Elv dd smooth embedding. Thus E(V) is a tubular neighbourhood of M.
Theoretical supplementaries of Section 3
D.1 Proof of Theorem 3.7
Under the setup of Theorem 3.7 put to d and we define map so Idd Rd N+ by
((x v) = (F(z), v), ... (2 v)).
26

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
Notice that the normal bundle N. to M is expressed by
NM - & (0) {(x, v) Rd x Rd (p(a,, v) - 0}.
Hence the tangent space T(a NMM c Rd x R to NM at point (æ, v) coincides with the orthogonal complement of
#F1 T
UF T
Opp 7" dur T
Opx öm Opx the
T
0
Dv
in Rdd TRd (cf. Remark C.8
We now employ the Method of Lagrange multiplier. That is, we paraphrase the condition that point (z, v) NM is critical point of the endpoint map
= EolNM: NM + RR"
(i.c., the differential (E( m) Rd, which is linear map, is degenerate) as follows. First, the condition is equivalent. to that there exists non-zero vector of (a,,) NM which vanishes by the differential (dEo) Rd x Rd Rd, i...
(( NM Ker(Eo)oo)) 2 (0).
Moreover, we have the following:
No. Ker(dE)) v)) 2 {0)
0F,
OF dm 0
001 7 Oa
OcN T Ox
E1
Eed
0m
T Out
T do
n
El
ed
2
(0)
3
OF De
OF Oe
T
dkp T Oe 0ep
Bee 7
[e]]
Fed
C
Rdd
x Rd
e1
ed
R
where fei.... ea} denotes the standard basis of IR.. Here used property orthogonal complements
(see Appendix C.1
Finally, it is equivalent to that the matrix
aFT 0p1 T 0ep T
0px Ogy Bu
Ed
Ond
Fdj
is degenerate. Performing elementary row and column operations, and by the definition of Rí (M), the conclusion of Theorem follows.
D.2 Curvature and the first injectivity radius of a curve
Let M be curve in Rd i.c., a one-dimensional manifold embedded in Rd. We see that, in this case, the first. injectivity radius (M) is derived from the curvature of M as follows.
dr
Definition D.1. Let 7: R & be arc-length parametrization of the curve M, i...,
1. Then the
ds
curvature & of M at point 7(8) E . is defined by the Euclidean norm of the second order derivative dy
ds2 (s).
Proposition D.2. Assume that 1. Let 7: R + I be arbitrary regular parametrization of the curve
M. Then the curouture of M is computted by
k(r(u)) = (u)
(11)
where" denotes the differential by
27

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Although this is well-known fact, we show it briefty follows.
Proof. Let and u denote arc-length parameter and arbitrary regular parameter of the curve M. Since it holds that
7 = s' dy ds
(12)
we also have that
dy
7 s"
ds + (5')² K.
(13)
d ds²
Since two vectors dy and form an orthonormal frame ds
where denotes the normalization of the vector of the curve M, it holds that
||>|1 = + .??
(14)
Now notice the following: it holds that
1n'11= (8)2
(15)
by Equation 12 and hence
(1',")) = s"
(16)
Applying Equalions 15 and
16 Lo Equation 14h, we have the claim.
Theorem D.3. Assume that = 1. Let denote the curvature of M. Then Ri RM(M) coincides with the
infinum of radii of curvalure 1/k.
Proof. See Lemma 1 of Litherland et al. 1999
D.3 Comments the computation of the second injectivity radius
In this paper we used the definition of (2( as-is for the numerical estimation.
We note that one can weaken the condition appearing to the definition of R2(M) as follows.
Proposition D.4. The second injectivity radius R2(M) coincides with the infimum of the set
1 | {
Fi, 22 E M, 21 + F2 and æ 22 1 T M }
xell
2
Proof. See $4 of Litherland al.
1999
We also have comment (2M Numerically, it seems to be possible compute R2(M) by using the persislent homology of the given data cloud. Indeed, the topology of the e-neighbourhood of the data cloud might change when two tubes touch each other.
E Other examples of injectivity radii
We have already seen that Theorem 3.7 works in the case that data manifold is the unit circle S C IR2. In this appendix, we verify the theorem by observing other typical manifolds.
28

---

Published in Transactions on Machine Learning Research (07 (025)
E.1 Torus T
Let r' > r> O, and define a function F: R$ + R by
Flx y, 2) (V y² - 22
Then we have a torus T = (0) embodded in R.3 We can see that vector fields
t (-v,I,0) 0),
t2 (az, y2, r' V x² y2² 1²-1²²
satisfy the assumption in Theorem 3.7
Then the matrix Lr (1x, y, 2); (U1, U2, vs)) is calculated as follows:
v,, (1) 1z))
2(v2² y2² r')
+
201 2r1z F2 + r avg Va²+y²
2(v +y² - r¹)
y
212 2yv3 y2 +
r' yvs
22²
/x²4y²
rv + y02 + x² + y² r' / + y
2
We now parametrise the torus 2 by (x, y, 2) = ((r' + cos t) cos u, (r' + cos t) sin u. cos of T C RR, Then the vector (U1, U2, 13) makes Lya (zz, 2), (V1, U2 U3)) degenerate if and only if
(2,, U2, v3) = (r cos cos u, cos sin ris ()
(01, 02, 02)
r' + r cost
(r cos t cos u, cos ssi u, sin
+
cost
Hence we obtain Rí 22 min(r, r). injectivity radius is R(T²) min v',
Moreover we can see that Rg( min r' r). Thus the
E.2 Unit Sphere S
Define function F: R by
F(,,,,)) = 2² 1 11
Then we have FF) Considering the rotation in 1 around coordinate axes, that vector fields
(-V, t2 0-2,0,z)) ts (0, -2,y).
satisfy the assumption of Theorem 3.7 (Here notice that the number of vector fields which we desire is needed to be greater than 2, by topological reason.) Then the matrix L ((x, y, (V1, U2, 0z)) is calculated as follows:
Lga (1x, 2), (V1, U2, 03))
2x
02 + y
U3 +
2y
U1
0
0's +
2z
x
02
y
This matrix is degenerate ((x, y, 2), (V), 02, va)) E N if and only if (vi, 02, vz) (-xx, (-x,-y) yy, =2). Hence we
obtain Rí (S2) ((=z)² + y² + (-22) 1. Moreover it is clear that R²(S²) Thus the injectivity
radius is R(S2)= =
E.3 Unit Sphhee Sm
As the final example, observe the unit n-sphere. Define function F: B
R by
F(x1, 32, .,, fn41
2 x
29

---

Published in Transactions on Machine Learning Research (07/2025)
Table Estimated injectivity radii of various manifolds.
DATA SET
Ri
Ra
51
1.005:: 0.003
0.999=10.006
ga
1.063: 0.032
1.997=0.088
g
1.068 0.023
0.922=0.056
Then have S (0). Considering gradient vector fields of the height functions (21, 22, .,....,,,,tt,, Fx411 + 2j (j = 1.2.. + 1), we see that vector fields
(-x -- I5-12z, 2²), -2j+1 In+14j) (j= 1,
satisfy the assumption of Theorem 3.7
Then the matrix Lg (xx v) is calculated as follows:
T. (xr, v)
2x1
22 Eit1 2.4; +2
11+10 In+1
010 + 2102
In+1ll + In+1
2x
- 11 1 1 + 21
2x Un+1 Eien+i 144; #n+1
where a = (æ1, T2 Ix+1), (01, .2. vx+1). Now notice that for point a E S and a normal vector v to x, there cxists a scalar c E R such that v = cæ. Using it and performing the clementary row and column operations, the matrix Lg (ax v) is transformed as follows:
Cc E x
0
Tp+1
>.
Hence the vector v makes the matrix Lgn (x, v) degenerate if and only if c = -1. Hence we obtain R₁ (S*) || - æ|| 1. Morcover it clear that R2(S") 1. Thus the injectivity radius R(S")= =1
F Algorithm for Estimating the injectivity radius
this appendix, we show the pseudo- algorithm for estimating the injectivity radius (see Algorithm I and some preliminary numerical experiments verify the proposed algorithm.
F.1 Numerical experiments to validate AIER
For the S¹ S' S?, 5128 cases, the estimated Rj and R2 using the proposed algorithm are shown in Table 5 We first generate dataset using the exact generative equations and add some Gaussian noise. The F then approximated using a neural network. The following Step Step 4 are executed using the neural network approximation F. We note that condition defined in the Step 4.
the cosine similarity instead of inner products for the discrimination
30

---

Published in Transactions on Machine Learning Research (07/2025)
Algorithm Algorithm for estimating the injectivity radius (AEIR)
Input: data D C 1d
Step O: Estimate a map F = (Fi...., Fa-n): R + Rd-n such that the point 0 is a regular value of F and the manifold (0) C R2 approximates data D. Put M == F0).
Step 1: Estimate vector fields t,, tg.....tt. ty (n S N) defined near M such that for every a e M the vectors t (2), (z)).... ...., (xx)) span the tangent space M.
Step 2: Put gs: Rd x R + R, ge(x, v) (,)))())) (i 1,2,..., .N). Calculate the matrix
aa d=i
0F4-n
041
801 8vi
8px dwi
0px dei
0s1
0ei
[A,,". Ad-u; B1, ... Bu] =
aF 2zg
8F
0:1
0.1
0pM dxg
Dpx thia
0zg
where x = (z1,-.. zd), v (01,-.. vd).
Step 3: Collect sufficient amount of samples from the set
F(x) = 0, =0 (1 1,,
(x) E IR x Rd
d Ad-n: Bry....,,,, 0
(15 i1
in SN
and estimate min |||||| on the set. Put this value R1. Step 4: Collect sufficient amount of samples from the set
F(x1) F(x2) 0, x1 F 22
(x1, 22) Rd x Rdd
(m) 22, 14 (x1)) (a) 22 t. (xx2) = 0
1d=1,2, .)
and estimate min ||z| 222l on the set. Put this value R2. Step 5: Calculate R = min R1, R2}. Output: R, which estimates R(M).
G Non equilibrium thermodynamics and phase transitions
In [ong et al. 2021
score-matching Hyvärinen (2005) and diffusion-based (Sohl-Dicksteil et al. 2015
Ho et al. 2020 generative models have been unified into a single continuous-time score-based framework where the diffusion is driven by a stochastic differential equation. This framework relies on Anderson's 's Theorem (Anderson 1982), which states that under certain Lipschitz conditions on the drift coefficient f: Rd x Rd R and on the diffusion coefficient, g Rdd R Rd Rdd and an integrability condition on the target distribution po(xo), forward diffusion process governed by the SDE
dx: = fe(x:)dt + g:dw;
(17)
where w: is standard Wiener process. The probability distribution px(x) of the forward SDF 17 istiisfies the Tokker-Plande equation:
a oy P((x)= = V.
(18)
= =p(()) (z) + V. Inp((x) (zu(z))
(19)
where the potential ur(x)) defined as follows Raya & Ambrogioni 2023
(x) /" f((z)dz + oi In (x)
(20)
2
Here, we naturally consider the free energy for non-equilibrium thermodynamics 20111):
Esposito den Broeck
31

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Definition G.1. Non-equiiirrium free energy in the system
Fueg (t) : / pe (x)))
(21)
Theorem G.2. The con-equilibrium free energy can be rewritten as follows.
9 Ra
In Px(x) In Pea dr.
neg (t)
(x()
(22)
Poq (2)
Proof. By the definition of the potential
20
o6 V In (z)
V (x)=
(23)
When the target system in equilibrium, the solution Pog of the Fokker-Planck equation the following equation exists:
18 that satisfies
① Pog V. Pag Pog(x)
(x) (-fi(()) Le v. In (x)) )]
(24)
Therefore, rewrite the drift coefficient of the forward SDE 17 using Peg as follows:
f((z) L V. Pog (x)
(25)
From the above, we obtain the following relation.
I" fe(2)dz
gi In (x)
Ue (x)
(26)
gē Pt (x)
+ In Peq (oo)
(27)
2
Peq (x)
On the other hand, the free energy in equilibrium thermodynamics is given by: Definition G.3. Equilibrium free energy in the system
g In Peq (2o)
(1
(28)
Therefore, the two free energies have the following relationship:
Theorem G.4. From the non-negativity of KT--divereecc,, the following inequality is obtained.
-² Fneg (t) - (t) DKL. (x) (pe (x)] 20
(29)
2
When the target system is in equilibrium at time Logs ie., 8 pe = Po = O, the following equality holds:
Dim DRL.
Fneg(lon) Fog(log) 2
=0
(30)
According to Landau theory of phase transitions, phase transitions in equilibrium thermodynamics are identified when the higher-order derivatives of equilibrium free energy with respect to the order parameters
32

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A1. A2......... An exhibit discontinuities divergences. This criterion serves fundamental indicator for detecting phase transitions within the framework of equilibrium statistical mechanics:
8 Fe =
(31)
axp
On the other hand, it remains unclear whether a simple criterion for critical points, like the one mentioned above, exists for phase transition phenomena in non-equilibrium systems such as the diffusion processes represented by diffusion models.
In recent research Raya & Ambrogioni 2023); it has been demonstrated that the spontaneous symmetry breaking of the potential ue(x), plays a central role understanding phase transition phenomena in the diffusion processes represented by diffusion models. Specifically, the spontaneous symmetry breaking of the potential (z) occurs when the first derivative Vzu((x) and the second derivative "2 vanishes (x) V (x)) 0 at the critical point of the space-time (a, () = (a), where the fixed point of the Bokker-Plancee equation appear.
For instance, we consider a simple one-dimensional example Raya & Ambrogioni 2023 with dataset consisting of two points y-1 = -1 and 1/1 = -V-1 sampled with equal probability. Up terms that are constant in 2, the potential given by the following expression:
(e (e-0,)2
(2t
(x(x) = B())
x - In
2(1 +e 2(1-08) 87)
(32)
4
where 3(1) = Bmin 1(8max Bmin), Amax 20, 3mi 0.1. At the critical point (I, (0,4e), 0.293, the first derivative (z) and the second derivative (z)( vanishes Vzu((=) (z)((z 0.
Lemma G.5. By the definition of the Fokker-Planck equation 18 the Fokker- Planck equation satisfies the following relations at the critical point (x, = (Ke,, (c) Raya & Ambrogioni 2023):
i ar pr (0)
(0)
V Viu. (0) + v Inp:. V (0) =0
(33)
=0
The key insight that the fixed point of the Paaker-Planck equation (G))tt)= (0,tc) can be interpreted a8 spontaneous symmetry breaking in the potential function wx()). This phenomenon not only elucidates the emergence of phase transitions but also highlights the role of symmetry breaking as mechanism that governs such transitions in generative diffusion models.
Here, we discuss the universal properties of diffusion models that hold under more general potentials uy (x)..
Lemma G.6. By the intermediate value theorem, there exists zz(t) (t e [0, 1]) such that
Jec de (( = / (x() [V) + V. lap( [zu( do jdd
(34)
= (xa((t)))) + V (x) Vau(ze(t))
(35)
(36)
where Sax 2 (x) & JE (x)dd - and (z,t) (z))
Lemma G.7. By the definition of the Fokker Planckk equation 18
(x) px(x) In (xr)
(37)
ai
p(())) V (x) V. lmp(()) -v)(()))
(38)
We get the following equation:
1 + V Inp((x) Vzu(x)
(39)
0t
33

---

Published in Transactions on Machine Learning Research (07/2025)
Proposition G.8. We introduce the Fisher information [(t) of time:
((t) = /
① In pr(xx) dx
Pr(x)
(40)
Rd
al
We propose the criterion to identify the critical points (x, = (In), (ln)),a) (n) at which phase transitions appear in the diffusion processes:
8² (a =0
ar In
(41)
In other words, the phase transition in diffusion models occurs at critical point in space-time (tr,(t)), th), where the Fisher information degencrates.
Proof. By the intermediate value theorem, there exists (t)(t) (t E [0, 1J) such that
T(t) = / p((x) Inpu(c))
de
(42)
2d
0t
Inpe(on(t))
(43)
According the Cramér-Rao inequality for an arbitrary stochastic function (z(x,, the Fisher information
I(L) has positive lower bound Nicholson et al. 2020
Yoshimura & Ito (002] Ito (2023
e (0)))))
B3
S ((t)
(44)
Var [0)]
We define the extremum of the Fisher information follows:
0 ōt
8
(tt)
= Inpe(zn(t))
t- de (n
= 0
(45)
Ōt
t=ty
The two conditions that yield the extremum of the Fisher information
45
In (2.. (2x()))) = and
36
8 In (n 1=th 82
0, cannot hold simultancously, as demonstrated in the following discussion.
8² In (x) = dt2
32 ((n) (t) - Öt In (T)((a)))
a
(46)
If assume 0 In (t) 0, it would result in I(t) = 0, which contradicts the Cramér- Rao inequality that generally imposes positive lower bound 44 Therefore, the extremum of the Fisher information is
g² determined by ăह
In (x (=th 0.
34

---

Published in Transactions on Machine Learning Research (07 (07/2025)
6/
613
270
63
0.00
3325
(a) The time dependence of the Fisher information I(L) (Overview)
(b) The time dependence of the Fisher information I(L) (Zoom in)
Figure 12: The time dependence the Fisher information I(t). The red dashed line represents the time &e 0.293 which the potential breaks symmetry Raya Ambrogioni (2023). On the other hand, the magenta dotted line indicates the time t, 0.312 when the Fisher information predicted by our proposed method degenerates. While these two values close, they do not match exactly.
H Theoretical analysis on the empirical results
H.1 (t)
Let <> O. Let M(c) be the e-ncighbourhood of compact oriented manifold M in the Euclidean space Rd defined in Definition 3.1 Suppose Px()) is smooth solution to the Fokker-Planck equation (3) with an initial condition Po(x)= = 6m(()) here (x) is Dirac's density function with its support M. We define a function (t) as follows:
(1) = S Pe(x)dx.
(47)
Roughly speaking, this is understood counting function of particles within the e-neighbourhood of M. Proposition H.1. Assume B(t): R3o + R is smooth function and fe(x) {z()) 9((x) = BBt) in
5 (f is some smooth vector field). We have:
lim 5 dt
I (t) =0
and
2 (() 0.
lim T
t00 Øt
82
Thus there crists at least one ta in (0, +00) such that
Me (1o) 00. Moreover if 3(1) > and 012
(apa n -pp(()) 5
(48)
for any E QM(c) and any t E Ry0 then [M)() is strictly monotonically decreasing. Here unit outward pointing normal vector field along @M(c). In particular we can express the derivative of the function I (!) in terms of the free energy u defined in (2
(t)
Jasses
V ndx.
Ot
35

---

Published in Transactions on Machine Learning Research (07 (025)
Proof. (informal) We may compute for t 0:
i - Smco Ot P((())de
Öt (t))
B(t) S (Vx ((())(()f(z) + (ap((x)) de
= (t)) (-/amis p((x)((x) (-JaMo nds + Samii VePe(a) nds
(49)
The second equality follows since (x()) satisfies the Fokker- Fokker-Planck equation 3 The third equality follows from the divergence theorem where is the unit outward pointing normal vector field along @M(c).. The last limit follows since lim Pe(x) = (x) and in particular lim (x)) 0 and lim <xx((x) 0 in a.((e). To be (+0
(-00
(-0
more precise the convergence of the limit we could make use the following chain of inequalities:
(-Jamo f((x)f(a) nds + JAMIS VaPe(T) -nds) nds
I
B(t)
/ Samic (pe(z))ds + Jam(c) VxPe(r) n2ds
< 13(0)
xEBM(c) {)(()))}
max
$ |8()))
max zEaM(c)
{lf(z))}
sup zEOM(E)
[pe(z)]
1ds + sup wEBM(e) (eP((E)) Sance ids
aM((e)
= |8(t))
1ds
max BEBM((e
{If())
sup xEAM(e)
[p(())]+ sup
(M(())
wEBM(c)
When + 0o, (x,, tends to be stationary ie., lim
40 Se Pa(xx) = Therefore
lim to0
(t)
Øt
The existence follows from the mean value theorem. Finally let us show it is strictly monotonically decreasing. The negativity of ör (t) follows from 449h and 48).
The time when the score vector field reverses
In this section we discuss the details Section
1.1 Variance Preserving SDE and score vector field
We consider the widely used Variance Preserving (VP-SDE) (DDPM):
dY, = 3() 3() (B()dWW
(50)
with corresponding generative dynamics:
xx B(T t)V XX, T T -t 3( x)x, dt + VB(T t)aW,
(51)
2
One expresses marginal distribution (x) of Variance Presorving (VP-SDE) (DDPM) follows:
Pa(x) J N(y00sa, (1 ())))o(()dd
(52)
36

---

Published in Transactions on Machine Learning Research (07 (07/2025)
where 0₂ = e A(r)dr and po(v) is the distribution at time at O. The score at point x is given by
V. (n = (1 - (2) J. (y 0.x) (1 (1-))))00(10)))))0)))
02
(53)
1.2 Analysis of the behaviour of the score vector field at the boundary of the tubular neighbourhood of a hypersphere
Proposition I.1. Suppose = 5th is n-sphere of radius R in RR. We predict the following observation: Let be R >> >> O. Let n be wnit outward pointing normal vector to the bowndary of e-neighbourhood aM(c). Assume
c + (1 - 0.)(R
2 Vd.
(54)
02
x e 0M(() and Po(y) is constant C greater than on M. Then:
Vzlnp(()) n 5. O.
Proof. (this proof is yet informal. Although we only perform this proof for the case and M is a 1-sphere of radius 1, we hope it can be done in general dimensions). Since V In (npr(r)= Vepe(a) it is enough to prove pe(x)
(x) < 0. Performing change of variables --0.- we have:
V1-03
0, (1
/ (y 0zx) C2I, (1 1
VaPe(x) n = 02)
/
=C
wN (w; 0, L)dw n
9,M
1-
w
C
n w [wW[ww 0, I ddw
M-0,z
|w
V1-03 03
|
C
n w/ (N:; O, T)dw
w
(n: O, T)d
N4 |wl
C
n (2 (z)dz
Jga
|z]
C
n (2; 1)N,, (z)dz,
2
where M-B,- is the image of the manifold M by diffeomorphism + v-O.z and N_ (resp. N+) is VI-0z
Vi-oz
{w E M-Bat w n < O(resp. 0)}. dz is volume form of Rd. Let 0 be the angle between z/1zl
V1-67
and n. If use the polar coordinates (l= E (0, [0,2T), since cos(O т) cos(O), (put
(0) [(12).0) 0) c0] x [0, (0,2 N for some s.t. cos A T n))) may estimate q follows:
7/2
cos 1 (f,²
(1) =
| N 0.
de
(55)
7/2
+ /
n/2
cos (f,
1,1) (.) d0
7/2
7/2
cos 0 (1=0,12N(( P2N(, :0 |=0_| PN : 0, 1)) d0,
(
-7/2
37

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Co2g for some cæ 0. This integral is negative or zero if where set 20, E N 20 E EN and 20
(1=0 PN :0,7) |20_ 2 ( 0, 0
(56)
for any 0. Since x E 0M((e), Iz0_ N 2 =((1-8X(-5)) holds. Since [=|= 0, 1) is strictly monotonically vi-03
decreasing if ||| 2 22, the inequality holds for |20+1 2 |=0_ 2 22. Thus when c+(1-0,)8-4 assertion follows.
the
V
Example I.2. Let = St in R2 and |a| = 0.99. Compute 54 and we understand that V, In Pe (2) points toward 2 02 > 0.712. Similar thing can be observed for R3 Therefore Proposition explain the Figure and Figure 22
Remark I.3. Conjecture 4.4 is kind of generalisation of Theorem D.1 in Stanczuk et al. 2024 The authors predict can formulate more general conjecture by using concept of injectivity radii for more general manifolds to illustrate and explain the behaviour of the score vector field of more general diffusion models.
Proposition I.4. Suppose M a compact oriented manifold embedded in Rd. We predict the following observation: Let E be an injectivity radius. Let n be a unit outward pointing normal vector to the boundary of e-neighbourhoud M.(().. Assume
c [
02) 02
2 Vd.
(57)
E OM(c) and Po(y) is constant C' greater than 0 on M. Finally assume a line segment with x and the origin as its vertices does not intersect M. Assume moreover the following condition.
(i) For any y E M with (y- - -0x)) (y' -0,)).
there exists y' E M and some > O such that -cly 0xxx
(i) Assume that for each y M such that (y - 0zz) O, there crists M and 0 such that -clū 0,x) (y 0gr). Then e S 1.
(m) For any y E M, {cly - 0,z)(e > 0f MM is finite set.
Then:
V Impp(( n S O.
Proof. (this proof yet informal. Although we only perform this proof for the case d and M is a curve, we hope it can be done in general dimensions). Since V, Inp))()) V.pr(a) it is enough to prove Pe(x)
V VzP((x) n < O. Performing change of variables w g-0.x we have: Vi-oz
0,
/ (y 0,x) (1, (1- 03) M
Vap((T) n2 n= (1 -C 02)
C
WN (w: 0, I)dw
0,0
V1-02
w
C
n w// WNw 0, Ddw
M-0xz
11
/
| (ww; L)d [
C
nn/w|NN (w; D)w
N
[w
N, [
C
2
[2; 7)5x_ (=)dz
Jaa
C
(2; (z)dz,
(!)
ga
38

---

Published in Transactions on Machine Learning Research (07 (07/)025)
where M-0,2 is the image of the manifold M by diffcomorphism y y-0.z and N_ (resp. N+) is V1-0z
VI-02
Orrepp. 0)}. dz is volume form of R2. Let 0 be the angle between z/12 and n.. If we use the polar coordinates (lzel.0) E (0, x [0, 2 since cos(0 + T) = cos(O), (put N₂ (0) 0) E (0, oo] x [0, 2 2 E N for some s.t. cos A [ि n))) we may estimate ! as follows:
{w
M -ez, w
V1-62"
x/2
cos 0 (%/ |ze N2 0, 1)x (100l)))) d0 +
| cos (% Izol2 o, 0,1) (0) (120)))))) do
1/2
(1)
T/2
1112
xx/2
|=0+ [N :0 Σ 1=0_ PN 0, n)) d0
Cos 0
-2/2
$ C"
cos 1 (1z0.
[20 0,,7) - -1 N= T)) db,
(
-2/2
where 20+ E N 20_ E N and 20, = co200_ for some Ca > 0. If there is no such 20+, we set 28, = 0.
Also we set 120, [N(z, 0, = max [ (| (2, 0, [)} and |z0 N(z :0, 7) == min(120_ (2 : 0, T]}.
Thus by the assumption (ii) we may obtain 0 This integral is negative or zero if
(1z0, 1 (20, 0, |zg_ N : 0) 0
(58)
for any 0. Since OM(() and by the assumption (ii), by the lemma below [ze_
20
st (1-0,) holds.
VI-03
Since (= 0, I) is strictly monotonically decreasing if v2, the inequality holds for Thus when s+(=((1-0.) 2 22 the result follows.
|
|20 2 v2.
0
V1-03
Lemma I.5. In the situation of the proof above, we have the estimate:
+ |x| 0s)
VI 02
Proof. The assumption (ii), |ze, 2 Izg
is evident. For any 7 such that l/ - x]| = we have
by 0,æl 2 1/ 0.2
(59)
Since
max (
[ 0-4l + (1 0z), we have the result.
Remark 1.6. The smaller the injectivity radius slower time of the turning of the score vector field becomes.
39

---

Published in Transactions on Machine Learning Research (07 /025)
J Experimental Details
J.1 experiments detail for ddpm
In previous studics (Rya & Ambrogioni (0233) ). the training diffusion models was performed using DDPM. The number of time sleps is set to 1000, and the noise schedule coefficient. 3 linearly increases from 1.0 x 10-4 to 2.0 x 10-2, -2 A key difference from prior work is that, for denoising, the MLP layers have been replaced with 1D U-Net. This adjustment is necessary to handle higher-dimensional data, such 16D, 24D, and 48D, where more complex model is required.
The model is trained using the mean squared error (MSE) loss function, with AdamW as the optimizer, The learning rate set to 1 x 10-3, and the batch size 32. For toy data experiments, the training dataset consists of 50,000 points sampled from a uniform distribution. The model is trained without using any advanced samplers like DDIM, relying solely on the standard DDPM reverse process.
J.2 Wasserstein Distance Computation
Definition. Let X {....... mnd and Y (y) ......., 9m be the sets of generated and training samples, respectively. We regard each a discrete distribution:
m
6g.
T2
m
Then, the 1- 1-Wasserstein distance (Earth Mover's Distance) is defined by
Wi (() = min TETT(
ΣΣ
1 Ms Tij.
(60)
i= 1=1
where
11( 1/) =
Jzj
Jaj
Taj 20.
m
We use the POT library solve this optimal transport problem in practice.
J.3
experiments detail for the analysis of tubular neighbourhood in circle
In Section 5.2 we conducted experiments using a uniform distribution on the unit circle embedded a higher-dimensional Euclidean space. The red line plot shows the proportion of particles outside the tubular neighbourhood at each time step when generation is performed over 1000 time steps, the same as during training. Here, the injectivity radius that defines the tubular neighbourhood is set to 1. Therefore, being outside the tubular neighbourhood means that a particle's 's distance from the unit circle exceeds 1. In the generation process, a point is first sampled from Gaussian noise. When the ambient space is sufficiently large, the proportion of particles outside the tubular neighbourhood is 1. As the time steps progress during the generation process, each data point approaches the data manifold, which this case is the unit circle. Thus, at the final time step, all particles are expected to lic within the tubular neighbourhood.
The blue line plot evaluates the accuracy of data generation using the Wasserstein distance when initialisation is delayed during the generation process of the diffusion model. The horizontal axis, Diffusion Time, indicates the number of time steps performed out of the usual 1000-step generation process. For example, in the case of 800 steps, the initialisation delayed by 200 steps, with the generation ion beginning from Gaussian noise and proceeding for the remaining 800 time steps. Following previous studies, we refer to this as late initialisation.
We can calculate the calculation of the injectivity radius as (see. 3.3
J.4 experiments detail for the analysis of tubular neighbourhood in hypersphere
In Section 5.2 experiments were conducted using a uniform distribution on unit hypersphere embedded
in higher-dimensionall Fuclidean space. The experimental setup is the same in J.3. Given that the
40

---

Published in Transactions on Machine Learning Research (07/2025)
injectivity radius of the unit hypersphere is [ssee E.3). being outside the tubular neighbourhood implics that the distance from the unit sphere is greater than or equal to 1.
In Fig. 5 (see Section 5_3) we discussed that the discrepancy between the proportion of particles outside the tubular neighbourhood and the rise Wasserstein distance can be attributed to the increasing distance between the distributions. To support this hypothesis, we conducted an experiment where we initialized the Gaussian noise using the lateinit scheme, with X NN I/Vd), where d is the dimension of the ambient space. Corresponding to the experiment shown in Fig. ambient space of R4S, As shown in Fig. 13 we observed that the Wasserstein distance starts increasing particles begin entering the tubular neighbourhood.
performed another experiment S with an
0.99 Threshold
10
Probability of the particee outside tubular ghouthood
10a
0.8
Wazserstein Distance
9x
0.6
80
0.0
-x
1000
800
600
400
200
Diffusion Time
Figure 13: 220 embedded in 14
J.5 experiments detail for the analysis of tubular neighbourhood in ellipse, torus
In Section 5.3 experiments were conducted using uniform distributions on an ellipse and torus, both embedded in higher-dimensional Euclidean space. The experimental setup is consistent with that described
in
For the ellipse, given the semi-major axis 2a and the semi-minor axis 26, the injectivity radius is calculated with semi-major axis of 4 and semi-minor axis of 2, and another with semi-major axis of 6 and a semi-minor axis of 2. The injectivity radii for these cases are I/ and 3, respectively. We can calculate the injectivity radius of ellipse as follow. Let us verify Theorem 3.7 through ellipse, given the semi-major axis a and the semi-minor axis Define a function F: IR2 > I by
as 6² In this experiment, tested two cases:
2
F(z.8)
a
62
Then we have M = FO) One of the normal vector field on M is given as grad(F) (r by ( 2g 3), so (% -)) is a tangent vector field which spans the tangent space to M at each point (x, v) M.
OF
Applying Theorem 3.7 the first injectivity radius (M( is calculated as follows. For point (z, y) M, the matrix
LM((z, (11, 02))
41

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Pproportion
0.1
0.5
0.9
0.95
0.99
0.999
1.0
Dataset
Ellipse (R = 2, 1) embedded in I
3.690
2.351
1.110
0.926
0.593
0.441
0.211
Torus (R -2, embedded in RI6
1.328
0.816
0.563
0.520
0.440
0.333
0.149
is degenerate (i.e., its determinant is zero) if and only (U1, U2) = (- b'z² 6' 6'm"+a'p"
Let (a, y)
2202x4 -'y
+
(a cos 0, bsin 0), then, (V1, 02) = (- cos? 04a" sin?
6² cos" @+a"sin²0)
The L norm of (21, 02) is minimized
cos to wad
Beord +bsin BE
wes at 0 when b, and in this case, (M)
62
For the torus, the injectivity radius is given by min(r' r), where r' is the major radius and is the minor radius (see E.1 In this experiment, we used two cases: one with a major radius of and a minor radius of 1, and another with a major radius of and a minor radius 1, In both cases, the injectivity radius is 1.
These calculations provide the injectivity radii usod our experiments on both the cllipse and the torus, guiding the analysis of the lubular neighbourhoods in these geometrie settings.
In the experiments presented in Section 5.3, included figures for ellipse with major axis of 6 and minor axis of 2, as well as torus with major radius of 3 and a minor radius of 1. Here, we provide addit ional figures for an ellipse with a major axis of 4 and a minor axis of 2, and a torus with major radius of and a minor radius 1.
000 touffool
101
Threshold
101
Threshold
Wasserstein Distance
0.8
Wasserstein Distance
0.8
0.6
0.6
10"
10a
0.0
0.0
1000
600
600
400
200
1000
800
600
400
200
Diffusion Time
Diflusion Time
Figure 14: ellipse R = 2, r = embedded in R
Figure 15: torus R = 2, r = embedded in 1R
J.6
experiments detail for the analysis of tubular neighbourhood in DisjointArcs cases
In Section 5.4 we conducted experiments using a data distribution composed of segments from two circles with different curvatures, both embedded in higher-dimensional Euclidean space. The dataset was constructed by uniformly sampling 50,000 points from two regions: one segment from circle with radius 1, covering the angle range from 7/6 to n/3, and another segment. from circle with radius 2, covering the angle range from 7 6// to 4T/3.
Next, we consider appropriate values for the injectivity radius. For the submanifold A, the injectivity radius is considered to be 1 (although, strictly speaking, the injectivity radius is undefined at the endpoints where the tangent plane cannot be properly defined, we exclude these points for our analysis). On the other hand, for the submanifold B, since a part of a circle with radius 2, the injoctivity radius is considered be 2. Therefore, the injectivity radius for the combined manifold formed by these segments determined to be 1.
J.6.1 Additional Experiments
To further investigate the behaviour of the score vector field under different curvature settings, we conducted
additional experiments using new datasets. These datasets include:
42

---

Published in Transactions on Machine Learning Research (07 (07/2025)
1. A segment from circle with radius 3, covering the angle range from T/6 to n/3, and segment from a circle with radius 1, covering the angle range from 7 to 4 4T/3.. See Figure 16 for the experimental result. To further analyse the behaviour of the proportion of particles outside the tubular neighbourhood (depicted as the red solid line in the experimental results), we conducted additional experiments to investigate how this behaviour changes with different values of the neighbourhood radius. Specifically, we considered a neighbourhood radius of injectivily radius 1). The proportion of partieles outside this larger neighbourhood region is plotted red dashed line in Figure I8 This result demonstrates that the behaviour of the red line varies depending the chosen value for the neighbourhood radius.
(which different from the
2. A segment from a circle with radius 3/2, covering the angle range from x/6 to x/3, and a segment from a circle with radius 1 1/2, covering the angle range from 7x/6 to 4T/3. See Figure 17 for the experimental result.
17
C.
Difasio
Figure 16: Disjoint ares case R = 3, embed- ded in R16
Figure 17: Disjoint are case R = 3322,r 1/2 embedded in R16
Figure 18: Disjoint are case R = 3, embed-
ded IR16
J.7 experiments detail for the analysis of tubular neighbourhood in natural dataset
In Scction 5.5 we performed experiments by embedding real-world datasets such as MNIST and Fashion MNIST onlo hypersphere, subsequently mapping them into a high-dimensional Euclidean space. For efficient generation and sampling in diffusion models, it is common practice to reduce the dimensionality into a latent similar to the approach in Latent Diffusion-and consider the transitions of the diffuusion model within this space. While Latent Diffusson utilizes VQ-VAE for embedding, we employ a Hyperspherical VAE in our methodology.
In our approach, several key modifications were made to the original hyperspherical VAE (sVAE) (Davidson) et al. 2018) setup used in prior studies. One significant change was transitioning from binary data represen- tation, where data was handled as binary values, to continuous data representation. As result, we replaced
43

---

Published in Transactions on Machine Learning Research (07 (07/2025)
the Binary Cross-Entrope (BCE) loss function with the Mean Squared Error (MSE) loss function. This modification allows for more accurate modelling and reconstruction of continuous data, particularly when working with datasets such as MNIST and Fashion MNIST.
Furthermore, to address the limitations of the previously used MLP layers for reconstructing natural images, we enhanced the model by adopting CNN-based layers. This adjustment particularly beneficial for reconstructing images from higher-dimensional latent spaces.
To demonstrate the improvements, present comparison between the normal VAE (nVAE) and the hyperspherical VAE (sVAE), focusing on the ELBO (Evidence Lower Bound) and log-likelihood (LL. values for different latent space dimensions (10, 15, and 20). Additionally, we compare the performance of both nVAE and 8VAE when using MLP-based and CNN-based architectures. The results are summarized in the following tables:
ELBO (4) LL (4)
nVAE
SVAE
dim 10
-23.7
22.9
-25.5
-24.0
dim 15
-23.8
-23.1
-26.7
-24.5
dim 20
-23.8
-2311
-27.6
-25.0
Table 6: Comparison of ELBO and LL for nVAE and sVAE with different latent space dimensions, using ML.P-based models on the Fashion MNIST dataset.
ELBO (4) / IL (4)
nVAE (CNN)
SVAE (CNN)
dim 10
-22.8
-22.1
-24.4
-22.7
dim 20
-23.1
-22.2
-26.8 -23.9
Table 7: Comparison of ELBO and LI. for nVAE and sVAE with different latent space dimensions, using CNN-based models on the Fashion MNIST dataset.
In Section 5.5 we explored the effectiveness of late initialisation accelerating image generation by embedding natural images onto a hypersphere and sampling on the hypersphere. is crucial for practical applications to ensure that points sampled through late initialisation in the latent space can generate realistic images when passed through the decoder. Here, we present generated images obtained by passing points sampled using late initialisation during the generation process of the diffusion model through the decoder of the hyperspherical VAE, and qualitatively evaluate the results.
44

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Images
2
6
1
8
5
/
/
2
1 5
7
5
X 2
9
7
7
7
5
7
3
9
6
3
7
9
2
3
9
1
3
2
3
8
2
2
O
/
2
3
0
5
8
1
7
4
5
2
6
2
8
6
8 5 2
/
6
2
6
3
/
1
7.
2
7
6
2
8
9
9
2 9 9 0 0 7
2
9
1
3
5
9
9
2
8
1
3
9
0
6
8
8
0
8
3
3
8
0
7
2
9
6
8
6
5
7
3
3
9
9
9
0
3
/
0
7
3
1
9
8
9
9
3
0
1
0
8
3
2
7
S
S
7
7
5
1
7
5
8
0
6
2
0
2
3
2
8
7
7
2
4
3
4
6
1
9
8
2
3
6
0
8
/
9
2 7
4
7
2
8
9
2
5
7
8
4
4
/
2
7
/
7
0
7
3
4
2
2
/
7
3
4
9
4
3
7
7
3
4
6
4
3
9
3
8
7
3
8
3
6
4 3
5
7
3
8
3
7
g 3
5
2
8
3
7 3 79 6 25 22
a
7
7
6
2
8
9
7
9
7
7
4
5
7
7
3
4
7
6
0
8
7
7
7
9
5
3
0
3
7
4
Q
5
3
3
7
4 O
5
x
6
D
5 9
0
4
8
6
2
2
7
1
0
7
8
8
2
2
9 2 3
6
0 4
8
8
G 4 8
2
6
$
0
/
1
4
2 6 2
5
6
5
0
9
4
3
6 5
0
0
4
0
2
0
b
4 3
2
3
1
2
4 8
0
2
4
3
8
8
8
/
A
7
6
3
8
4
8
7
4
5
6 3
8
2
8
Figure 19: MNIST images decoded using SVAE after diffusion times of 1000, 500, 200, 100, 50, and (arranged from left to right, top to bottom) in the latent space.
Images Late time
Images Late, Sine
Images at Late Sime 800
Images
Images
Images
Figure 20: Fashion MNIST images decoded using SVAE after diffusion times of 1000, 500, 200, 100, 50, and (arranged from left to right, top to bottom) in the latent space.

---

Published in Transactions on Machine Learning Research (07 (025)
There are limitations regarding embedding onto hyperspheres. Previous studies have shown that the dimensionality of the embedding onto the hypersphere increases beyond a certain value, the accuracy of the embedding decreases. This phenomenon is related to the fact that the surface area of a hypersphere approaches zero the high-dimensional limit. Therefore, although we conducted experiments with MNIST and Fashion MNIST, for larger datasels; the accuracy of the embedding would deteriorate to the point where considering diffusion model would no longer be meaningful.
However, despite these current challenges, there potential solutions. Previous research focused embeddings onto unit hyperspheres, but it is possible to consider hyperspheres with radius of vn. When the dimensionality is n, the surface area of hypersphere with radius of vn increases monotonically, suggesting that the embedding could remain effective even as the dimensionality increases. In this case, efficient generation using diffusion models that leverage the concept of tubular neighbourhooss could become meaningful.
Analysis of score vector
K.1 Score Vector field
We present additional experiments detailing the score vectors of DDPM. This section includes two experimental setups concerning the score vector field. Firstly, for the S case, the experimental setup includes grid size of 32 32 and trained DDPM with 7 1000. The training data, is St, with the red circle at the centre representing S*, See Figure 21 for the corresponding visualization.
Secondly, for the S case, the experimental setup includes grid size of 16 x 16 16 and a trained DDPM with 7' = 1000. The training data is S². Except for the grid size and training data, all other settings remain the same. See Figure 22 for the corresponding visualization.
Thirdly, for the ellipse case described in Section 5.3 we visualized the score vector field for the initial two dimensions of 16-dimensional latent space. The experimental setup involves training data generated from ellipse with radii R = 2 and == The grid size is 32 32, and the visualization highlights the behaviour of the score vectors in these two dimensions. See Figure 23 for the corresponding visualization.
Finally, for the disjoint ares case described Section 5.4 we visualized the score vector field for the initial two dimensions 16-dimensional latent space. The experimental setup involves training data composed of two disjoint arcs, one from circle with radius and the other from circle with radius r = The grid size is 32 32, and the visualization illustrates the interactions between the two ares. See Figure 24 for the corresponding visualization.
46

---

Published in Transactions on Machine Learning Research (07/202) 2025)
4.403
4.40
-.322
-3322
22245
-2244
1.56
2
0.086
0.086
(a) T=0, Sa
(b) T=500, S²
4,403
4.401
3.322
-3322
-2243
-2.44
0.008
0.005
(c) T-785, S²
(d) 1-1000, S'
Figure 21: Time evolution of score vectors in the backward process of DDPM, $1
K.2 Square of the Jacobian the Score Vector Field
In this section, we extend our analysis to the square of the Jacobian of the score vector field. We utilise updated experimental setups for both the 2D S and the 3D S2 cases. For the 2D S case, the grid size is 128 x 128 with trained DDPM using T = 1000. The training data remains St, and compute and analyse the square of the Jacobian of the score vector field for this setup (Figure 25
47

---

Published in Transactions on Machine Learning Research (07/202) 2025)
00.75
1.5
0..75
1.5
0.50
00.50
00225
II
10225
0.00
1.0
00.00
1.0
00.2
0.5
0.5
1.0
1.0
0.5
0.0
0.5
0.0
-0.5
-0.5
0.0
0.5
0.0
(4) T=0, Sa
(b) T-500, 8a
1.00
1.00
10.75
-1.5
0..75
115
XI
00.50
-0.55
1.0
1.0
0.22
-022
0.55
00.50
0.75
0.5
0.75
1.00
1.00
0.5
0.0
0.5
0.0
-1.0
0.0
-1.0
0.0
0.0
0.5
0.0
0.5
0.5
1.0
-1.0
0.5
1.0
-1.0
(c) T-823, S
(d) T=1000, S
Figure 22: Time evolution of score vectors in the backward process of DDPM, $2
Similarly, for the 3D S² case, the grid size is 128 x 128 128 with trained DDPM using 7 = 1000. The training data remains 5 and we compute and analyse the square of the Jacobian of the score vector field for this setup (Figure 26
48

---

Published in Transactions on Machine Learning Research (07/202) 2025)
-424
4214
-2.25
33175
1,088
1.090
0.050
0.060
(a) T=0
bb) T-500
4226
-224
-3175
33175
1.99
0,060
0.060
(c) T=700
(d) T=800
-3175
33175
-2137
2.137
-1
-2
49
-1

---

Published in Transactions on Machine Learning Research (07/202) 2025)
-4328
4.328
-2236
3225
11.10
6.037
0.037
(a) T=0
bb) T-500
4.320
-.320
3325
33256
2.18
0.037
0.037
(c) T=700
(d) T=800
3.236
33256
-2183
2.83
-1
-2
50
-1

---

Published in Transactions on Machine Learning Research (07/2025)
0.00012
-0.00012
0.20008
-0,00008
0.20008
0.00000
-0.00004
0.00004
0.00002
0.00000
(a) T=0, s¹
(b) T-500, s
00000022
-0.00012
-0.00008
-0.00008
0.20004
00.00044
0.00002
(c) T=785, S²
(d) 1=1000, S
Figure 25: Time evolution of the squared Jacobian of score vectors in the backward process of DDPM, 5
51

---

Published in Transactions on Machine Learning Research (07/202) 2025)
1.0
-155
1.0 16-7
0.5
0.5
0.0
0.0
0.5
0.5
0.5
1.0
1.0
1.0
0.5
0.5
-0.5
0.0
0.0
-10
-1.0
(a) T=0, 5a
(b) T-500, S²
1.
1-0
1.5
1.5
0.5
1.0
1.0
0.5
0.5
1.0
1.0
1.0
1.0
0.5
0.0
0.5
0.0
-1.0
0.0
0.0
0.0
-.5
0.0
-05
0.5
1.0
1.00
0.5
1.0
11.0
(c) T-823, 8
(d) T 1000, 5
Figure 26: Time evolution of the squared Jacobian of score vectors in the backward process of DDPM, 22
52


Reviews:
[{"review_id": "OAs6ojbz5D", "reviewer": "Reviewer_RQ3i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n- The idea of using Tubular neighborhoods for describing phase transition (feature emergence) is creative and to my knowledge is novel. It provides a nice geometric perspective on emergence of features which is linked to phase transition. \n\n- The writing and the back ground is generally clear although it can be improved. \n\n**Weaknesses:**\n- Limitations and assumptions of the theory can be more clearly delineated (see next section)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The goal of this paper is to offer a new and different perspective on phase transition observed in previous work during the generative process of diffusion models. To this end, the authors borrow some ideas and theorems from the manifold analysis and differential geometry literature, notably, tubular neighborhoods and injectivity radius. They propose an algorithm to estimate the injectivity radius of data manifold, which they employ on simple synthetic data as well as real data. They demonstrate empirically that phase transition happens around the borders of Tubular neighborhoods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Necessary adjustments:**\n- An important condition required for the analysis is the smoothness of the manifold (also finite curvature). For clarity, it’s best that these assumptions are stated early on in the text. This is specially important in the context of images, since there is both empirical evidence and theoretical arguments that these assumptions do not hold for image distributions. \n- Beyond simple analytic manifold, how does one measure the infectivity radius for a data manifold? It is not clear if that is possible for real world data manifolds. But having access to this value is required to then use the analysis in the paper. One solution to this problem, implemented by the authors, is to embed the image manifold into a known simpler manifold as a preprocessing step. But the problem with this approach is that then the resulting analysis does not tell us much about the actual manifold and trajectory towards it but an $S^{20}$ manifold. (And it is not clear if even such trick would hold beyond MNIST)\n\n- The link between the injectivity radius and Wasserstein distance is not made clear in the text, resulting in a disconnect between the theory and experiment parts. The relationship can be made more explicitly discussed. Why is it a relevant measuremtment? \n\n- Generally my main criticism of the work is that although using the idea of injectivity radius is an innovative and geometric way to explore the phase transition phenomenon in diffusion generation, and the authors successfully capture the relationship between injectivity radius and phase transition in smooth simple manifolds, the paper is not convincing that this analysis would hold in the case of the image manifolds. So my proposal is to edit the text to reflect the scope and limitations of the analysis \n\n **Recommended adjustments:**\n\n- A more explicit and in depth discussion of phase transition phenomenon in an introduction or background section will improve the motivation and analysis proposed in the paper. (This is as opposed to only citing the papers which introduce this observation). Since the entire paper is built on the assumption that these abrupt phase transitions exits and are detectable, it seems necessary to first establish them for the reader. (Again this is important for manifolds arising from real work data beyond simple manufactured manifolds). \n- In section 2.2, the manifold hypothesis is attributed to more recent work. However, this hypothesis was introduced earlier: \n  - Roweis, S.T. and Saul, L.K., 2000. Nonlinear dimensionality reduction by locally linear embedding. science, 290(5500) \n  - Tenenbaum, J.B., Silva, V.D. and Langford, J.C., 2000. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500)\n  - Probably more original paper which can be found by a quick search \n- In the experiment section, the more interesting manifolds are the Ellipse and the Torus, because in these cases the injectivity radius can be manipulated by changing the major and minor axes. The hypersphere cases do not carry much information in my opinion, so they can be removed from the text. Instead you can show how the tubular neighborhood changes in an Ellipse with different major to minor axes ratio (which can be more revealing of the theory in the earlier part).", "broader_impact_concerns": "In my opinion the broader impact of this type of work which aims at shedding light on understanding deep neural network is positive in the long run.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wsJhtqdcSQ", "reviewer": "Reviewer_L9AT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- Idea: The idea of injectivity radius and tubular neighborhoods for understanding diffusion model phase transitions is highly creative and highly original \n- Clarity: The theoretical parts of the paper, even though they involve more advanced mathematical concepts/new definitions, are generally clearly written. The motivation of the paper is also presented clearly. \n- Evidence: The experimental evidence, while focusing on relatively simple cases, covers enough ground to illustrate the plausibility of the theory. \n\nWeaknesses: \n\n1. There are a number of spelling mistakes, capitalization/other formatting issues in the appendix of the paper. I urge the authors to take a detailed pass to fix these grammatical/spelling issues. \n\n2. This is not really a weakness, but I still would like to point out a limitation of the study. Basically, the authors introduced the notion of tubular neighborhoods and injectivity radius, and showed some theory that the probability moves towards the manifold. There is actually little theory that points towards a phase transition (if I am incorrect in my understanding I welcome any corrections). The evidence towards the relevance of the injectivity radisu towards phase transition come purely from the experiments. It would be nice to see some theory that actually links the injectivity radius to phase transitions. However, the current material in the paper more than suffice for TMLR, and I am NOT requiring the author to make any edits/do anything about this. I am just suggesting a future direction.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a new concept known as the injectivity radius to explain the emergence of phase transition phenomena in diffusion models. The core intuition is that once the diffusion process enters a certain so called tubular neighborhood, where projection to the manifold is uniquely determined, generating data that conforms to the structure of the manifold becomes easier, and that the boundary of such tubular neighborhoods help understand phase transition. The authors propose an algorithm to estimate the injectivity radius and empirical evidence to support the theory.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- correct any typos and formatting issues.", "broader_impact_concerns": "No ethical concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hNWcS5fFc1", "reviewer": "Reviewer_Mx5F", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nThe paper appears to provide a novel perspective with potential for increasing sampling efficiency in diffusion models. The theoretical analysis seems thorough, and the empirical study confirms its results.\n\nWeaknesses\n\nThe mathematical analysis is very dense in parts. Adding some comments to set the theoretical results into perspective would help the paper be more accessible. Also, assumptions should be discussed a bit more. Overall, the language is good, and the paper is relatively easy to follow. I comment on a few issues below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors provide a novel perspective on phase transitions in diffusion models. In particular, they analyze them from a geometrical perspective. Leveraging tools from tubular neighbourhood theory, they can define regions where phase transitions occur, which is validated by empirical results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) Especially in Section 4.2, the analysis gets very dense. The authors should try to comment more on their results. Stating beforehand what we will prove now and why and afterward commenting on the implications of the results.\n\n2) In Section 2.2, the authors simply state that their paper is based on the hypothesis that data often concentrates around a lower-dimensional manifold. Could you comment on the implications of this assumption? Also on the one following, that you assume all data manifolds are compact and embedded in the Euclidean space?\n\n3) Could you elaborate a bit more on Remark 4.6? In what sense is it a generalisation, and what exactly is the result that you are citing?\n\n4) There are a few language issues/typos. Between (2) and (3), there seems to be an article missing before \"probability distribution.\" Also, the \"we could\" reads a bit strange, maybe that could just be deleted. In the second sentence of Section 3, there is an \"in\" too much. Before Theorem 6, I think the \"come into\" is too much.", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)