Are We Really Learning the Score Function? Reinterpreting Diffusion Models Through Wasserstein Gradient Flow Matching
Authors: An Vuong
Michael Thompson McCann
Javier E. Santos
Yen Ting Lin
Venue: Accepted by TMLR ()

Abstract: Diffusion models are commonly interpreted as learning the score function, i.e., the gradient of the log-density of noisy data. However, this learning target is a conservative vector field (i.e., a vector field that is the gradient of some function), a property not enforced by neural network architectures used in practice. We show numerically that trained diffusion networks violate both the integral and differential constraints that conservative vector fields must satisfy, indicating that the learned vector fields are not score functions of any density. Despite this, the models perform remarkably well as generative mechanisms.  To explain this paradox, we propose a new theoretical perspective: diffusion training is better understood as \emph{flow matching} to the velocity field of a Wasserstein Gradient Flow (WGF), rather than as score learning for a reverse-time stochastic differential equation. 
Under this view, the "probability flow" arises naturally from the WGF framework, eliminating the need to invoke reverse-time SDE theory and clarifying why generative sampling remains successful, even when the neural vector field is not a true score. We further show that non-conservative errors from neural approximation do not necessarily harm density transport.  Our results advocate adopting the WGF perspective as a principled, elegant, and theoretically grounded framework for understanding diffusion generative models.

Published in Transactions on Machine Learning Research (12/2025)
Are We Really Learning the Score Function? Reinterpret- ing Diffusion Models Through Wasserstein Gradient Flow Matching
An B. Vuong" Department of Electrical Engineering & Computer Science Oregon State University
vuongat@orrgonsslllecco
Michael T. McCann
mocann@lanl..om
Los Alamos National Laboratory
Javier E. Santos
jesantos@lanl.co
Los Alamos National Laboratory
Yen Ting Lin Los Alamos National Laboratory
yentingl@lanl. gov
Reviewed OpenReview: https: // openreview. net/ pet/ffrum? id= CZyJqXQRNJ
Abstract
Diffusion models are commonly interpreted as learning the score function, i... the gradient of the log-density of noisy data. However, this learning target a conservative vector field (i.e., vector field that is the gradient of some function), a property enforced by neural network architectures used in practice. We show numerically that trained diffusion networks violate both the integral and differential constraints that conservative vector fields must satisfy, indicating that the learned vector ficlds are not score functions of any density. Despite this, the models perform remarkably well as generative mechanisms. To explain this paradox, we propose a new theoretical perspective: diffusion training is better understood as flow matching to the velocity field a Wasserstein Gradient Flow (WGF), rather than as score learning for reverse-time stochastic differential equation. Under this view, the "probability flow" arises naturally from the WGF framework, eliminating the need to invoke reverse-time SDE theory and clarifying why generative sampling remains successful, even when the neural vector field is not a true score. We further show that non-conservative errors from neural approximation do not necessarily harm density transport. Our results advocate adopting the WGF perspective as a principled, elegant, and theoretically grounded framework for understanding diffusion generative models.
1 Introduction
Diffusion models are typically described as follows: Given D-dimensional samples E RP drawn from data distribution μo, one defines forward Itô process that gradually corrupts into noise. Throughout this (OU) process for concreteness!
paper, we use the contimuous-time
dX,- = X,, di v2dW,,
X, x Pos
(1)
Work partially done during the Applied Machine Learning Summer School at LANL.
Santos Lin (2023) established the equivalence of the OU process with the discrete-time Denoising Diffusion Probabilistic Model Ho et al. (2020) and the score-based fornulation (Song al., 2021). This sctup often called "varianco-preservviii time (which, in most scientific contexts, analysis extends naturally the standard Brownian motion
(VP), though this term is misleading: for each sample, the variance is not constant the definition of "preserving"), but grows as v1 21, process dX, dW, termed "variance-exploding (VF) by Song al. (2021).

---

Published in Transactions on Machine Learning Research (12/2025)
where each component of W, is standard D-dimensional Wiener process. The process Eq. (1) converges to the limiting distribution Moo as + 0o, which is an isotropic Gaussian in IRD Due to the diagonal form of the drift and diffusion terms, each component of Xx follows the canonical one-dimensional OU process.
Equivalently, the forward dynamics can be described in terms of densities. The transition kernel p(g, tls, s) satisfies the Fokker-Planck Equation (FPE):
0,p(E,(((,8) VElE (()(E
(2)
with the initial condition p(s. = 8(5 - x) for each of the drawn samples x ~ Pos where 6(-) denotes the Dirac delta distribution.
The modern understanding of diffusion models is grounded in Anderson's theory (Anderson, 1982), which guarantees the existence of a reverse-time Itô process that transforms samples from the simple, D-dimensional isotropic Gaussian, distribution Moo back into data-like samples as t: 0o - 0:
dx, = , + 28 (X,, -T]] + V2dW,, X-o0 ~ Moa-
Here, we define := -t, T -0 - O, p(x,t) denotes the forward density with initial distribution μoa ((. == Ve log P(S, E RD is the score function of the corrupted (forward) distribution given the initial distribution Noo and dW, is again D-dimensional Wiener process. The central training objective of diffusion models is thus framed as learning the score function s(z, t) (Song et al., 2021). In practice, a neural network R² + RD is used approximate s(x, which is then plugged into Eq. (3) during sampling.
A key point is that the score function has distinct mathematical structure: it is conservative field, the gradient of scalar field log p. Neural networks used in practice are not constrained to produce conservative vector fields and, therefore, do not necessarily preserve this structure. This raises the central question of this study:
Does a trained neural network actually learn a valid score function, or merely a useful vector field for generative sampling?
2 Literature Review
Our work attempts to connect the theory of Diffusion Models (DMs) and Wasserstein Gradient Flows (WGFs). Diffusion-based generative modeling originated from the observation that adding noise to data and then removing it can reveal its underlying structure, principle first formalized in the denoising autoen- coder by Bengio et al. (2013), which viewed training as lcarning to invert a stochastic corruption process. This idea was later generalized by Sohl-Dickstein et al., establishing a forward reverse Markov chain for- mulation that defines data generation as a learned inversion of a diffusion process. The seminal Denoising Diffusion Probabilistic Model (DDPM) proposed by Ho et al. (2020) greatly simplified and stabilized this framework by adopting fixed Gaussian noise schedules and a mean-squared "denoising" objective, hinting at connection between diffusion model training and denoising score matching (Hyvärinen, 2005; Vincent, 2011). Subsequently, Song et al. (2021) generalized the approach in continuous time using Itô stochastic dif- ferential equations (SDEs), showing that diffusion models can be viewed as lcarning the score function t( of noisy data distributions and enabling sampling via reverse-lime SDEs, established by Auderson (1982). Ghimire et al. (2023) investigated the Riemannian structure underlying score-based generative models, offer- ing valuable insights into their manifold geometry. However, their goal was the geometric analysis of existing models, not a reinterpretation of the algorithm through our proposed WGF flow matching. Together, these developments established the standard score-based generative modeling pipeline of modern DMs.
Recent research has extended the diffusion paradigm beyond Gaussian noise and continuous states. Camp- bell et al. (2022) formulated a continuous-ting limit for discrete-state denoising diffusion, while Santos et al.
2Since the OU process decomposes into D independent one-dimensional processes, the density factorizes across coordinates: p(g,t (.)) = II (i(E s,
2

---

Published in Transactions on Machine Learning Research (12/2025)
(2023) developed exact theoretical framework for arbitrary discrete-state Markov processes without vari- ational approximations, deriving the discrete-time analog of Anderson's 's reverse-time SDE. We remark that this study does not extend to these generalized DMs which leverage non-Gaussian noise.
Wasserstein Gradient Flow (WGF) originates from the theory of optimal transport (OT), but it has become increasingly central to the theoretical understanding of modern generative models. Classic monographs such as Ambrosio et al. (2008) and Figalli & Glaudo (2023) provide comprehensive foundations for this frame- work. The connection between WGF and diffusion models lies in the forward evolution of the probability density p(z, t), governed by the Fokker Fokker-Planck equation (FPE (2)). In their seminal work, Jordan et al. (1998) demonstrated that an implicit Euler discretization of the FPE can be reinterpreted as a variational problem: each timestep corresponds to minimizing a free-energy functional composed of a potential energy term (æ)) and a negative Shannon entropy of the evolving distribution, plus a Wasserstein-2 distance term between successive densities (scaled by the time step). The entropy and the Wasserstein-2 distance can both be viewed as regularizers of the optimization problem for the potential energy. Morcover, the sum of the po- iential energy and the negative entropy equivalent to the Kullback Leibler divergence (T(pe]|T)) from the evolving probability density Pt to the Gibbs measure x (Ix) seheme establieet that the FPE can be viewed as gradient flow in the space of probability mcasures. Building on this foundation, Otto (2001) introduced a formal Riemannian calculus on the manifold of prob- ability distributions, showing that the FPE describes the steepest descent of free energy under Wasserstein geometry. This formulation, now widely known as Otto calculus, provides a rigorous mathematical language for describing the evolution of probability densities as particles slide down an energy landscape shaped by optimal transport. Furthermore, Otto introduced the generalized Liouville equation (GLE, Gerlich (1973) and establishes the link between macroscopic density evolution and microscopic transport dynamics. To- gether, the JKO scheme and Otto's formulation provide the foundation for WGF, unifying PDE evolution, entropy maximization, and optimal transport.
exp(-V(zx)) This insight saighttknown the JKO
More recently, Wassorstcin-gradiont concepts have gained renewed attention across machine learning, partic- ularly in sampling, variational inference, and generative modeling. Early work by Wibisono (2018) framed Langevin dynamics as optimization in the space of probability measures, highlighting the close relationship between stochastic sampling and gradient flow in Wasserstein geometry. Subsequent studics extended this principle to practical optimization and inference algorithms in high dimensions, such as the Wasserstein prox- imal gradient (Salim et al., 2020) and variational WGF formulations (Tambert et al., 2022; Fan et al., 2022). These methods treat WGF primarily as computational framework for density evolution or inference, rather than as a reinterprctation of diffusion- based generative learning. Other lines of work have explored large- scale and neural-network-bassel implementations of Wasserstein flows (Mokrov et al., 2021; Alvarez-Melis et al., 2022), and recent efforts have proposed scalable or unbalanced formulations for generative modeling (Choi et al., 2024). Despite these advances, the majority of these studies focus on constructing or optimizing explicit Wasserstein flows rather than reinterpreting existing diffusion models within the WGF framework.
3 Numerical experiments
We first perform numerical experiments to verify the central question we have in score-based generative modeling: Are we learning the score function?
Due to the definition of the score function, s(z,t) := V, log p(z, t), the fundamental theorem of calculus (or gencralized Stokes' theorem in high dimensions) states that the line integral of the score function along a closed path in the state space must be equal to zero:
$ s0,))) - dF d== 0.
(4)
We distinguish GLE from the "contimuity equation", term more broadly used in the physics literature to describe conser- vation of various quantities, such mass, heat, etc. GLE is special case of contimuity equations that describes the evolution of normalized probability density functions.
3

---

Published in Transactions on Machine Learning Research (12/2025)
We refer to Eq. (4) the integral constraint. The second constraint, also following directly from the definition of the score function, states:
s
dr, (z, (), for any pair (i,j) E {1 DJ.
(5)
Jzj
We refer to Eq. (5) as the differential constraint. Our goal is to numerically investigate whether either of the constraints is met in trained diffusion models.
3.1 Models and datasets
To present a minimal working example, we trained an MNIST diffusion model using a lightweight U-Net implementation. The model is composed of SeusSleee residual bottlenecks and depthwise convolutions. The time indices are embedded, passed through an MLP, and added the feature maps in each block. It employs simple encodor-decoder blocks with downsampling, upsampling, and skip connections, keeping the model lightweight (around 4 MB) We used the cosine schedule (Nichol & Dhariwal, 2021) and total discrete time index T = 1000, which corresponds to observing time-homogeneous OU process (1) at discrete times (Santos & Lin, 2023):
f(k) t log f(0)' f(k) cos < + 0.008 π
(6)
0.008 2
We also performed the same test with latent diffusion, using a VAE with an 8 8 latent space". The diffusion process employs the same network as before but operates in the latent space of the VAE.
In addition to the MNIST dataset, we perform experiments several other datasets and neural architectures, including publicly available state-of-the-arl Dilfusion Models:
CIFAR-10, using a latent DDPM.
CelebA-HQ-2550 using the pretrained variance-explodingg model from Song et al. (2021).
Neal's funnel distribution (Neal, 2003), using the same lightweight neural architecture as in the MNIST experiment.
Our motivation for carrying out comprehensive numerical analysis is to demonstrate the validity of our claims popular high-dimensional datasets in simple settings where the data distribution is analytical, using commonly used neural architectures. As we observed very similar behavior, we present the results of the MNIST dataset in Fig. in the main text and the rest in the Appendices 6.2 and 6.3.
3.2 Integral constraints
To numerically evaluate the integral constraint (Eq. (4)), introduce three mechanisms for generating closed paths over which the intogral is computed:
Brownian path. Starting from a corrupted sample x, E RDD generated by the forward diffusion, we perform a random walk on D using a Brownian bridge, which generates a path in RD starting and ending at Zt. The path of a Brownian bridge XBB W, - wWu/U with fictitious time u E [0, U). We choose U 9, uniformly sample 1,000 discrcte time steps in between, and add the resulting path forward sample It, i.e., Yust F, BBB This method does not guarantee that the path stays close to the the typical region induced by the forward process, illustrated in Fig. 1 (a). We include this path way to study the behavior of out-of-distribation samples.
4The neural network implementation he found http:://ggiaab..mm//ottt/////tt 5 Implementation based https:/ /ttpps//github.....

---

Published in Transactions on Machine Learning Research (12/2025)
Corrupted distribution p (ma)
Mean of the corrupted distribution ape
Integration path
Linearly interpolated points
Corrupted samples 24
Projected interpolated points
a) Brownian path
b) Rotation path
c) Projection path
Figure 1: Mechanisms for assessing integral constraints. Illustration the three mechanisms we used construct closed paths for evaluating integral constraints within the high-density regions of the data distribution.
Rotation path. Following the typical application of image corruption process, the corrupted sample Fy = Tge /1 e= where N (), We randomly pair each of the D components of e, (e,,e)) forms two-dimensional vector. Then, we rotate each of the D/2 two-dimensional vectors with respect to the origin, i.e,, {u( = c2sTu), + sin(2ru)e, and (2Tr) = sin(2ru)s; + cos(2ru)=j. Note that we rotate all D/2 pairs with the same "angular velocity". The resulting vector is used to generate closed loop in the æ-space, i.e., Vuzt Ige x + v1 - e-2ic'(u), w: 0 1. With this construct, the probability density of noise realization E'(u) is identical to that of the original noise realization E, ensuring the closed path in the r-space sits in the region where most the probability mass is.
Projection path. We first generate multiple corrupted samples ot from the same initial mp, then find way to connect these points such that the connoctions lie in the typical set of corrupted distribution. To achieve this, we propose simple mechanism: to connect two corrupted samples 2, and z/, first generate points that linearly interpolate between the two samples, and then project the interpolated points back to the corrupted distribution. Since Gaussian diffusion in high-dimensional space induces the structure of thin shell around the clean samples, the projection can be carried out by projecting the samples radially back to the shell in RD,, whose radius is estimated through Monte Carlo sampling (which we also know would be & VD from asymptotic analysis). An illustrative schematic diagram provided in Fig. 1 (c).
Figures 2 (a) and (b) show the results of evaluating the integral constraint using these three methods of generating closed paths on MNIST. Figure 3.(a-b) reports the same statisties for the funnel and CclebA- HQ-256. Summary statistics of these distributions are provided in Fig. & Fig. 12 in the Appendix.
Clearly, the integral condition is not satisfied in the trained neural network. One may argue whether the magnitude matters for the reverse-time dynamics. To answer this, we notice that the score-induced drift 2s(x, is added to linear term ztt) in Eq. (3); this provides us a din-dmensiioall quantity:
2 ( dg
(7)
$ 10 dd
where ÿ is a dummy vector looping over the generated path. Results of this quantity are presented in Figs. and in the Appendix, showing a significant deviation from
3.3 Differential constraints
Because computing the full Jacobian matrix is computationally intensive, we instead randomly sample 64 components of the predicted score s(x, t) and components of the corrupted samples v, compute a sub-Jacobian matrix using torch. func. jvp functionality. The statistics for MNIST were collected from 256 samples for each time step and presented in Fig. (c) and (d), both showing non-zero contributions.
5

---

Published in Transactions on Machine Learning Research (12/2025)
b)
109
Brownian
10
Rotation
10
Projection
10
10
10
109
0
200
400
600
800
1000
0
200
400
600
800
1000
k:
k
c)
d)
10
10"
10
0,,5. 8,
-0,8
10-
10
[02,5
10-3
10
10-3
0
200
400
600
800
1000
0
200
400
600
800
1000
k
k
Pixel space, Right: Latent space) Results of integral and differential constraints, as functions of discrete time index a) shows the absolute value of the integral condition & 59-dū; b) presents the same quantity but for the latent dynamics; c) reports the differential condition 10,, 85 0z si| in normal diffusion; d) shows the corresponding differential condition in latent diffusion.
Figure 2: (MNIST- Left:
For CelebA-HQ-2255 256, given the scale of the input and the presence of custom layers in the implementation from Song et al. (2021), running torch. func. was computationally infeasible. Consequently, we utilized fiite-differrenee methods to estimate the sub-Jacobian components for this setting. Results are presented in Fig. 3 (d).
The observed behavior is strikingly consistent across all experiments and configurations. From analytically tractable distributions such as the funnel distribution increasingly complex real datasets including MNIST, CIFAR-10, and CelebA, and across wide range of neural architectures, lightweight U-Net, the latent- diffusion VAE, the DDPM baseline, and the publicly released variance-exploding (VE) model in (Song al., 2021) we consistently observe the same violations of both integral and differential scoro-function constraints. This consistency persists under both variance-preserviine (VP) and variance-explodingg (VE) diffusion setups. Together, these results rule out the possibility that our finding that trained neural networks diffusion models do not learn truly conscrvative score fields is an artifact particular dataset, model capacity, or diffusion configuration. Instead, it points to robust and universal characteristic of current diffusion-molt training practices.

---

Published in Transactions on Machine Learning Research (12/2025)
a}
b
10
10-3
Brownian
Rotation
10-5 5
10
Projection
10-7
10
10
10-11
10-5
10-13 13
0
200
400
600
800
1000
500
1000
1500
2000
k
c)
dl
10
10
1
0, 8
10
02
10-1
[0,.3
10 2
10-3
10-3
0
200
400
600
800
1000
500
1000
1500
2000
k
Figure 3: (Left: Funnel, Right: CelebA-HQ-256) Results of integral and differential constraints, as functions of discrete time index k: a) shows the absolute value of the integral condition & 89 fg-dē normalized by the path length on data samples drawn from funnel distribution; b) presents the same quantity but for the CelebA- HQ-256 dataset; c) and d) show the differential condition [0z,35 02, s,1 for these datasets respectively. We report the normalized integral here because the variance is not. controlled (variance-exploding) in this setting, normalizing by path length makes the comparison fairer between each methods.
4 Connecting Diffusion Models with Wasserstein Gradient Flow Theory
The numerical evidence clearly suggests that the trained newral nelwork does not learn the score function, which is a conservative field. However, the trained network can definitely perform the generative task. The observation raises an interesting question: what is the trained neural network actually learning in order perform the gencrative task?
We propose bold hypothesis, leveraging the WGF theory, to understand what happens in the diffusion modeling. Our assertion is:
Existing diffusion modeling better understood as modeling a normalizing flow (Chen al., 2019), through performing flow matching (Lipman al., 2022) the WGF velocity (Eq. (13)), rather than learning the reverse stochastic differential equation established by Anderson (1982) and popularized by Song et al. (2021).

---

Published in Transactions on Machine Learning Research (12/2025)
We emphasize that this interpretation does not invalidate the reverse-time SDE framework, which remains malhemalically correct when the true score function is available. Rather, the WGF formulation provides a complementary and unifying perspective that explains why diffusion models remain empirically robust, even when the lcarned vector field deviates from a perfectly conservative score. In this view, standard diffusion training can be regarded as performing approximate flow matching to the WGF velocity field, ensuring correct marginal density transport despite imperfections the neural approximation. This complementary interpretation highlights the consistency between the WGF and SDE descriptions: when the score exact, both yield identical dynamics; when it is approximate, the WGF view naturally accounts for the observed robustness of diffussin--asse generative models.
We now shift our focus to establishing direct theoretical correspondence between the learned vector field in diffusion training and the velocity field of a Wasserstein Gradient Flow. This reinterpretation explains why modern diffusion networks remain empirically successful even when the learned field is not guaranteed to be conservative, and situates diffusion modeling within a principled continuum of variational and geometric formalisms. One powerful result. of WGF theory is:
While the sample paths of the diffusion process that FPE describes are fundamentally stochastic, the marginal distribution of the paths at a specific time, p(.,t, is identical to the marginal distribution of the trajectories driven by deterministic WGF.
In the context of diffusion modcling with Ornstein-Uhlenben forward process, let us consider setting the energy functional the sun of a quadratie potential and the negative Shannon entropy
x" E ((0())0)= (t, dx + f /
(1x (vr, 1) log p (z,1) dr.
(8)
Here, the first term accounts for the drift/advection and the second for the diffusion in the FPE (2). The goal is to identify the steepest descent direction that minimizes the energy the most in the space of probability density functions induced by a deterministic velocity field v(x, t). Applying d/dt to the energy functional:
i de
= /
8E (-(tt)) (x, dz,
E (.()))))
(9)
6p(z,t) al
where the functional variation of E with respect the density function p can be explicitly computed: =
8E (c,t))
1
/
x²
8p (x, + (p+ 8p) log (o + 8p) dx- - / p (z. t) log (z, t) dr
6p(x,1)
(p(x,() 1)
2
1
/
x²
log p(z, t) + 5p (z, t) dx = x² + llggp(z.t).
(10)
kp(x,t) t)
2
2
In the last two equations, we neglected higher-order O(6p(z, f)) terms (using the asymptotic symbol ~)) and applied the normalization condition that the functional perturbation ôp(z, t) dr because J p(x, t) dr I (p + 6p) (x, dr. Next, inserting GLE (see footnote 3):
Oep (z, t) = Vx [u(z, ()),
(11)
and the functional variation Eq. (10) into Eq. (9) leads to
d E(p(,t)) = /
xx2 + log (z, V. [u(x, t)p(s, f)) dr
dt
2
f u u(x,t) [x + Vo (z, t)] (x,t) dr,
(12)
where we used intogration by parts and assumed vanishing boundary terms. The above equation can be interpreted as an inner product of the functions ((,))) and V, p(;() under the measure p(, 1). Clearly,
p)(:1) referred as the marginal distribution because it is only the distribution of X time marginal distribution of the joint distribution specified the stochastic process, p
8

---

Published in Transactions on Machine Learning Research (12/2025)
the velocity field that corresponds to the steepest descent of the energy functional should align with the opposite direction of v log(;:() (up to global multiplicative constant):
UWGF (z, == -- Va log ((,, = = -
(13)
The probability distribution of the resulting flow system with the above velocity field evolves under the GLE:
0i 12, 1) vx [owar (z, O)(a, () = Vx [(x + Va log ll,, (z,
(z,
(14)
which is exactly the FPE (2) describing the OU.
We remark that Song et al. (2021) rediscovered the WGF velocity field (Eq. (13)) through manipulating the reverse-time FPE and noticing Vap(a, t) = p(x,t) t) V, log p(x, t). They did not recognize the fundamental WGF structure of the forward process, as we ralustrated above. They used the term "probability flow", without referencing the JKO scheme, Otto calculus, and WGF. We believe it is beneficial to point out the origin of this theoretical framework, given its doeper connection to OT and the variational nature of the diffusion process. We also remark that in recent pre-print by Ghimire et al. (2023), the equivalence wee diffusion models and WGF was also reported.
Now that the structure of the forward WGF is established, we turn our attention to how we can use Flow Matching (Lipman et al., 2022) to learn the WGF. Contrary to typical flow-based models (Chen el al., 2019), which learn the velocity field by maximizing the end-to-end likelihood, the flow-matching method (Lipman et al., 2022) matches the neural velocity field to a target velocity field at all times in a continuous time domain, connecting the initial and final distributions. The target velocity field is often analytically derived for a prescribed transport from the data distribution to an easy-to-sample distribution (often an isotropic Gaussian distribution in high dimension), and evaluated on sampled training data. Here, we use the WGF induced by the energy functional (Eq. (8)) as the prescribed transport, and match its velocily field (Eq. (13). More precisely, we only match the flow induced by the entropic term in Eq. (8).
There are several advantages to understanding the diffusion model as the flow-matching WGF. First, the "probability flow" is trivially included in the WGF framework. Secondly, we can formally bypass the necessily to invoke the reverse-time Itō process, which can be confusing and counterintuitive. As will be seen below, within the WGF and Otto calculus framework, the deterministic probability flow ODE arises trivially: it is just the time-reversal of the forward WGF velocity Eq. (13). Consequently, the theory of learning and inference is elegantly simple, without the need to explicitly route through Anderson's 's reverse-time SDE. Finally, flow-matching WGF naturally explains why the trained neural flow, which fails to obey the differential and integral score conditions, can still perform well in generative modeling.
Let us now introduce self-consistent narrative of flow-matching problem:
1. Training. Our goal is to learn the WGF Eq. (13) through flow now-matching. We minimize the L2 error between the neural velocity field vi and the entropy-inducet velocity field s (,)t) in Eq. (13):
min E. Unit.. Exmp(.)) [.a((,)) s (.,t)]1
(15)
Here, we remark that the target time for performing the "matching" is crawn from priori selected distribution (here, Unif(O, 7)) between the initial (0) and final (T) times. In practice, we would choose T > 1 such that the distribution P(,T) can be reasonably approximated by an isotropic Gaussian distribution, which is the limiting distribution, formally correct only al T > co. For discrele-time algorithms such as in DDPM, discrete times [t+]k=1 are often selected uniformly, but discrete times ty 's are often non-uniformly distributed in the time domain due to various noise scheduling functions (Santos & Lin, 2023).
(a) Data generation. Samples to perform Monte Carlo approximation of the above -2- will be drawn from the distribution at time t, induced by the energy function (Eq. (8)). Instead of using the WGF in the forward dynamics, which requires knowing s(x, f) tth object attempt to learn in high
9

---

Published in Transactions on Machine Learning Research (12/2025)
dimension-wee use the equivalent OU process (Eq. (1)) to generate samples. Due to the stochastic nature of the OU process, we can formally generate infinitely many samples at any arbitrary time for parametrizing Vg. Also, due to the existence of an analytical expression of the OU process (Santos & Lin, 2023), samples can be generated efficiently.
(b) Parametrizing in practice. In general, we do not have the training labels s(x, == Vz "ol plr, t) for the randomly generated samples a at time t. The difficulty can be solved by denoising score- matching proposed by Vincent (2011) which transforms learning ve using the instantaneous distri- bution p(z+) to another learning problem using the joint (two-time) distribution 20)),
min En-Uni(( -o [| () - (xa|=0))12
(16)
whereas the "labels" for training becomes the conditional score function ( V. log p( which is analytical for this OU process, evaluated at randomly sampled time t. We remark that the choice of L cost function is necessary to ensure that the minimizer of the problem Eq. (16) is the same as that of the problem Eq. (15).
2. Sampling/ Inference. To perform the generative task, terminal samples drawn from the isotropic Gaus- sian are transported from & + to t = 0 by integrating the ordinary differential equation backward in time. That is, dz(r)/dr= = -ewGF(x(T)) = xx()) + NN(x(r),-7) -T), where 7 = -t, so signs flip relative to forward time. x(00) ~ N (0, I) and T: 000 - 0. The corresponding GLE (Gerlich, 1973) is:
07 p () v [(<r + ug- (c)- (z,))
(17)
where 0r stands for the trained neural weights.
Operationally, the above descriptions are identical applying the "score-matching" for training and per- forming "probability flow" for inference (Song et al., 2021). However, due to the deterministic nature of the WGF, would not need to invoke the reverse-time stochastic process (Anderson, 1982) in the theory. The simplicity is the primary benefit of recognizing the existing approach as a Wasserstein gradient flow Now-Matching problem.
By framing the learning as a flow-matching problem, it most natural to weight each time equally, which is the de facto training procedure for both discrete-time (Ho al., 2020) and contimuous-tine (Song et al., 2021) diffusion models. The procedure would seem ad hoc if one aims to parameterize a neural network for learning the reverso-time diffusion process by a more theorotically grounded log-likelihood (more prociscly, the bound of which) maximization as shown by Sohl-Dickstein et al. As Ho el al. (2020) pointed out, the log- likelihood approach involved weights which are not uniform in time; by removing such non-umiform weights, DDPM achieved a better performance by effectively solving a flow-matching problem.
Next, assuming that we learn the WGF perfectly, we can treat the reverse-time WGF as dynamical system:
x z(r) + NN(x(N), -T) = x<7) + V, (
(18)
dr
This system is identical to a Wasscrstein Gradient Flow with the energy functional,
x² [(((,T))= - - faddr--oe (z, (1) log (x, r) da 2 = =-1 --12plan (( - 2 (og, (2, dx + / p (I,T)) (z, dæ
(19)
Reverse-time drift
Reverse-time diffuslon
which equivalent to the reverse-time Itō process (Eq. (3)). This suggests that we would not need to invoke Anderson (1982)'s seminal proof of the existence of the reverse diffusion for generative task. This justifies the second advantage of the WGF framework. We remark, however, that to rigorously establish the equivalence of the forward and reverse path measures, Anderson's 's theory remains necessary. hevesth
10

---

Published in Transactions on Machine Learning Research (12/2025)
because generative diffusion models only require consistency at the level of marginal densities, it is not necessary to invoke path measures in practice. We emphasize that our results concern density transport (marginals). We do not make claims about sample-path equivalence, which requires Anderson's 's reverse-time construction. However, the corresponding reverse-timg time Itō process can not only can be used as a stochastic process for sampling, but also coincidentally the reverse-lime process established by Anderson (1982).
Finally, as suggested by our numerical analysis, the neural network is not learning a gradient of a scalar potential, i.c. t2) # s(z,t) for all t, both globally (because it violates the integral conditions) locally (because it violates the differential conditions.) It is thus puzzling and challenging to analyze how the violations affect the reverse-time diffusion, and consequently, the quality of the generated samples. The flow representation can provide some insight here. Suppose we use the trained, yet imperfect neural velocity field t) = Ve log p(z, t). Denote the error by e(æ,t) == s(z,t) - NN(x,t), t). Then, the GLE governing the distribution driven by the neural velocity field is
Ør ((,
[<r + + (a, -T)) p -7)
Ox
0
[ix + s (x. -T] p (z, -r) + ① dr
[e(zz- -T)p -T)
=
Ox
0
|(r + 8 (z, - T)) p (25 -T]
= Fx
+ V. c(x. -T) + ST (z. -T) e (x, -T] p(z, -T).
(20)
Immediately, we can identify a condition that the error field e(x, t) satisfies
0 Vo e((,t) + S (z,t) ex,
(21)
the induced distribution is identical to the true distribution. In other words, if e t) lives in the null kernel of the operator V. + s²(z,,,, t), the trained neural network can perfectly perform the generative task, even if it is not perfectly capturing the score function. We remark that this vector operator is related to the Stein operator (Liu Wang. 2016) and is the key construct several recent papers on sampling (Chen & Ghat 2020; Fan el al., 2024; Tian et al., 2024). In Fig. 4, we computed the error field on a trained latent diffusion model using forward generated samples, showing that indeed a significant e(x, f) is induced (which is of order 102, significant comparod to the order 100 of deterministic decaying flow, +(t) (t)),, but the error field is statistically confined in the null kernel. This analysis suggests that:
Even when NN(x, t) is not the score function lo t), the trained neural network can still be effective to perform generative modeling.
We remark that this analysis is only possible by recognizing the underlying flow structure.
5 Discussion and Concluding Remarks
We acknowledge that we are not the first to highlight conceptual link between diffusion and flow-based generative models. Song et al. (2021) recognized the "probability flow" ODE formulation and discussed its use for density and likelihood estimation, while Gao et al. (2025) recently analyzed the formal resemblance between diffusion processes and Gaussian flow matching. In parallel, Ghimire et al. (2023) independently examined the geometric foundations of diffusion models through the lens of Wasserstein geometry, focusing on the Riemannian structure of based generative modeling, Their work is complementary to ours but does not explicitly develop the Wasserstein Gradient Flow-Matching interpretation to the Stein operator emphasized here. To our knowledge, no prior study has unified these perspectives by formulating diffusion modeling as normalizing flow parameterized through Wasserstein Gradient Flow and Otto calculus. Establishing this link provides a theoretically grounded view that also suggests new avenues for forward random sampling and regularization in flow-based generative learning.
the connection
We averaged over 256 randomly generated forward samples zı- For each sample, sufficient, condition does not seem to be met but the average seems agree, noting the significant variance for small &.
11

---

Published in Transactions on Machine Learning Research (12/2025)
a.
b
100
25
80
0-
|||[T &x)ll2
60
-25
40
-50
20
-75
200
400
600
800
1000
200
400
600
800
1000
k
Figure 4: a) L2 norm of e( () and b) Stein operator value of e(x,
To conclude, we advocate for this theoretical framework because first, it was developed over 20 years ago, and yet its direct connection to modern diffusion generative models has received relatively little explicit attention in the machine learning literature; and second, the framework is self-consistent, simple, concise, and elegant. We dedicate this work to the pioncers of Wasserstein Gradient Flow theory-Jordan, Kinderlehrer, and Otto whose foundational insights continue to shape and inspire cutting-edge machine learning research today.
References
David Alvarez-Melis, Yair Schiff, and Youssef Mroueh. Optimizing Functionals the Space of Probabilities with Input Convex Neural Networks. Transactions Machine Learning Research, April 2022. ISSN 2835-8856.
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient Flows: In Metric Spaces and in the Space of Probabilily Measures. Birkhäuser, Basel, 2nd ed edition, 2008. ISBN 978-3-7643-8822-88
Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 (0804--1449(82)00055-5 URL https://linkknghub.
(3):313 326, May 1982. ISSN 03044149, doi: elsevier http:///ttiieeeep////01110000000
Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. Generalized Denoising Auto-Encoders Generative Models. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
Andrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Ar- naud Doucet. A continnous time framework for discrete denoising models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 28266-28279. Curran Associates, Inc., 2022. URL https://procceddngs neurips. cc/ paper
pdf.
Peng Chen and Omar Ghattas. Projected Stein Variational Gradient Descent, June 2020.
Ricky T. Q Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural Ordinary Differential Equations, December 2019.
Jaemoo Choi, Jacwoong Choi, and Myungjoo Kang. Scalable Wasserstein Gradient Flow for Generative
Modeling through Unbalanced Optimal Transport. In Proceedings of the 41st International Conference on Machine Learning, pp. 8629 8650. PMTRR July 2024.
12

---

Published in Transactions on Machine Learning Research (12/2025)
Jiaojiao Fan, Qinsheng Zhang, Amirhosscin Taghvaci, and Yongxin Chen. Variational Wasserstein gradient flow. In Proceedings of the 39th International Conference on Machine Learning, pp. 6185-6215, PMT.R, June 2022.
Mingzhou Fan, Ruida Zhou, Chao Tian, and Xiaoning Qian. Path-Guided Particle-based Sampling, In Proceedings of the 4 1st International Conference on Machine Learning, pp. 12916-12934. PMLR, July 2024.
Alessio Figalli and Federico Glaudo. An Invitation to Optimal Transport, Wosserstein Distances, and Gra- dient Flows. EMS Press, Berlin, Germany, second edition edition, 2023. ISBN 978-3-98547-550-6
Ruiqi Gao, Emiel Hoogeboom, Jonathan Heek, Valentin De Bortoli, Kevin Patrick Murphy, and Tim Sal- imans. Diffusion Models and Gaussian Flow Matching: Two Sides of the Same Coin. In The Fourth Blogpost Truck at ICLR 2025, February 2025.
G. Gerlich. Die verallgemeinertte Liouville- 8914. doi: 10.1016/003118914411 00111-88811133390008--
Liouville-Gleichung Gleichung, Physica, 69(2):458 466, November 1973. ISSN 0031-
Sandesh Ghimire, Jinyang Liu, Armand Comas, Davin Hill, Aria Masoomi, Octavia Camps, and Jennifer Dy. Geomelry Score Based Generative Models, February 2023.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models, 2020.
Aapo Hyvärinen. Estimation of Non-Normalized Statistical Models by Score Matching. Journal of Machine Learning Research, 6(24):695-709, 2005. ISSN 1533-7928.
Richard Jordan, David Kinderlehrer, and Felix Otto. The Variational Formulation of the Fokker -lanckk Equation. SIAM Journal on Mathematical Analysis, 29(1):1-17, Jamuary 1998. ISSN 0036-1410, doi:
10.1137/8003611119633333
Mare Lambert, Sinho Chewi, Francis Bach, Silvère Bonnabel, and Philippe Rigollet. Variational inference via Wasserstein gradient flows. Advances Neural Information Processing Systems, 35:14434-14444 December 2022.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matthew Lc. Flow Matching for Generative Modeling. September 2022. URL https://openrevieet http:p///pppprreeee..eaeeeaaaaap htt./orrrrdddddPPWa...te.tt
Qiang Liu and Dilin Wang- Stein variational gradient descent: A general purpose bayesian inference al- gorithm. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc., 2016.
Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, and Evgeny Burnaev. Large-Scale Wasserstein Gradient Flows. In Advances in Neural Information Processing Systems, vol- une 34, pp. 15243 15256. Curran Associates, Inc., 2021.
Radford M Neal. Slice sampling. The annals of statistics, 31(3):705-767 2003.
Alex Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models, 2021.
Felix Otto. The geometry of dissipative evolution equations: The porous medium equation. Communications in Partial Differenlial Equalions, 26(1-2):101 174, 2001. doi: 10.108//PDE-100002245
Adil Salim, Anna Korba, and Giulia Luise. The Wasserstein Proximal Gradient Algorithm. In Advonces in
Neural Information Processing Systems, volume 33, pp. 12356-12366. Curran Associates, Inc., 2020.
Javier E. Santos and Yen Ting Lin. Understanding Denoising Diffusion Probabilistic Models and their Noise Schedules via the Ornstein OrnsteinUUllebbeck Uhlenbeck Process, October 2023.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Javier E. Santos, Zachary R. Fox, Nicholas Lubbers, and Yen Ting Lin. Blackout diffusion: Generative diffusion models in discrete-state spaces. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 9034-9059. PMLR, July 2023.
Won Seong. Simple Latent Diffusion Model. 2024.
Jascha Sohl-Dickstein, Erie A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibriun Thermodynamics.
Yang Song, Jascha Diik-Dickst,in, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Basod Generative Modeling through Stochastic Differential Equations, 2021. Comment: ICLR 2021 (Oral).
Yifeng Tian, Nishant Panda, and Yen Ting Lin. Liouville Flow Importance Sampler. In Forty-Firsl Inter- national Conference on Machine Learning, June 2024.
Pascal Vincent. A Connection Between Score Matching and Denoising Autoencoders. Newral Compulation, 2367)1166111674, July 2011. ISSN 0899-7667. doi: 10.1162/NECO_1 a_0042.
Andre Wibisono. Sampling as optimization in the space measures: The Langevin dynamics as composite optimization problem. In Proceedings of the 31st Conference On Learning Theory, pp. 2093 3027. PMLR, July 2018.
14

---

Published in Transactions on Machine Learning Research (12/2025)
106
A.1. Brownian path
B.1. Latent Brownian path
2000
d.
1000
50
1000
200
400
600
800
1000
200
400
600
800
1000
A.2. Rotation path
B.2. Latent Rotation path
20-
0++
1000
-20
200
400
600
800
1000
200
400
600
800
1000
k
A.3. Projection path
B.3. Latent Projection path
1000
10
0
10
1000
200
400
600
800
1000
200
400
600
800
1000
Figure 5: (MNIST) Summary statistics of & Ro dæ calculated by different path-generating mechanisms, in normal and latent diffusions.
6
Appendix
We provide more statistics of the non-dimensionallzed quantity 1 Sadæl/$ |ate |dd (Eq. (7)), as well as experiment results on the CIFAR-10 dataset.
6.1
Additional numerical results on MNIST
Refer to Figs. 5, 6, 7.
15

---

Published in Transactions on Machine Learning Research (12/2025)
a
b
10a
Brownian
10
Rotation
10a
Projection
www.mm
10
10
10-6
10-3
10
0
200
400
600
800
1000
200
400
k 800
600
1000
k
Figure 6: (MNIST) Results of integral constraints, as functions of discrete time index k: a) shows the absolute value of the integral condition & 5n dē normalized by the path length and the strength of the deterministic flow, f 1 [dž]; b) presents the same quantity but for the latent dynamics.
6.2 Numerical results on CIFAR-10
For CTFAR-10, we utilized the models from Seong (2024), implements the standard DDPM and VAF with latent dimension of 16 16. We also tried training these models from scratch, which exhibits similar behaviors the pretrained ones. Results are presented in Figs. 8, 9, 10, 11.
6.3
Additional numerical results on Funnel & CelebA-HQ-256
Summary statistics shown in Fig. 12.
16

---

Published in Transactions on Machine Learning Research (12/2025)
A.1. Brownian path
B.1. Latent Brownian path
40
0.5
20
0
0.0
-0
0.5
40
200
400
600
800
1000
200
400
600
800
1000
A.2. Rotation path
B.2. Latent Rotation path
7500
0.25
5000
0.00
2500
0-
-0.25
-2500
-0.50
200
400
600
800
1000
200
400
600
800
1000
k
A.3. Projection path
B.3. Latent Projection path
0.2
0.1
0.0
0.1
200
400
600
800
1000
0
200
400
600
800
1000
k
Figure 7: (MNIST) Summary statistics of 8 50 dæl/$ [ calculated by different path-generating mech- anisms, in normal and latent diffusions.
17

---

Published in Transactions on Machine Learning Research (12/2025)
b)
10
10a
Brownian
Rotation
10
Projection
10
100
10a
10
100
10"
0
200
400
600
800
1000
0
200
400
600
800
1000
k:
k
C)
109
10-1
10-1
0,,5
[0,,9 0,ys1
10
10
10
10-3
10
0
200
400
600
800
1000
0
200
400
600
800
1000
k
k
Figure 8: (CIFAR-10) Results of integral and differential constraints, as functions of discrete time index k: a) shows the absolute value of the integral condition & 5o dT; b) presents the same quantity but for the latent dynamics; c) reports the differential condition 10,,s O2, sil in normal diffusion; d) shows the corresponding diffcrential condition in latent diffusion.
18

---

Published in Transactions on Machine Learning Research (12/2025)
10
A.1. Brownian path
10 B.1. Latent Brownian path
0.5
0.5
de
di
0.0
0.0
$
-0.5
-0.5
-1.0
-1.0
200
400
600
800
1000
0
200
400
600
800
1000
k
k
A.2. Rotation path
B.2. Latent Rotation path
2500
20000
2500
400000
55000
0
200
400
600
800
1000
200
400
600
800
1000
k
A.3. Projection path
B.3. Latent Projection path
50000
2000
da
0
500000
-2000
200
400
600
800
1000
200
400
600
800
1000
k
Figure 9: (CIFAR-10) 10) Summary statistics of & 59 di calculated by different path-generating mechanisms, in normal and latent diffusions.
19

---

Published in Transactions on Machine Learning Research (12/2025)
a
b)
10
Brownian
10a
10-1
Rotation
10a
Projection
10
10 1
10
10
10-7
10-9
10
0
200
400
k 800
600
1000
200
400
k 800
600
1000
Figure 10: (CIFAR-10) Results of integral constraints, as functions of discrete time index k: a) shows the absolute value of the integral condition f S dē normalized by the path length and the strength of the deterministic flow, f | |dž]; b) presents the same quantity but for the latent dynamics.
20

---

Published in Transactions on Machine Learning Research (12/2025)
A.1. Brownian path
B.1. Latent Brownian path
20
50
10
0
0
-10
20
200
400
600
800
1000
200
400
600
800
1000
k
k
A.2. Rotation path
B.2. Latent Rotation path
50000
Spila
50000
50000
200
400
600
SÓ0
1000
200
400
600
S00
1000
k
k
A.3. Projection path
B.3. Latent Projection path
10
100
0-
-10-
-100
200
400
600
800
1000
200
400
600
800
1000
k
k
Figure 11: (CIFAR-10) Summary statistics of $ 8o džl [ mechanisms, in normal and latent diffusions.
calculated by different path-generating
21

---

Published in Transactions on Machine Learning Research (12/2025)
A.1. Brownian path
B.1. Brownian path
0.5
0.0005
0.0
0.0000
-0.5
-0.0005
200
600
800
1000
1101
1501
2000
k
A.2. Rotation path
B.2. Rotation path
50
0.01
Spril (a
0.00
50
0.01
-100
200
400
600
800
1000
31
501
1101
1501
2000
k
A.3. Projection path
B.3. Projection path
0.5
0.004
8
0.0
0.002
[1
0.000
00.5
0.002
1.0
0.001
200
400
600
800
1000
31
501
1101
1501
2000
k
Figure 12: (Left: Funnel, Right: CelebA- Summary statistics of 1 dFl/f |d calculated by different path-generating mechanisms, for data samples drawn from funnel distribution (left) and CelebA- HQ-256 (right). Note that for CQ265A-HQ-256, we presented more data points at carlier time k.
22


Reviews:
[{"review_id": "3vfNgRsxQ7", "reviewer": "Reviewer_sdU7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper revisits the theoretical interpretation of diffusion generative models.\nTraditionally, diffusion models are trained to approximate the score function (the gradient of the log-density of the noisy data) and use it to define a reverse-time SDE for sampling.\nThe authors argue that, in practice, trained networks do not produce conservative vector fields (a requirement for being true scores) and thus may not actually learn the score function.\nThey provide numerical evidence by checking integral and differential conditions that a true score must satisfy and find them violated for trained diffusion models (on MNIST and a latent-diffusion variant).\nTo reconcile this, the paper proposes viewing diffusion models as performing flow matching to the velocity field of a Wasserstein Gradient Flow (WGF) rather than strict score learning.\nThey further connect the resulting error field to the Stein operator, showing why models can still transport densities correctly even when the learned field is not a true score.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper presents a well-motivated reinterpretation of diffusion models as performing Wasserstein Gradient Flow (WGF) flow matching rather than strictly learning the score function of the noisy data. The derivations are clear and appear mathematically sound, and the empirical checks on integral and differential constraints (Eqs. 11–12) provide an interesting first indication that trained diffusion networks may not produce conservative vector fields.\n\nThat said, the empirical support for the strongest claim — that diffusion models do not in practice learn the true score function — feels limited. The numerical analysis is carried out only on a single lightweight U-Net trained on MNIST and a latent-diffusion variant. While these experiments show that the integral and differential constraints are violated for those models, this does not convincingly demonstrate that more expressive architectures, larger datasets, or more thorough optimization could not approximate a true score. In the limit of highly flexible models and sufficient training, one might still expect the score function to be reachable.\n\nIn addition, the paper introduces an interesting characterization of the approximation error  $e(x,t)$ through Eq. (19–20) and its relation to the Stein operator. However, this insight is not validated in a controlled synthetic setting where the true score and its error are known — for example, in low-dimensional Gaussian mixtures or other analytically tractable distributions. Such experiments could test whether diffusion models fundamentally fail to learn the score even when the target density is simple and fully known.\n\nOverall, the evidence is thought-provoking and suggests that current training practices do not enforce conservativeness, but it is not yet strong enough to make the broader claim that diffusion models cannot or generally do not learn the score function. Strengthening the empirical validation — for instance, through synthetic experiments and larger models — would make the central claims more convincing.", "audience_yn": "Yes", "audience_explanation": "Yes. The reinterpretation of diffusion models as performing Wasserstein Gradient Flow (WGF) flow matching rather than strict score learning is timely and relevant to the machine learning theory and generative modeling communities. Many researchers are currently interested in the foundations of diffusion models and in connecting them to optimal transport and normalizing flows. Even if the empirical validation is limited, the conceptual reframing could influence how future algorithms are designed and analyzed. Readers who work on the mathematical underpinnings of generative models, as well as those exploring new training objectives beyond score matching, would likely find the ideas and the error analysis around Eq. (19–20) valuable.", "requested_changes": "**Critical changes (needed to support acceptance)**:\n\n- Strengthen empirical validation of the main claim.\n\nTest whether the violation of the integral and differential constraints (Eqs. 11–12) persists with larger and more expressive models, different datasets, or more thorough optimization.\n\nInclude at least one controlled synthetic experiment where the true score is analytically known (e.g., Gaussian mixtures or other low-dimensional tractable densities). This would allow the authors to verify whether the observed violations are fundamental or specific to the tested lightweight models.\n\n- Clarify the scope of the claim.\n\nSpecify whether the work argues that current training practice does not guarantee learning the true score, or whether it suggests that learning the true score is fundamentally unnecessary or unlikely even for very flexible models.\n\nMake it clear whether the WGF view is meant to fully replace the SDE/score-based view or to serve as an alternative but compatible perspective.\n\n**Important but not critical (would strengthen the paper):**\n\n\n- Offer a short conceptual explanation or visual intuition for Wasserstein Gradient Flow and Otto calculus for readers without deep background in optimal transport.\n\n- Clearly separate theoretical reformulation from empirical observations to help non-experts follow the argument.\n\n- Explore practical implications. Discuss how imperfect score learning might affect generative quality, density estimation, or mode coverage (similar to issues known in GANs).", "broader_impact_concerns": "Nothing to remark", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "a9k8qC47FJ", "reviewer": "Reviewer_w6So", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work analyses diffusion models. It first provides numerical evidences that the usual practice of approximating the score function with a neural network, does not actually learn the score function as the neural networks are not constrained to produce conservative fields. More preciselly, there are two constraints satisfied by scores: the integral along closed path should be null, and a differential constraint. And the authors show that they are usually not respected. Then, they raise the question how it is still able to perform generative modeling, and propose to see it through the lens of flow matching, where the velocity field would be chosen as the Wasserstein gradient of the KL with a standard Gaussian as target, as its Wasserstein gradient flow corresponds to the forward SDE solved in diffusion models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in the submission are mostly accurate. The numerical evidence that neural networks used to approximate the score actually don't learn the score seem accurate. And the proposition to see this problem through the lens of flow matching is interesting.\n\nHowever, there are few things that could be improved, few statements which does not seem totally right, and the connection to some relevant relative works is missing. Moreover, the proposed algorithm using Flow Matching is not shown to work, and not very clear.", "audience_yn": "Yes", "audience_explanation": "Understanding better diffusion models is an important field, and thus this paper could interest a large audience of TMLR.", "requested_changes": "First, I believe that the paper could be better written and organized. For instance, in the Background, there is a Section 1.1, but not Section 1.2; there is no conclusion; the relation with flow matching, which is an important part of the paper is in a Section \"Discussion\". I would advice to improve the presentation of the paper.\n\nIn Section 1.1, it is said that \"each timestep corresponds to minimizing a free energy functional that combines Shannon entropy with a Wasserstein-2 distance penalty\". But for the equivalence between the Fokker-Planck equation, and the Wasserstein gradient flow, the minimized functional is the relative entropy (Kullback-Leibler divergence), not just the entropy that would give the Heat PDE.\n\nIn Section 3, it is not clear if the Wasserstein Gradient Flow-Matching is actually used in practice. I would expect it to be compared with the usual score-based diffusion method, to verify that it works as well or not. \n\nMoreover, I did not understand some things about the algorithm. It is stated page 6 \"More precisely, we only match the flow induced by the entropic term in equation 4.\". Does it change anything to learn only the entropic part of the flow, and not the full velocity field? In the Data Generation step, it is said that $s(x,t)$ is computed analytically. Could you precise how it is computed? Is there a closed-form? (I know there is one at least if $\\rho_0$ is Gaussian but is it the case more generally?)\n\nWhen I see this algorithms, I can't help of thinking that it will be much slower than usual flow matchings since you need first to do the forward step to get all the time steps. How does it compare?\n\nThe fact that the backward process is equivalent to a Wasserstein Gradient Flow of the functional given in equation (18) has already been observed in [1] (Theorem 2). Moreover in this paper, they also observed the relations between Wasserstein gradient flows and diffusion. A discussion of this paper, and what is different should be added.\n\nThe claim page 8 that this theoretical framework \"has been largely ignored in the machine learning literature\" is wrong, as there is a big literature on Wasserstein gradient flows to solve many tasks ranging from sampling to generative modeling, see e.g. not exhaustively [2,3,4,5] for sampling and [6,7,8,9,10] for generative modeling.\n\n\nTypos:\n- Equation (13): lacks \".\"\n- Equation (18): \"$\\rho(x,s)$\" in first line\n\n\n[1] Ghimire, S., Liu, J., Comas, A., Hill, D., Masoomi, A., Camps, O., & Dy, J. (2023). Geometry of score based generative models. arXiv preprint arXiv:2302.04411.\n\n[2] Wibisono, A. (2018, July). Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem. In Conference on learning theory (pp. 2093-3027). PMLR.\n\n[3] Salim, A., Korba, A., & Luise, G. (2020). The Wasserstein proximal gradient algorithm. Advances in Neural Information Processing Systems, 33, 12356-12366.\n\n[4] Lambert, M., Chewi, S., Bach, F., Bonnabel, S., & Rigollet, P. (2022). Variational inference via Wasserstein gradient flows. Advances in Neural Information Processing Systems, 35, 14434-14447.\n\n[5] Huix, T., Korba, A., Durmus, A., & Moulines, E. (2024). Theoretical guarantees for variational inference with fixed-variance mixture of Gaussians. arXiv preprint arXiv:2406.04012.\n\n[6] Alvarez-Melis, D., Schiff, Y., & Mroueh, Y. (2021). Optimizing functionals on the space of probabilities with input convex neural networks. arXiv preprint arXiv:2106.00774.\n\n[7] Mokrov, P., Korotin, A., Li, L., Genevay, A., Solomon, J. M., & Burnaev, E. (2021). Large-scale wasserstein gradient flows. Advances in Neural Information Processing Systems, 34, 15243-15256.\n\n[8] Fan, J., Zhang, Q., Taghvaei, A., & Chen, Y. (2021). Variational Wasserstein gradient flow. arXiv preprint arXiv:2112.02424.\n\n[9] Choi, J., Choi, J., & Kang, M. (2024). Scalable Wasserstein gradient flow for generative modeling through unbalanced optimal transport. arXiv preprint arXiv:2402.05443.\n\n[10] Terpin, A., Lanzetti, N., Gadea, M., & Dörfler, F. (2024). Learning diffusion at lightspeed. Advances in Neural Information Processing Systems, 37, 6797-6832.", "broader_impact_concerns": "None", "additional_comments": "I think there are some interesting things in this work such as the remarks on the score which is not well learned when doing diffusion models, and the proposition to use instead a Flow-Matching loss to learn the velocity field instead.\n\nHowever, the paper could be improved a lot on the organization and writing, the connections with related works, and the analysis of the proposed algorithm. In particular, I think it should be compared with score-based models, flow matchings, and on image data.", "extra_scores": {}, "extra_text": {}}, {"review_id": "fmzKZXIpe9", "reviewer": "Reviewer_8CrX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper offers a reinterpretation of diffusion-based generative models and challenges the conventional perspective that views the neural network as a score function approximator. The authors numerically demonstrated that trained diffusion models do not preserve a conservative vector field, which violates the constraints required of the true score functions. Instead, the authors propose to understand the learned model as performing flow matching to a Wasserstein Gradient Flow. Nonetheless, the authors argue that the generative performance of the diffusion model remains robust because approximate flow matching is sufficient for density transport in practice, even if the learned vector field is not a perfect gradient field. \n\nThe key contributions of the paper include: (1) highlighting a mismatch between theory (strict score matching) and practice (non-conservative fields); (2) introducing Wasserstein Gradient Flow as an alternative conceptual perspective on diffusion. A potential limitation is that this paper may overstate the “incorrectness” of standard reverse-time SDE interpretations based on partial or local violations of conservativeness. One could argue that the SDE view remains theoretically valid when given a perfect score matching, while in practice, the score function is approximated by neural networks, and the SDE process is robust to small estimation errors [1]. Nonetheless, the discussion in this paper could deepen connections between diffusion models, optimal transport, and flow-based generative modeling.\n\n[1] Sitan Chen, et al. \"Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions.\" The Eleventh International Conference on Learning Representations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The theoretical statements are sound. The authors present convincing numerical experiments on various path-based integral evaluations and Jacobian-symmetry checks on MNIST and CIFAR-10. The numerical results support their theoretical claims and demonstrate clear violations of the integral and differential constraints required of gradient fields.", "audience_yn": "Yes", "audience_explanation": "This paper studies the connections between score approximation in diffusion models and Wasserstein gradient flow matching, which is a central topic in the theory of diffusion-based generative models.", "requested_changes": "(1) Discussion on the validity of the SDE frameworks: \nAs mentioned above, the SDE framework is still valid with a perfect score function, and it is robust to small estimation errors. I suggest that the authors emphasize their theoretical finding as a complementary interpretation of the robustness of diffusion models to imperfect score approximations, and further discuss the effectiveness of the trained neural networks from the perspective of Wasserstein gradient flow matching. \n\n(2) Further evaluation of the integral and differential constraints:\nThe violation of the integral and differential constraints originates from an imperfect approximation to the true score. Ideally, a neural network with enough capacity is able to fully optimize the objective in equation (15), given sufficient training samples and training steps. Therefore, it is interesting to numerically compare the behaviors of neural networks with different capacities and different training steps and discuss the practical implications of the theoretical findings.", "broader_impact_concerns": "This paper has no ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)