Closed-Form Diffusion Models
Authors: Christopher Scarvelis
Haitz Sáez de Ocáriz Borde
Justin Solomon
Venue: Accepted by TMLR ()

Abstract: Score-based generative models (SGMs) sample from a target distribution by iteratively transforming noise using the score function of the perturbed target. For any finite training set, this score function can be evaluated in closed form, but the resulting SGM memorizes its training data and does not generate novel samples. In practice, one approximates the score by training a neural network via score-matching. The error in this approximation promotes generalization, but neural SGMs are costly to train and sample, and the effective regularization this error provides is not well-understood theoretically. In this work, we instead explicitly smooth the closed-form score to obtain an SGM that generates novel samples without training. We analyze our model and propose an efficient nearest-neighbor-based estimator of its score function. Using this estimator, our method achieves competitive sampling times while running on consumer-grade CPUs.

Published in Transactions on Machine Learning Research (05/2025)
Closed-Form Diffusion Models
Christopher Scarvelis MIT CSAIL
scaro edu
Haitz Sáez de Ocáriz Borde University of Oxford
dhri6704@orr uk
Justin Solomon MIT CSAIL
jsolomon edu
Reviewed on OpenReview: ht tps: //op en re ew .n /f or um JrMs fr 17
Abstract
Score-based generative models (SGMs) sample from target distribution by iteratively transforming noise using the score function of the perturbed target. For any finite training set, this score function can be evaluated in closed form, but the resulting SGM memorizes its training data and does not generate novel samples. practice, one approximates the score by training a neural network via score-matching. The error in this approximation promotes generalization, but neural SGMs are costly to train and sample, and the effective regularization this error provides is not well-understood theoretically. In this work, we instead explicitly smooth the closcd-form score to obtain SGM that generates novel samples without training. We analyse our model and propose efficient wearret-neighhoo--besee estimator of its score function. Using this estimator, our method achieves competitive sampling times while running on consumor-grade CPUs.
1 Introduction
Score- based generative models (SGMs) draw samples from a target distribution pi by sampling Gaussian noise and flowing through a possibly noisy velocity field e,. This velocity field depends on the score function of the perturbed target distribution Pt. which existing SGMs parameterize as a neural network and learn via score-matching (Hyvärinen & Dayan, 2005) denoising (Vincent, 2011; Ho et al., 2020). Although the target distribution Pi (for example, the distribution over human face images) is typically assumed to be continuous, in practice score-mateine matecina and denoising problems are solved using an empirical approximation P1 to the target distribution constructed from a finite training set.
When タ is the empirical distribution over a finite training set { the perturbed target distribution Pt is a mixture of Gaussians, whose score function Vlogp,(2) has simple closed-form expression. This score function is vector pointing from toward distance-weighted average of all N rescaled training points and is the eract solution to the score-matching problem (Miyasawa, 1961; Raphan & Simoncelli, 2011; Karras et al., 2022). By evaluating this closed-form score during sampling, one obtains training-free sampler for Å1. While this approach seems tempting at first glance, two flaws render it unsuitable for real-world applications:
Many applications involve large training sets, prohibiting O(N) computation of the closod-form score.
2. Flowing base samples through the closed-form velocity field simply outputs training samples Z;. which is not useful in practice.
Existing work avoids these issues by neurally approximating the score of Pe. By compressing training data into the score model's weights, neural score functions replace over N training points with neural network

---

Published in Transactions on Machine Learning Research (05/2025)
evaluation whose complexity does not depend directly on N. Moreover, neural SGMs generate novel samples given finite training data thanks to approximation error (from limited model capacity) and optimization error (from undertraining) in learning the score (Pidstrigach, 2022; Yoon et al., 2023; Yi et al., 2023). While neural SGMs are successful, they are costly to train, and sampling them requires many (typically GPU-bound) neural network evaluations. Furthermore, the form of the error that enables neural SGMs to generalize is unknown, making it difficult to characterize the distribution from which these models sample in practice.
Our key insight is that the flaws of naive closed-form SGMs (in particular, lack of generalization and poor scalability) can be addressed without resorting to costly black-box neural approximations. To this end, we make use of a well-known score formula and introduce smoothed closed-form diffusion models (smoothed CFDMs), a class of training-free diffusion models that require only access to the training set al sampling time. Smoothed CFDMs generate novel samples from a finite training set by flowing Gaussian noise through a velocity field built from a smoothed closod-form score. Our method is efficient, has few hyperparameters, and generates plausible samples in high-dimensional tasks such as image generation. By developing this algorithm, we remenstrate that a closed-form score formula can be adapted to build non-neural sampler that scales to non-trivial generative tasks.
Our specific contributions are as follows:
1. In Section 4, we show that smoothing the exact solution to the score- matching problem promotes generalization by encouraging the score function to point towards barycenters of training samples.
2. Using our smoothed score, in Section 5.1 we construct closed- form sampler that generates novel samples without requiring any training, and characterize the support of its samples.
3. In Section 5.4, we accelerate our sampler using a nearest-neighhor- bbased estimator of our smoothed score, and show in Section 6.2 that in practice, one can aggressively approximate our smoothed score at little cost. to sample accuracy.
4. In Section 6, we scale our method to high-dimenssonall tasks such as image generation. By operating in the latent space of a pretrained autoencoder, we generate novel samples from popular image datasets at speeds competitive with existing GPU-bound methods while running on a consumer-grude loptop with no dedicated GPU.
2 Related work
Diffusion models (Sohl-Dickstein et al., 2015; Song & Ermon, 2019; Ho et al., 2020) have recently achieved state of the art performance in image (Rombach et al., 2022; Zhang & Agrawala, 2023) and vidco generation (Ho et al., 2022a;b). They have also shown promise in 3D synthesis (Luo & Hu, 2021; Poole el al., 2022; Walson et al., 2022; Lukoianov et al., 2024) and in crucial steps of the drug discovery pipeline such as molecular docking (Corso et al., 2023) and generation (Hoogeboom et al., 2022; Schncuing et al., 2022). Despite this progress, however, diffusion models remain costly to train and sample from (Shih et al., 2023). Prior work has sought to accelerate the sampling of diffusson models via model distillation (Salimans & Ho, 2022), operating in a pre-trained autoencoder's latent space (Vahdat et al., 2021; Rombach et al., 2022), modifying the generative process (Song et al., 2020), using alternative time discretizations for sampling (Zhang & Chen, 2023; Liu et al., 2022; Wu et al., 2023), or by parallelizing sampling steps (Shih et al., 2023). Latent diffuusio models also benefit from lower training expenses (Rombach al., 2022), but publicly-roported training costs for stale-of-the-art diffiusion models remain high (Bastian, 2022).
Recent works propose alternative diffusion-like models that discard the Markov chain and SDE formalisms from carlier work. Liu et al. (2023) introduce a unified framework for flow-based generative modeling that subsunes diffusion models and show that straightening their model's flows enables few-step sample generation. Heitz et al. (2023) use a similar objective to construct a straightforward Delbracio & Milanfar (2023) concurrently generalize this framework to arbitrary data perturbations and apply it to image restoration and generation tasks. All of these methods parametrize their flows by neural networks that require extensive training.
sampler, and
2

---

Published in Transactions on Machine Learning Research (05/2025)
While diffusion models draw inspiration from mathematical theory (Feller, 1949; Stroock & Varadhan, 1969a;b; 1972), there have been limited attempts to develop a theoretical understanding of their behavior. Salmona et al. (2022); Koehler et al. (2023) study the statistical limitations of diffussion models trained via score-matching, De Bortoli et al. (2021); Lee et al. (2023) present convergence results for diffusion models with absolutely continuous targets, and De Bortoli (2022) extends these results to manifoldssupporeed distributions. However, as diffusion models are trained on an empirical approximation to their target distributions, these results can only show that a diffusion model converges to the empirical distribution of its training set, whereas one is typically interested in generating novel samples.
Pidstrigach (2022) takes an initial step in this direction by studying the support of an SGM's model distribution and providing conditions under which an SGM memorizes its training data or learns to sample from the true data manifold. Oko et al. (2023) further show that diffusion models can attain nearly minimax estimation rates for the true data distribution provided its density lies in an appropriate function class. Yoon et al. (2023) propose and empirically test memorization- gegoraaiinnnggnnaaiiaiiooooonon dichotomy, which states that diffusion models may only generalize when they are parametrized by neural networks with insufficient capacity relative to the size of their training set. Yi et al. (2023) note that standard training objectives for diffusion models have closed-form optima given finite training sets and show via experiments that the approximation error of neural score functions enables existing diffusion models to generalize. Recently, Kadkhodaie et al. (2024) study generalization in diffusion models using techniques from applied harmonic analysis and demonstrate that SGMs trained on sufficiently large datasets learn a distribution that is effectively independent of the training set, and Aithal et al. (2024) show that neural SGMs "hallucinate" by generating data that lies outside the support of the target distribution because they learn smooth approximations to the ground truth score function. Whereas these works study the generalization of existing SGMs, we construct novel SGM that explicitly perturbs the closed-form score to generalize without the indeterminate approximation error and training costs of a neural score.
Recent works in graphics and vision have also noted that neural networks are unnccessary for tasks such as novel view synthesis, where neural radiance fields (NeRFs) had previously achieved SOTA results (Barron et al., 2022). In light of this, Kerbl et al. (2023) use efficiently optimized 3D Gaussian scene representations to achieve SOTA visual quality in novel view synthesis while operating in real time. In this work, we adopt a similar perspective and investigate the extent to which neural networks can be replaced with efficient and well-understood classical approaches in generative modeling.
3 Preliminaries: The closed-fomm score
Flow-based generative models draw samples from target distribution Pr by sampling from known base distribution Po (typically (00,, I)) and flowing these samples through velocity field from t= 0 to t = 1. For an appropriately-chose Vt, the samples will be distributed according to the target distribution at = 1. SGMs employ a it that depends on the score function V Pr of the perturbed data distribution pt- For example, when Pa = (O,,,) I), one velocity field satisfying this property is of (z) = (z + (1 - t) (i (Liu et al., 2023), where PE the marginal distribution of the random variable 2 = tx + (1 t)c, whose samples are larget samples
p1 that have been rescaled by L and perturbed by Gaussian noise l)e (10 1)) The score function V log (2) is typically learned via score-matching or denoising.
In practice, one learns an SGM from finite training set { In this case, the target distribution ọ1 is the empirical distribution over { and for the field og defined above, the perturbed target distribution pt is a mixture of Gaussians with mcans tx; and common covariance matrix (1 (I- (We will subsequently use the fact that p is a mixture of Gaussians to accelerate our sampler in Sections 5.2 and 6.2.) Its score v log pf (z) has a closed-form expression:
v log () =
(ke(2) - 2),
(1)
(1 8)2
(-113 [_1==-1h) 13--10 -x1112
where kx(=)
softmax
2(1 1)2 ), ta.
(2)
3

---

Published in Transactions on Machine Learning Research (05/2025)
in which we let || - tX denote the vector whose i-th entry is - tx,l1². ||=--x; This V Vlog (2) is vector pointing from toward distance-weighted average (z) of all N rescaled training points and is the eract solution to the score-matching problem given finite training data. Equation 1 is well-known, having appeared in the empirical Bayes literature as carly as in Miyasawa (1961) and more recently works such as Raphan & Simoncelli (2011) and Karras et al. (2022, Appendix B.3). has also inspired machine learning methods such as denoising score-matching (Vincent, 2011) and score interpolation (Dieleman al., 2022).
We define closed-form diffusion model (CFDM) to be the SGM that flows NV I) base samples through this v7(z) while evaluating the score V log (z) in closed form needed during sampling. As this model can only generate samples from the empirical distribution over training data, CFDMs are not useful in practice.
4 Smoothed closed-form diffusion models
Pidstrigach (2022); Yi et al. (2023) find that existing diffusion models generalize due to approximation error incurred during score-matching. Rather than studying the generalization of neural SGMs, we take inspiration from this observation and construct a training- free SGM that generalizes by explicitly inducing error in the closed-form scorc.
4.1 Definition
Deep neural networks fit the low-frequency components of their target functions first during training, phenomenon known as "spectral bias" that results in excessively smooth approximations to the target function (Rahaman et al., 2019). Hence, to model the bias of a neural SGM, we induce error in the score function by smoothing it. To smooth a function f, one chooses a zero-mean noise distribution Ps and replaces [(=) with the convolution f(x) = E [[(=++) In practice, we compute the smoothed score function (2,((2)
emp,
by fixing a smoothing parameter a 2 0, drawing M noise samples 6m Pe, and computing
M
kz + TEm) 02m)-2) 2
(x) = (1 1)2
1
1 M
Σ m=1
(3)
That is, we average the weights & in Equation over M small perturbations dcn of 2; as - O, we approach the unsmoothed score in Equation 1. We do not add noise to the -2 term in the score because it vanishes in expectation. The smoothing procedure in Equation 3 is the key ingredient enabling without a learned approximation to the score function. The smoothed score Sa, can then be inserted into an SGM sampling loop to yield closed-form sampler that generates novel samples.
model to generalize
To develop intuition for why this simple modification of the closed-form score promotes generalization, we consider the behavior of the closed-form score as ( + Figure 1 depicts the closed-form score (Equation 1) and its smoothed counterpart (Equation 3) at t 0.95 for simple case where the training data consists of two points zo (in blue) and xı (in red). In this regime, the temperature (1 - f)2 of the softmax in Equation 2
(a) Closed-fome score
(b) Sedothed score
Figure 1: Efffect of smoothing on the closed-form score (yellow streamplot).. Colors represent distance weights in (z);); blue regions of space are drawn to the blue point the left, and vere-versa.
4

---

Published in Transactions on Machine Learning Research (05/2025)
is low, and (2) is effectively equal to the nearest neighbor of within the training set. Flowing points through velocity field such as Liu et al. (2023)'s i (z(=) (= + (1 - 1)V log (T)) causes them to flow towards their nearest training sample. As result, an SGM based on this score function simply outputs training data.
On the other hand, the small perturbations dem in Equation occasionally push points near the Voronoi boundary between z0 and zj into their neighboring Voronoi cell. Averaging k over these perturbations yields score function that instead points towards the line segment connecting Ho and 21. An SGM based on the smoothed score function will hence cause samples to flow towards weighted barycenters of the training points, which promotes generalization, especially when the data lic on a manifold of sufficiently low curvature. We will make these intuitions rigorous in the following section by proving Proposition 4.1, which will enable us to constrain the support of our model's samples.
4.2 Effect of smoothing the score
In this section, we show that as t + 1, the smoothed score points towards barycenters of these tuples rather than towards training points, thereby enabling our sampler to generalize. We first note that via a straightforward computation,
N
|| [ 1 + otug,m
ke + dem)
Σ
softmax
lat
2(1 t)2
where Ui = 22mm, vx) is scalar random variable. This shows that smoothing the score acts by perturbing the distance weights -||2 tzill. so one can directly add scalar noise ti.m Pu to these weights instoad of perturbing the inputs a with noise cm Pe. To simplify our exposition, we will frame the remainder of our results from this perspective.
We now show that smoothing the closed-form score yields a function (=) that points from = towards a convex combination (z,((z) of barycenters to₃ 1 Em= (Ts(k,m) of tuples tC2s = (fm:(kkm)) m 1,..... M) of rescaled training points. In this notation, i(k, m) picks out an index corresponding to one of the N training points {Ii,... .N }o that depends both on the identity of the tuple tC₁ (represented by the argument k and the index's position in the tuple (represented by the argument m). In this way, the k-th tuple ICx contains M training samples Ts(k.m) for I,.,, M. The weights of this convex combination depend not only on the distance ||= tčx |l between and the barycenters tč,, but also the variance of the tuples tC2 and the noise terms यı M Em (i(k.m). Tuples of tightly-chustered points have low variance and hence receive large weights in (2); whereas tuples of distant points have high variance and receive small weights in (z), We prove the following proposition in Appendix B.1.
M
Proposition 4.1 (Sa,s points towards barycenters of training points). The smoothed score 80,(2) can be expressed as:
Sa.(2)) (1 - 1)2 (Kio,(2) - 2), where
N
M (I|= 18x 12x112 Var((C*) + otūR) tōk-
(z,(22)
softmax
(5)
2(1 - 1)2
k=1
5 Sampling algorithm
5.1 Forward Euler scheme for sampling
Armed with the smoothed score 86,t, we are now position to define our sampler. Following Liu et al. (2023),
we draw (0, 2) base samples and flow them through
1 (2 + (1 t)sa.
(2) = t
(6)
5

---

Published in Transactions on Machine Learning Research (05/2025)
Algorithm 1 Sampling
Input: Training set faili=1. noise {uimh step size h = 5. initial sample 20 (O(01) do
for O...,
in == 3
Zn+1* - Zn + hVo,e (Zn)
end for return 2T
from = 0 to t 1. We discretize this ODE using a forward Euler scheme, leading to Algorithm 1 for sampling using the smoothed score.
The smoothed score in Equation 3 and Algorithm jointly define our smoothed closed-form diffussion model; given a smoothing parameter a, we call this o-CFDM. Using Algorithm 1, we can sample from a-CFDM given access only to the training data {z,)a= and noise samples. Notably, no training phase neural network is needed for this procedure. By explicitly smoothing the closed-form score rather than relying on a neural network's approximation error, we can determine the support of our o- o-CFDM's distribution. For sufficiently small step sizes, our model's samples will lic at the barycenters of tuples training points.
Theorem 5.1 (Support of o-CFDM samples). All samples returned by Algorithm are of form 2s S (zs-1). As the number of sumpling sleps S - xc (equivalently, as the slep size - 0), the model samples converge towards barycenters zş = Cx of M M-tuples of training points.
We prove this theorem in Appendix B.2. While our sampler is easy to implement and training-Tree, it may be costly if the number of training samples N and the number of sampling steps S are large. We address these issues in the following soctions. In Section 5.4, we show how to approximate our smoothod score using efficient nearest-neighhor search. In Section 6.2, we demonstrate that one may take fewer sampling steps by initializing the sampler at non-zero start time at little cost to sample accuracy, and provide complementary analysis in Section 5.2. This will permit our method to scale to high-dimensional real-world datasets while achieving sampling times competitive with existing methods and running on consumer-grade CPUs.
5.2
Taking fewer sampling steps
As a CFDM's distribution P is simply a time-dependent mixture of Gaussians centered at the training points, it can be directly sampled at any time / by uniformly sampling a mixture mean lx, and then sampling from a > 0 with
Gaussian centered at tzş, We use this fact to sample a o p-CFDM with fewer steps by starting at T> samples from its corresponding unsmoothed CFDM. As a-CFDM does not have the same distribution as an unsmoothed CFDM, this approximation incurs some error, which we bound the following theorem.
Theorem 5.2 (Approximation error from starting al T >> 0). Let pf be the model distribution al l - obtained by starting sampling a a c-CFDM at T > with samples from the unsmoothed CFDM, and let Po,1- be the corresponding o G-CFDM model distribution when sampling starting at T O. Then for any fixed T and e,
Po 0(o).
where W2 is the 2- Wassersteii distance.
Following De Bortoli (2022), stop sampling at time 1 - for some truncation parameter e >>> 0 to account for the fact that the smoothed score 8,,t blows up as t - 1 due to division by (1-t)2. We prove this theorem in Appendix B.3.
This result shows that initializing a o- C-CFDM with samples from the unsmoothed CFDM pf at time T > results in bounded error that scales lincarly with a. Intuitively, increasing o causes the unsmoothed velocity field of be a worse approximation the smoothed velocity field Vo, at any time t; Theorem 5.2 confirms that the cost. to sample accuracy is linear in
6

---

Published in Transactions on Machine Learning Research (05/2025)
5.3 Distribution of one-step samples under Gumbel weight perturbations
When the scalar noise Us, perlurbing the distance weights in Equation 4 is drawn from Gumbel 1) distribution, we can precisely characterize the smoothed model's distribution when performing single-step sampling by starting sampling at the final Euler iteration in Algorithm 1.
Proposition 5.3. Suppose we begin sampling smoothed CFDM at iteration S - of Algorithum 1 usiin samples 28-1 ~ -is-1 from the unsmoothed CFDM at ts-1- Suppose also that the perturbations usmm to the dislance weights in Equation 4 are drawn from Gumbel(O,, 1) distribulion. Then, the namber of Euler 1
steps S + 20, the model samples 2ş are of the form 25 = M Xlo, where X E RDxN is the matrir whose i-th column is training sample z; E IR. and Ig ~ Multinomial( No. M... The probability Tt' of training point xı is given by Th= - sofbmax (-=1l=s-1 - x; 1
We prove this proposition in Appendix B.4. This result provides further intuition on the role of the smoothing parameter in determining the distribution of smoothed CFDM's samples: It is the temperature of the softmax determining xy = softmax (- - - x= When a = 0, the softmax simply picks out the training sample z₁ that is closest to 28-1- Conversely, as a + co, the event probabilitios Th become uniform and Σş becomes the barycenter of M uniformly-sampled training points.
5.4 Fast score computation via approximate nearest-neighbor search
Each sampling step in Algorithm 1 requires the evaluation of the smoothed score so,(2) and hence a sum O(N) terms. For large datasets, each evaluation of (z,( is therefore costly and places substantial demands memory.
In the t + 1 regime, the temperature of the softmax in Equation 4 is low, and the large sum is dominated by the handful of terms corresponding to the smallest values of || ta:;1l atu,m- Tf a sufficiently small, these terms correspond to the nearest neighbors of z among the rescaled training points tx; This suggests that we can approximate the smoothed score 8,, by subsampling terms in the O(N) sum while ensuring that the nearest neighbors of are included with high probability.
Noting that the closed-form score log (t)) = [p)(s) is the score of a Gaussian kernel density estima- o,(2)
tor (KDE) pt, employ Karppa al. (2022)'s unbiased nearest-neighhor estimator for KDEs to estimate the denominator, and take ils gradient to obtaín unbiased estimate of the numerator. We provide details this estimator in Appendix A. Our estimator is computed using the K nearest neighbors of in the training set and L random samples from the remainder of the training set; we study the accuracy tradeoffs associated with K and L in Section 6.2.
Impoctt of stnuthiig on sample possinity Luee diccibutiio..
Given this estimator for the closed-form score, we estimate the smoothed score 50, via convolution against a smoothing kernel as in Section 4.1. By using the approximate nearest neighbor search algorithms implemented in Faiss (Douze et al., 2024), are able to scale our method to high-dimensional real-world datasets and achieve sampling times compelitive with neural SGMs while running on consumer-gres CPUs; see Sections 6.3 and 6.4 for examples and runtime metrics.
0.00
6 Results
6.1
Impact of on generalization
63
We now show that a o-CFDM's model distribution approaches the true distribution P of its training samples x; ~ for appropriate values of a. We fix a continuous target distribution P1 and draw
Figure 2: We between a- o-CFDM model samples and true samples. We depict model samples for {0, 0.26, 1).
N 5000 samples y! to serve as discrete approximation to Pt. We
then draw a smaller subset of n 500 training samples 2; and construct g-CFDM on these samples while varying
7

---

Published in Transactions on Machine Learning Research (05/2025)
For each o, we measure the 2- Wasserstein distance W2 between the o-CFDM's generated samples and the true samples yi Pl, and use this a tractable proxy for the distance between the a-CFDM's model distribution and the true distribution P1- We present the results of this experiment for the "Checkerboard" distribution in Figure 2.
When a = O, the support of our model's samples (left side of Figure 2) coincides with the training samples æ;- The 2- 2-Wasserstein distance between the model samples and true samples y; decreases for small values of a as the model samples become convex combinations of nearby points in the training set; depict model samples for == 0.26 in the center of Figure 2. However, convex hull of the training set (right side of Figure 2) and the distance between our model's samples and true samples yi grows rapidly. These results suggest that for appropriate values of a, our method can use fixed training set (r to generate novel samples that remain close to the target distribution p1- Experiments of this type may be used select appropriate values of for a given application.
a grows large, the model samples spread out to fill the
In Figure 3, we demonstrate that with an appropriate choice of o, our method can sample from 2D surface embedded in I given sparse blue noise sampling of the surface; this is a low-dimensional case of a manifodd-supportedd distribution, which is typical in machine learning applications. Our method's samples (blue points) fill in the gaps between the sparse training samples (red points) while remaining close to the true manifold. This occurs because o-CFDM samples are barycenters of tuples of nearby training points, with a controlling the variance of these tuples. For appropriate values of a and sufficiently dense samplings of training points, these barycenters will approximately lie on tangent planes to the surface, and hence lie near the surface but away from the training data.
(a) a = 0.2: 33.9%%L $ in W2
(b) a = 0.375: 13.4% & in W2
(c) o= 0.4: 34.1% 1 in W2
Figure 3: Sampling a CFCD (blue points) yiclds a dense point cloud given sparse mesh samples (red points). We report % drop in W2 distance to a dense mesh sampling when using our o-CFDM's samples relative to the sparse training samples. We render these point clouds in Polyscope (Sharp al., 2019).
6.2 Ablation and computational trade-offs
In this section, we investigate the impact of the start time T and the parameters of our nearest-neiggbor-bbsee score estimator (9) on the distribution of our model's samples.
Impact of T. As a CFDM's distribution pt simply a time-dependent mixture of Gaussians centered training points, it can be directly sampled any time i by uniformly sampling mixture mean lx, and then sampling from Gaussian centered at tx,, We use this fact to sample a o-CFDM with fewer steps by starting at T >>>0 0 with samples from its corresponding unsmoothed CFDM. We show here that for practical values of o, one can begin sampling at T close to with little accuracy loss.
We fix a contimuus target distribution Pl, draw n = 500 training samples I,, and construct a o-CFDM on these samples for a E {0, 0.2, 0.4, 0.6, 0.8, 1.0). We then vary the initial sampling times T and compute the 2- Wasserstein distance We between model samples generated starting at 0 and T 0. We compare this the average W2 distance between batches of o-CFDM samples generated by starting at T = 0 (which
8

---

Published in Transactions on Machine Learning Research (05/2025)
is nonzero due to randomness in sampling) and report the percent change in W2 relative to this baseline value. We present the results of this experiment for the "Checkerboard" distribution in Figure 4.
For o < 0.4, there is little accuracy loss when starting at T 0, even for start times close to 1. When o 2 0.4, the accuracy of this approximation begins to decline for start times 7 2 0.4, with large reductions in approximation quality when both o and T are large. As we have found that our model has performed best with I 0.4 in the applications considered in this work, this section's results support the use of few sampling steps in practice. The results in Sections 6.3 and 6.4 further support the use of late start times 7' for image generation; we find in these experiments that we can start sampling as late as T 0.98 while maintaining good sample quality.
% chg in W2 incurred by sampling starting at
0.2
0.6
0.8
% chg in W2
300
100
Impact of K and L on the NN-based score estimator. In Section 5.4, proposed an efficient score estimator based on fast nearest-neighhor search. We now study the impact of the number of nearest neighbors K and the number of random samples L from the remainder of the training set on our model's samples.
0.0
0.2
0.4
0.6
0.8
1.0
T
Figure % change in W2 between a- CFDM model samples generated start- ing at T = 0 and samples generated starting at T > 0.
We fix a continuous target distribution Pl; draw n = 500 training samples z;, and construct a o-CFDM on these samples for a 0.3; this value is typical for real-world applications. We then vary the number of ncarest neighbors K and the number of random samples used to compute the score estimator Equation 9 and measure the 2-Wasserstein distance between model samples generated using the full smoothed score and using the estimator Equation 9. We present the results of this experiment for the "Checkerboard" " distribution in Figure 5. We center the diverging color scheme at the W22 distance between two batches of samples from a o-CFDM using the full smoothed score; this noise threshold encodes the inherent randomness in our model's samples across batches.
The error arising from the NN-based estimator is decreasing in K and L, with especially poor approximation quality when using single random
W2 error incurred by NN-baseed soore estimator
sample rg. However, the accuracy of the model samples approaches the noise threshold for small values of K, L. For example, with K = L = 15 (which samples just 6% of the lerms in ki,,), the W2 distance bet ween samples generated using the full score and the NN-based estimator is 0.1865, value close to the noise threshold of 0.1791. In Scctions 6.3 and 6.4, we additionally show that one can generate high-quality images while subsampling ko, at far lower rate, thereby enabling method to scale to real-world datasets.
50 100
1.2
samples
of random samples
20 25
0.8
13
0.6
10
6.3 Image generation in pixel space
0.2
25 # of nearest neighbors
10 15 20
50
100
In this section, are similar to images from the "Smittsoonan Butterflies" dataset," rescaled to 128 x 128. We benchmark our model's sample quality, training time, and sampling time against a denoising diffusion probabilistic model (DDPM) (Ho et al., 2020) and provide training details in Appendix C.1.
o-CFDM to sample images in pixel space that
Figure 5: W2 between c-CDDM model samples generated using the full score and our NN-based estimator for varying # of NN K (horizontal axis) and # of ran- dom samples L (vertical axis).
We display images from a held-out test set along with DDPM samples and our model's samples in Figure 6. Both our model and the DDPM generate images that qualitatively resemble the test images, but as our model can only output barycenters of training samples (see Theorem 5.1), our samples exhibit softer details than the test and DDPM samples. Table records sample quality metrics and training and generation times for our method and the DDPM baseline. Our training- free method achioves
Dataset available Hugging Face: huggan/ thsonian, butterflies subset

---

Published in Transactions on Machine Learning Research (05/2025)
(a) Ground truth images
(b) DDPM samples
(c) a-CFDM samples
Figure 6: Ground truth images from Smithsonian Butterflies (left), DDPM samples (center), and o-CFDM samples (right).
comparable sample quality to a DDPM that has been trained for 5.34 hours on a single V100 GPU, and achieves over 2.9 times the sample throughput of a DDPM running on a V100 GPU while running on a Macbook M1 Pro CPU with GB RAM.
As 128 128 RGB images lie in 49, 10.5mmdaimeli space, this experiment demonstrates that our method scales to high-dimensional problems. As our method is able to generate plausible samples despite being restricted to oulputting barycenters of training samples, it also demonstrates that there exist image manifolds for which our o-CFDM's inductive bias is reasonable. However, we do not expect this inductive bias to be suitable for most real-world image data, where barycenters training samples typically lie off-manifold and fail to resemble ground truth samples. To narrow this gap between theory and practice, we show in the following section that by sampling in the latent space of an autoencoder, our method can generate plausible and diverse images of human faces, comparing favorably with VAE marginally higher sampling costs.
Table 1: Metrics for sample quality and gencration time in pixel space. Our o-CFDM achieves competitive sample quality and generation time while requiring no costly training.
Method
Metric
Butterflies
Inception score 1
1.87 + 0.225
KID 1
0.0220 = 0.0038
DDPM
Training time
5.24 h
Sampling time (GPU)
1.20
Inception score
2.20 + 0.150
KID 1 Training time
0.0114 + 0.0048
c-CFDM
0 0.4124
Sampling time (CPU)
6.4 Image generation in latent space
Theorem 5.1 shows that in the limit of small step sizes, a o-CFDM's samples are barycenters of nearby training points. This is typically poor prior for images in pixel space, but appropriately-chosen autoencoder may map the training data to latent manifold that more closcly satisfies this local lincarity assumption. To this end, we train the nuclear norn-regularized auloencoder proposed by Scarvelis & Solomon (2024), which encourages latent vectors to lie on a low-dimensional manifold. We then sample from a o-CFDM in the latent space of this pretrained autocncoder for the CelebA dataset (Liu et al., 2015) and discard samples that are identical to their nearest latents from the training set. As our method relies on tractable nearest-neighbor queries in the training set sampling time, this feasible post-processing step for
10

---

Published in Transactions on Machine Learning Research (05/2025)
(a) Ground truth samples
(b) VAE samples
(c) a-CFDM samples
Figure 7: Ground truth images from CelebA (left), VAE samples (center), and o-CFDM samples (right).
sampler. We benchmark our method against a Variational AutoEncoder (VAE) (Kingma, 2013) trained for the same number of epochs and employing the same architecture as the muclear norm-regularized autoencoeer. We train both autoencoders using the log hyperbolic cosine reconstruction loss, which has been found to improve sample quality in VAEs (Chen et al., 2019).
A VAE is appropriate baseline for a latent o-CDDM because both models train regularized autoencoder to obtain well-structured latent reprosentations, and then employ a training-free procoss to generate new samples in latent space. A VAE's sampling procedure is simple and data-independdet: One draws a normally-distributed latent and decodes it. However, one must use heavy regularization to ensure the latent distribution is nearly Gaussian, and VAFs suffer from poor sample quality as a result. In contrast, a o-CFDM's data-dependent sampling procedure merely requires that the latent distribution be supported on a manifold of sufficiently low curvature, so that barycenters of nearby latents continue to lie on this manifold. We consequently expect that one may sample a o-CFDM in a weakly regularized latent space to obtain better-quality decoded samples than a VAE while preserving the ability to sample on CPU without requiring additional training.
We display our model's samples, along with VAE samples and ground truth samples from the CelebA dataset in Figure 7. Barycenters of natural images in pixel space typically do not resemble natural images unless they are well-registered (as with the butterflies in Section 6.3), but operating in an autoencoder's 's latent space allows our method to generate plausible and diverse images of human faces. In particular, our method's samples exhibit greater qualitative diversity than the VAE samples, at times including features such as hats and glasses that seldom never appear in the VAE baseline's samples.
Table 2: Metrics for sample quality and generation time in latent space. Our o-CFDM improves significantly on a VAE's sample quality at a marginal sampling cost on CPU.
Method
Metric
CelebA
Inception score 1
1.68 + 0.08 0.108 = 0.0066
VAE
KID 4
Sampling time (CPU)
Inception score t KID +
2.22 + 0.19
c-CFDM
0.092 = 0.0075
Sampling time (CPU)
44 ms
We report sample quality and generation time metrics, including inception scores (Salimans et al., 2016) and kernel inception distances (KID) (Binkowski et al., 2018) between generated samples and samples from the CelebA test partition in Table 2. Our c-CFDM results in 15.0% improvement in KID and 32.4% improvement in inception score compared to the VAE baseline. While the VAE's sampling cost, which
11

---

Published in Transactions on Machine Learning Research (05/2025)
amounts to the cost of generating Gaussian noise, is negligible, our method's sampling time is just 44 ms per sample. For the sake of fairness, this cost is amortized over the number of o-CFDM samples left after discarding nearest neighbors to ensure novelty.
7 Conclusion and future work
In this work, we introduced smoothed closed-form diffusion models (smoothed CFDMs):: a class of training-free diffusion models requiring only access to the training set at sampling time. Smoothed CFDMs leverage the availability of an exact solution to the rcoro-matching problem wwich alone does not yield generalization- and explicitly induce error by smoothing. This results in a model that generalizes by provably outputting barycenters of training points. Our method is efficient and scalable, and runs on consumer-grade laptop with no dedicated GPU.
Our results suggest, that it is possible to design SGMs that generalize without relying on neural score approximations. They also suggest that smoothness is among the inductive biases enabling neural SGMs to generalize in spite of the uninteresting global optimum of their training objective, which only allows for memorization. However, because our method generates barycenters of training points, its inductive bias is unsuitable on its own for sparsely-sampled manifolds in high-dimensional space, which one typically encounters in modern applications such as image generation. In this work, partially address this shortfall by using our method to sample in an appropriately-strrcdaree behind that of state of the art neural SGMs. We therefore encourage further work to close this gap in sample quality, and describe some potential directions for future work below.
latent space, but our sample quality lags
State of the art SGMs are typically built upon convolutional architectures with sell-attention layers, which both feature unique inductive biases. Concurrent work by Kamb & Ganguli (2024) investigates the impact of locality and equivariance constraints the optimum of the scoro-matching objective, and Niedoba al. (2024) empirically investigate whether a locality bias can explain the behavior of neural denoisers. Combining these constraints with our smoothing approach and explicitly modeling the inductive biases of self-attention layers may yield further insights into the generalization of neural diffusion models and lead to new strategies for building training-fi free diffusion models that generalize.
Most interesting image generation tasks are conditional For instance, a user may provide a text prompt and seek an image whose subject and style match the prompt. While state-of-the-art diffusion models typically employ fassifier-free guidance (Ho & Salimans, 2021) to introduce conditioning information, it is unclear how to extend our training-free free method to include an analogous form of guidance. On the other hand, Dhariwal & Nichol (2021)'s classifier guidance would likely be a fcasible addition to our method, amounting to augmenting our velocity field (6) with the gradient of a pretrained classifier. As classifier guidance is known to improve diffusion models' sample quality, this may have the additional benefit narrowing the gap between our samples and those gencrated by statc-of-the-art neural methods.
Acknowledgements
The MIT Geometric Data Processing Group acknowledges the generous support of Army Research Office grants W011NF2010168 and W91INF2110290 of National Science Foundation grant IIS2335492, from the CSAIL Future of Data program, from the MIT IBM Watson AI Laboratory, from the Wistron Corporation, and from the Toyota-CSATI. CSAIL Joint Research Center.
Christopher Scarvolis acknowledges the support of the Natural Sciences and Enginecring Research Council of Canada (NSERC), funding reference number CGSD3-557558 2022,, and a 2024 Exponent fellowship.
References
Sumukh Aithal, Pratyush Maini, Zachary Chasc Lipton, and Zico Kolter. Understanding hallucinations in diffusion models through mode interpolation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URT https://openreview..rr httpe/rreereeeeetttttmmaaa/aaaaa http://rrrrmmt.aaaaaaaaatttaa
12

---

Published in Transactions on Machine Learning Research (05/2025)
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEF/CVP Conference Compuler Vision and Pattern Recognition, pp. 5470 5479, 2022.
Matthias Bastian. Training cost for stable diffusion was just $600,000 and that is a good sign for ai progress. https://the- wwwpe/ccoooer..cc/aaaaaaaaaa wtt.cooo//aaaa 8 -ore-oo--iii-----rrrr-------ieii
sost--or-stable--iff wtt-ddfffooooows----a----------- jetas-just-oe jiraa--ust000000-aaar that-i 2022.
Mikolaj Binkowski, Dougal Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. URL httpps///ppeneeewe.eeeee/orroeee
d=r11UOzMCW.
Pengfei Chen, Guangyong Chen, and Shengyu Zhang. Log hyperbolic cosine loss improves variational auto-encoder, 2019. URL https://openrevien net het//orrurriddrrg.vs0053
Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi S. Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking, In The Eleventh International Conference Learning Representations, 2023.
Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothosis. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL aXo3gB. Expert Certification.
Valentin De Bortoli, James Thortoon, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications score-based generative modeling. Adoances in Neural Informalion Processing Systems, 34: 17695-17709, 2021.
Mauricio Delbracio and Peyman Milanfar.
Inversion by direct iteration: An alternative denoising diffusion for image restoration. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openrevier. net http:://pppaneeeeeeeeeef/ooorr/
Featured Certification.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs image synthesis. Tn A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Newral Information Processing Systems, 2021. URL /ttps://openreviee wwtp:/totrmmddd//aiitaaaoo
Sander Dicleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, et al. Contimuous diffusion for categorical data. arXiv preprint orXiv:2211.15068 2022.
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. 2024.
Mazaré,
William Feller. On the theory of stochastic processes, with particular reference to applications. 1949.
Eric Heitz, Laurent Belcour, and Thomas Chambon. Iterative blendding:::: A minimalist deterministic diffusion model. In ACM SIGGRAPH 2023 Conference Proceedings, SIGGRAPH '23, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701597..
doi: 10.1145 10.1145/3588432 /3515443...39155400 URL
https://doi. org/10..//...... 10. 25443388333.33915540. 3591540.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NearIPS 2021 Workshop on Deep Generalive Models and Dounstreum Applications, 2021. URL KxfYbI.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc., 2020.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey A. Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Flect, and Tim Salimans. Imagen video: High definition video generation with diffusion models. ArXiv, abs/2220000303, 2022a. URL https:/ /ttps://api... semanticscho lar . wow..rg//oooooo arr//oorpusID::52715588..
13

---

Published in Transactions on Machine Learning Research (05/2025)
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. ArXiv, abx/2204.003458, 2022b. URL https://apt sep....smanticscoolarr org/CorpusID: 248006185.
Emicl Hoogcboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3D. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 8867-8887. PMLR, 17-23 Jul 2022.
Aapo Hyvärinen and Peter Dayan. Estimation of non non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, and Stéphane Mallat. Generalization in diffusion models arises from geometry-adaptive harmonic reprosentations. In The Twelfth International Conference on Learning Representations, 2024. URL https: /ttp::///ppeeeeeeeeeeee wttp:iiwwwweeeeeeeeee hett/oorunidd-Alia//aa.
Mason Kamb and Surya Ganguli. An analytic theory of creativity in convolutional diffusion models. arXiv preprint arXiv:2412.2023 2024.
Matti Karppa, Martin Aumüller, and Rasmnus Pagh. DEANN: speeding up kernel-density estimation using approximate nearest neighbor search. In AISTATS, volume 151 of Proceedings of Machine Learning Research, pp. 3108 3137. PMLR, 2022.
Tero Karras, Miika. Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: //penreve/errrviee wwtt:/orrmm
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics (ToG), 42(4):1-14, 2023.
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprinl arXiv:1312.6111 2013.
Fredoric Kochler, Alexander Heckett, and Andrej Risteski. Statistical efficiency of score matching: The view from isoperimetry. In The Eleventh International Conference on Learning Representations, 2023. URL
https://openreevie peet/forummiddTT/AAnnie
Dohyun Kwon, Ying Fan, and Kangwook Lee. Score-based generative modeling secretly minimizes the wasserstein distance. Advances in Newral Information Processing Systems, 35:20205 20217, 2022.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of scoro-basod generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory, pp. 946 985. PMLR, 2023.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. ArXio, /t////222.007777 2022. URL h oatp :1: 247011732.
Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference Learning Representations, 2023. URL
httts://openreewee http////nrrreeeeeeeeeffrrreeeeee
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep lcarning face attributes in the wild.
Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference Learning Representations, 2017. URL. https://aps semanticcchooaa..r.... wwtp:/oorppsII:5555522220
Artem Lukoianov, Haitz Sáez de Ocáriz Borde, Kristjan Greenewald, Vitor Campagnolo Guizilini, Timur Bagautdinov, Vincent Sitzmann, and Justin Solomon. Score distillation via reparametrized DDIM. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: //openrevien iwrpearrviee.nntttoorrriiadeadee
14

---

Published in Transactions on Machine Learning Research (05/2025)
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IBEF/CVF Conference. on Computer Vision and Patlern Recognition, pp. 2021.
Koichi Miyasawa. An empirical bayes estimator of the mean of a normal population. Bulletin of the International Statistical Institute, 38(4):181-188, 1961.
Matthew Niedoba, Berend Zwartsenberg, Kevin Murphy, and Frank Wood. Towards mechanistic explanation of diffusion model generalization. arXiv preprint arXic:2411. 19339, 2024.
Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. ArXie, abs/2303.01e 018611, 2023. URL https: semanticscholar, www.aar..g/ooooeaao CrggCoopusID: 257353832.
Jakiw Pidstrigach. Score- generative models detect manifolds. Advonces in Neural Information Processing Systems, 11355552-35855 2022.
Bon Poolc, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arv, 2022.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International Conference on Machine Learning, pp. 5301-5310. PMLR, 2019.
Martin Raphan and Eero P Simoncelli. Least squares estimation without priors or supervision. Neural computation, 23(2):374-422 420, 2011.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffuusio models. In Proceedings of the IEEE/CVF conference. on computer vision and pattern recogmition, pp. 10684 10695, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022. URL
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Antoine Salmona, Valentin De Bortoli, Julie Delon, and Agnès Desolneux. Can push-forward generative models fit multimodal distributions? In Alice H. Oh, Alekh Agarwal, Daniclle Belgrave, and Kyunghyun Cho (eds.), Advances in Neurol Information Processing Systems, 2022. URL https: http:////pppeeeeeeee/eo net/f orun?id-Tsy9Wco_FE
Filippo Santambrogio. Optimal transport for applied mathematicians, Birkäuser, NY, 55(55-63)))))) 63).944 2015.
Christopher Scarvelis and Justin Solomon. Nuclear norm regularization for deep learning. arXiv preprinl arX5v:2405.14544, 2024.
Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, and Bruno Corrcia. senuctrr-based drug design with equivariant diffusion models. arXio preprinl 2rx::22000.1111 13695, 2022.
Nicholas Sharp et al. Polyscope, 2019. www.polyscope.rm.
Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models, 2023.
Jascha Dick-Dicks,ein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynanics. In Francis Bach and David Blei (ods.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2256 2256-2265, Lille, France, 07- 07-09 Jul 2015. PMLR.
15

---

Published in Transactions on Machine Learning Research (05/2025)
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXi::2010.02502, October 2020. URL bttps://arxhhe w0wp:/i...r//b//110...o. 2ttg/bbs/2000005500.. 02502.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' AlAch-BBu,, F.. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Daniel Stroock and SR Srinivasa Varadhan. Diffusion processes with continuous coefficients, i. Commund- cations on Pure and Applied Mathematics, 2233):34,- 1969a.
Daniel W Stroock and SR Srinivasa Varadhan. Diffusion processes with continuous coefficients, ii. Communi- calions on Pure and Applied Malhemalics, 2274):479 530, 1969b.
Daniel W Stroock and SR Srinivasa Varadhan. Diffusion processes. In Proc. Sirth Berkeley Symp. Math. Slatist. Probah, volume 3, pp. 361-368 1972.
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems, 34:11287 11302, 2021.
Pascal Vincent. A connection between score matching and denoising antoencoders. Neural compulation, 23 10:1661-16774 2011.
Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, and Mohammad Norouzi. Novel view synthesis with diffusion models. ArXio, abs/2210.04628,, 2022. URL https:
Zike Wu, Pan Zhou, Kenji Kawaguchi, and Hanwang Zhang. Fast diffusion model. ArXio, 0bss/230600999 2023. URL /ttp:://api..smmatiiccoto
Mingyang Yi, Jiacheng Sun, and Zhenguo Li. On the generalization of diffusion model. ar preprinl 2rXiv:2305.14712 2023.
TaeHo Yoon, Joo Young Choi, Sehyun Kwon, and Ernest K. Ryu. Diffusion probabilistic models generalize when they fail to memorize. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling, 2023. URL https://openreever. wtwpiiiwwwee/fforrtteee ner//oorrrniicrshccicchhhh
Lvmin Zhang and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models. ArXio, abs/ abs/2332.05553, 2023. URL https://api. httpp:///pp..mmmaiiic.mmmm ccchaatrrrccllrr......omooooaaae org/Corpgs : 256827727.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusson models with exponential integrator. In The Eleventh International Conference Learning Representations, 2023. URL https: https:://oooo http://ppeeeeeeeee//// orunPidsLoekhhfb44.
16

---

Published in Transactions on Machine Learning Research (05/2025)
A Details on nearest-neighbor estimator of closed- form score
Karppa et al. (2022) propose an unbiased estimator of kernel density estimate KDE(2). Given kernel lunction (x(=) with bandwidth h > 0 and dataset { their estimator first searches for the K-nearest. neighbors {anx ) of in the dataset, then draws L random samples {#a) from the remainder of the dataset, and approximates K DDE(2) as follows:
N K Σ Kn(
KDE(=) Σ Kn(o,,
(8)
LN
This estimator is unbiased for any subset of points I& {x: drawn in the first stage. In particular, using approximate nearest-neighbors (ANNs) rather than exact nearest neighbors of increases the variance of Equation 8 but does not introduce bias.
As the fossed-form score Vp; is the score of Gaussian KDE of with bandwidth h = 2(1-t)?, we approximate the closed-form score using the following ratio estimator:
V (lggp(()) = [pi(z)
V
(9)
(2(2) (E(2)
where (2) Karppa et al. (2022)'s 's estimator Equation 8. Since the gradient operator is linear, both the numerator and denominator in Equation are unbiased estimates of their respective terms the closed-form score.
B Proofs
B.1 Proof of Proposition 4.1
For each k I,... Ad let IC2 (lug)).) m L,...,M, M) be M-tuple of rescaled training points tx.. (The same point tæ; can appear multiple times in an M-tuple.)
Define the barycenters and variances of these tuples as follows:
M
Ük
1 M
Σ Wi(k.m)
Var(tC*) 1 M
Σ [ - 1
(10)
M
We will show that up to constant factor, the smoothed score Equation itself the score of mixture of NM Gaussians. Rewriting the seoothed score in gradient form, we have:
17

---

Published in Transactions on Machine Learning Research (05/2025)
1
M
||2 - xX + otug
80,(22
softmax
(Lx, 2)
(1 t)
M
2(1 1)2
M
||2 t0/;1 ? + atus.m
V
Σ
log
exp
M
2(1 - t)²
m=1
i=1
M
N
||2 tx: 11 + otuim
V
log
exp
M
2(1 t)²
m=
i=1
Nd
2m=l (1|= tkk 11? otu)skkm))
M
V
log
exp
M
2(1 t)2
k=1
NM
M (1|= tēx112 Var(tCk) + otus)
V
log
exp
M
2(1 tt2
NM
1 M
((Var(tCk) + otüs)
M| tēx
V
log
exp
exp
2(1 - t)2
2(1 - 1)²
k=1
NMM
1 V. log
M tē 11? 2(1 - t)²
(x(() exp
M
1
V. (z)
M
This shows that up a constant factor J. the smoothed score Sn,(2)) is the score of large mixture of ((I=-EE)11")
Gaussians (z(z)= Ek=1 (t) exp
NM
2(1-r)² The mean of each Gaussian is the barycenter to, of some
M-tuple 1Cx of training points tz)(kkm)) and its common covariance matrix is (1+)) 1. The time-dependent mixture weights wx(() x exp (- [((aa(tCC))+ are decreasing in the variance of the M-tuples tC₃ but 2(1-1)²
are subject to the presence of noise terms atū = at Σm Ui(kmm)-
M
Finally, by expanding the gradient in (+), we straightforwardly obtain:
N
80,((2) (1 8)2
1
Σ
softmax (-
M
(I]= -0x Var(ICk) + +otūs) tox
2(( 1)z
B.2 Proof of Theorem 5.1
Define
N
M
(12 - tēx + Var(t(+) + +atux) ) tor
kz,(2) = Σ softmax
(11)
2(1 - 8)2
that (2, (1-1 (Ko, (k)) 2). Then
18

---

Published in Transactions on Machine Learning Research (05/2025)
(o,((2)
(2 + (1 - t))),((2))
2 + (1 (1- t) (foo)(2) - =2)
| hhoo(()) -
Expanding the formula for the final Euler step using this expression for vz,(2) and ts-1 g
8-1
we obtain:
28 2S-1
(28-11
5
1
1
- +
5 1 5-1
28-1 S-1
k 2- (2s-1)
5
-s-11 +
li 1-s (2s-1) - 25-1
5
8
Ko, 1 (28-1)
8 1
NM
S
MS²
S-1
--1
S-1 S-1 S-1 (2) 2k Ch
softmax
||z--1
cx
1|2 + Var
S
S k
CK*
$4000
In the final line, we use the fact that - oo, the temperature of the softmax goes to and picks out a single index k' such that
k = argmax - (1 Ex1l + Var(Ck) + toux)
argmin (1 - 2 Var(Ck) + +mx))
B.3 Proof Theorem 5.2
We divide the proof of this thoorem into three propositions. We first sketch the proof and state the propositions, and then prove each proposition in subsections below.
Our first result shows that flowing Po through two similar velocity fields ot Cos yields similar model distributions Pr Pa.r at some terminal time T:
Proposition B.1. Suppose a measure Po is pushed through velocily fields vi, Va,t and denole the respective pushforword measures at time by Pi Pa.t- Then,
(' B(t),
Wa(pr Po.T) <
E (of(2) (2) smup
(12)
where B(1) = exp (JTT Lug ds) and Lu; 2 0 is the Lipschitz constant of v
The result above applics to any two velocity fields, subject some weak regularity conditions. To apply this result to the unsmoothed and smoothed velocity fields oz and Vo we bound B(t) and Ello? (2) vo.((2)12 in terms of a:
19

---

Published in Transactions on Machine Learning Research (05/2025)
Proposition B.2. Let ot be velocity field of an unsmoothed CFDM, and let vō be the velocity field Equation 6 of the corresponding a a-CFDM. Then,
D2 ( (1-T)² -
1 11-1
2
3(1) S exp
(13)
and
E (10((2)) - (x)=)12 - zmppt
S C1 2(1 - 1)3
(14)
where D is the diometer of the training data and C1 is constant depending the training data and the distribution Pu of the scalar noise Usm perturbing the distance weights in Equation 4.
Combining these results, oblain the following bound on W2(pr,Poz
W2.T)>PoTT) O
(15)
This shows that one can approximate a o-CFDM's model samples at some time by model samples from corresponding unsmoothed CFDM (i.e. a mixture of Gaussians) when the smoothing parameter a small.
We now show that flowing two similar distributions ph and PG,T through a-CFDM's velocity field from time T to 1 yiclds similar terminal distributions po.1-e: Po,1-e- Following De Bortoli (2022), stop sampling at time 1 for some truncation parameter >>> 0 to account for the fact that the smoothed score 8r, blows up as t + due to division by t1-
Proposition B.3. Suppose prr and pa,r pushed through the velocity field Vo,t of o-CFDM, and let PT. Po.1- denote their respective terminal distributions at time 1 - Then
((-T))
D2 ( )
Po.- 50
exp
6²
Wa(pr,
(16)
where D is the diometer of the training data.
By combining Equation 15 and Equation 16 and treating T and the truncation parameter as fixed, we finally obtain global upper bound on W2(p",1-e P Po.1-)):
Walp-11-oPo--d O(o)
(17)
where P5.1-c the model distribution obtained by starting sampling at T > with samples from the unsmoothed CFDM and Po.1-. is true model distribution of the c-CFDM.
B.3.1 Proof of Proposition B.1
Our proof for this proposition employs techniques similar to those used to prove Theorem 1 and Proposition in Kwon et al. (2022).
We begin with the following well-known result (Santambrogio, 2015, Corollary 5.25):
Suppose that two mcasures p" and p are each pushed through velocity fields vt, v, respectively and denote the pushforward measures at time by pi, pt. Then:
1 d W (e =
(u - x, (2) (x))
(18)
2 dt
20

---

Published in Transactions on Machine Learning Research (05/2025)
where Y is the W2 coupling between pi and pr-
For any 2, y we can use Cauchy-Schwarz and the triangle inequality to obtain the following bound:
x. (v) vx(a)) - ly zl1 (1lo7 (v) - 2 (x)]l + (1) - (=)))
(19)
We can then bound || - of (z) in terms of maximum of the Jacobian Def of ot on the line segment
[x, v] == (ty + (1 - t) O S 1} to obtain:
t x, x - ur (x)) <
max
[Dee_]] 11 - 1 + 11 - x11 1100 (x(2))
(20)
This constant is in turn upper-bounded by the Lipschitz constant Luz of ot on the convex hull of supp(p;) U supp(p:), so we in fact have:
( x, (x) - vx(x)) s Lut 11 - x112 + 11 - xl1 --1--1ut (10((()) (x))l
(21)
Adding
E back in, we get: (x.
1 d
W2
E (z.0)~>
(y 1x, (1) vx(x))
2 dt
s Log (u.y)~7
E
lly x=
E
bly x1| (107(x)) (x))
(a,v)~>
Log W2 (Pi:
+
E
|y æ|1 [1)](x) - "x(x))l
(0.0))
S Lo (2 Pt) +
E
l1 2012
E (2.0)mme
|| - ey (x)112
V
(2.9)~7
= -w: + W2 (p
IE xpppp
1lu (x) v=(x))12
where we use Cauchy-Schwarz for random variables in passing from the third to fourth lines and then the fact that o is one of the marginals of Tt. Using the chain rule on the LHS and cancelling a factor of W2(Pt, Pt) from both sides, obtain the following differential incquality:
d Wa(pi, s Wa(piii)))) + v Vampe (1)( vr (x) di
E
(22)
We can now solve the diffcrential incquality Equation 22 to obtain:
We(pT.Pr) $ ('
B(t), E aapp
[1]((x) (x)
(23)
where
B(1) = exp (J." Lunds)
(24)
21

---

Published in Transactions on Machine Learning Research (05/2025)
B.3.2 Proof of Proposition B.2
We first estimate {()) exp (S² Lo As
(x) t(1 f) (2) t²
(25)
bound its Lipschitz constant by Log max [(1-1) T,y; } Our next step therefore to bound Lag- We will do so by bounding the spectral norm of the Jacobian k| (2)]l2 of ky for any 2 E R..
To this end, we first note that for any z E IR this Jacobian has the form of weighted covariance matrix:
(z) (1-t)² t)² Σ (xx -x(()))
where (x) is the i-th component of the weight vector softmax norm, we first apply the triangle inoquality and use the absolute homogeneity of norms:
|
To upper bound its spectral
2(1-t)=
N
[
wil (tt;
- (z)) -())(tt-())))))( (=))
(1
1)2
1
we (x(2))l((tr)) - (=))))(to= k=)))
t)
But it is well-known that the spectral norm of a rank-1 matrix uo given by ||ue" TiaTTp 1|2 = |||||||||||z, so fact
(=)ll2 < (1 1)2 -
(26)
Now, because K is a convex combination of the tr,, it lies in the convex hull of the tz,, The diameter of a convex hull is bounded by the diameter of extreme points, so D diam (a,)) and ID (iam(((r,) ) then Itz, - kt (2) < (tD) Substituting this back into equation 26 and using the fact that the wx(2) sum to for any we obtain the bound [x)](2)]22 ( (1))). This bound holds for all E RR, so it follows that
Lk; I 1-t
IDs 1 Hence Luj S 2 max
{
(1-133: 1-e
We this bound Twp to estimate 3(1). Let 5 (0, denote the time from which
=D= 2 1-3
[1-57
Then 5 == (D2 Dy D2 + 2). Decomposing the integral that defines log ((),, obtain:
22

---

Published in Transactions on Machine Learning Research (05/2025)
J. T
Lu {( Lo + { Lo
1 2
1 - ds + 2D2
(1 8)3 ds
8
1-t
27
25 -
2
+ D
8
(1 - T)
(1-3)2
1-t
2T -1
D2 - DVD + +4+11
2
+ D
4
1-T
(1-1)2
(D2 D2 4)2
Substituting this bound into ((t) = exp(S" Loz) and simplifying, we obtain:
(1-4)
3())C((T)
(27)
1-
where C(T) = exp training data.
(22
27-11 (1-77²
4 D2_ DVD244+1
(D2-DvD DVD )) = =exp (D1(2--1)) and D =; Co depends only on the
We now estimate
E' xmppl
11u| (x) (2)113
We first observe that (2) vi (z) æ(11) (k7 (k
- kc(z)). Once again letting X E PXN be the matrix
of training data and w²(z) = softmax 2(1-1)3 E RWW #m (2) = softmax (- (-1le-ta (=ll=ttr||++atuu tatugm
||3-10|3
E RN be the
2(1-1)²
vector of weights, we have that (z) = xxw" and hence
M
M
(z) ve(=)]l = = - (x)=(())
Σ
Tm (2))1 -
1 XI|
Σ
|w" (z) - (m (28)
M
1-t
M
m=
Once again using the Lipschitz continuity of w()), obtain the bound and by substituting this into our bound for |1 (=) - (x) we obtain:
(2) - um (=)ll otu;
(29)
2(1 1)2
2 1 4(1 t)o
[=) wx())) <
(30)
1 where ū; = 37 Em Him.- As this bound holds for all 2, it also holds in expectation, so we finally conclude that
x~pp (lu (2))1 < ta |||
(31)
2(1 1)3
B.3.3 Proof of Proposition B.3
We now begin with the differential inequality
We(pi s Lu; W2(Pi, +
E
[1) - 2y (2)
(32)
dt
V x=mp:"
23

---

Published in Transactions on Machine Learning Research (05/2025)
that derived in the proof of Proposition B.1, which bounds the rate of change in W2(Pt: pi) when flowing Pe and through two velocity fields ez and vt, respectively. As we now consider the case where pT and Pa.t both flow through the smoothed velocity field Vott from time T to 1 6, V xxmpp
E (1v mx())12 = 0 and the
differential inequality becomes:
d We(pi.Po.) S Lun, W2(Pi, dt
(33)
Solving this differential inequality, we obtain
1P = Po) S Po..)
(34)
where 3(T) exp ds do). that Uo,t is at least as smooth as v7. we obtain
Using the same bounds as in our proof of Proposition B.2 while noting
1-T
1 2E
D DVDD -
B(T) S exp
2log(
+
D2
22
(122 DV
2
2
D32 DV
D2
exp
2
(D2 DV 4)2
T
D2
2c)
exp
22
where D the diameter of the training data, which we treat as constant for given CFDM. Substituting this into Equation 34, we obtain
((--))
1-T 2
D² (
26)
W Wold So
exp
Wa(PrsPo:T).
(35)
c2
B.4 Proof of Proposition 5.3
We showed in Theorem 5.1 (see Appendix B.2) that as the number of sampling steps S - oo, the samples from a smoothed CFDM converge towards barycenters zs ck- of M-tuples of training points for indices k* such that:
ht (25-1) argmax - (|28-1-2a) + Var(C + oūx)
(36)
&
Using an equivalent expression for Ro.,, these barycenters can also be written as
M
2g
T2-
Σ
T (3s-1-m)))
(37)
M
where
24

---

Published in Transactions on Machine Learning Research (05/2025)
(2s-1, m) argmax (1|2--1- - -x,112 tuss
11s--1 xill? + Wim
argmax
If dem (umbel(O, 1), then by applying the Gumbel max-trick, we conclude that ("(2s-1, m) Categorical(Tt). This is distribution over the indices 1, .V of training samples, with event probabilities given by
- 1 12:1-1-2;113) 22:12)
T softmax
(38)
If we represent Ti- as Xee, where X E DxN is the matrix whose i-th column is training sample æ; and Cj- E R. the i--th standard basis vector, then
M
25
(=8--mm)
M
m=1
(Xec-)
M
M
E
M
XI
M
But I, = DM e₁. is realization of a Multinomial(7.,.M] M) random variable; this fact completes the proof of Proposition 5.3.
C Additional Experimental Details and Results
In this appendix, we provide details for our pixel space and latent space image generation experiments.
C.1 Pixel space DDPM training details
Our training data is drawn from the dataset huggan/smi sisoniitan butterflies i susssssuutt,,teee which is publicly available on huggingface and contains 1000 images of butterflies. We extract RGB images from the image column of their dataset and reshape them to 128 x 128 before using them in our experiments. We construct an 80/20 train-test split and use the train partition to train the DDPM and to construct our o-CFDM, and use the test partition to compute metrics.
We the DDPM implemented in the lucidrains library denolssingaiifuuusipppooocch as our baseline. We use their UNet with dim mults= (1, 2, 4, 8) backbone. We use 1000 time steps during training, and use DDIM sampling with 100 time steps during sample generation. We train the diffusion model with a batch size of 8 at a learning rate of 5 x 10-5 for 20,000 iterations.
We center and normalize the training data to lie in the unit ball before using to construct our cCCFDM. We set M 2 and a 0.1 for this experiment, and compute the smoothed score exactly rather than using our nearest neighbor- baaedd estimator from Section 5.4 due to this dataset's relatively small size. We start sampling at. T 0.98 and step size 0.01. We filter out model samples whose Fuclidean distance is within
25

---

Published in Transactions on Machine Learning Research (05/2025)
10-6 of their nearest neighbor in the training set; with these hyperparameters, roughly 60% of the model samples remain after this filtering step.
roughly
We compute our metrics using the torchmetrics implementation of the kernel inception distance (KID) and inception score. We compute KID scores with subset_size=50 the test partition and our CFDM and DDPM samples.
size=50 between 500 randomly- images from
C.2 CelebA latent space generation details
Our method uses the nuclear norm-regularized autoencoder from Scarvelis & Solomon (2024). This autoencoder operates 256 256 images from the CclebA dataset. To reduce the memory and compute costs of our autoencoder, we perform a discrete cosine transform (DCT) using the torch-dct package and keep only the first 80 DCT coefficients. We then pass these coefficients into the autoencoder.
The autoencoder consists an encoder fo followed by a decoder 9o. The encoder fo is parametrized as a two-layer MT.P with 10,000 hidden units; the latent space 700-dimensional. The decoder 9e consists of a two-layer MLP with 10,000 hidden units and 3 * 80 80 19200 output dimensions, followed by an inverse DCT, and finally UNet. We set the regularization parameter to l/ 4 (see (Scarvelis & Solomon, 2024, Appendix B.3) for details the training objective) and use log-cosh reconstruction loss (Chen et al., 2019) for improved sample quality. We train for 100 epochs at a learning rate of 10-4 using the AdamW optimizer (Loshchilov & Hutter, 2017).
We then sample our o- o-CFDM in the latent space of this pre-trained autoencoder. We center and normalize the training data to lie in the unit ball before using it to construct our o-CFDM. We set M = 2 and a = 0.025 for this experiment and use the nearest neighbor- bassed score estimator described in Section 5.4. We start sampling at T = 0.99 and use step size 0.01. We filter out model samples whose Fuclidean distance is within 10-6 of their nearest neighbor in the training set; with these hyperparameters, roughly 34% of the model samples remain after this filtering step.
Our baseline is a VAE with the same architecture as the muclear norm-regularized autoencoder and the same log-cosh reconstruction loss. We set the regularization strength at 10-3 and train for 100 epochs at a learning rate of 10-4 using the AdamW optimizer. At sampling time, we decode Gaussian samples drawn from (0, 101); we find that this results in improved sample quality relative to sampling from standard normal distribution.
We compute our metrics using the torchmetrics implementation of the kernel inception distance (KID) and inception score. We compute KID scores with subset subset_size=50 size==0 between 500 randomly-chosen images from the test partition and our CFDM and DDPM samples.
D Impact of M on model samples
In this appendix, we demonstrate the impact of M - the number of noise samples used to computed the smoothed score Equation 3 on a cCCDDM's model samples. In Figure 8, we use a simple training set of points (in blue), fix a = 1, generate 100 o-CFDM samples (in red) for different values of M. Note in particular that. for large values of M, the model samples cluster around the centroid of the two training points. We conjecture that this phenomenon may be explained by the law of large numbers: As M -> oc, 1 EM= ke(x + aem) + Ee + ae), which deterministic quantity lying on the line segment connecting the two training points. In this regime, the reasoning used in the proof of Theorem 5.1 suggests that conditional on the second-to-last sampling iterate #s-1, the output of c-CFDM becomes deterministic and all randomness in the model samples originates from 25-1-
In Figure 9, we carry out a similar experiment with a training set consisting of 500 samples from the checkerboard distribution and a 0.3. Note that for large values of M, the model samples recede from boundary of the convex hull of the training data; we conjecture that this an instance of the same phenomenon in Figure& 8.
26

---

Published in Transactions on Machine Learning Research (05/2025)
We finally experiment with the impact of M on the latent sampling algorithm introduced in Section 6.4. We fix all hyperparameters but M to the values described in Appendix C.2 and use our o-CFDM to sample with M E 6,8) We depict grids of decoded samples in Figure 10 and report sample quality metrics in Table 3. Using small values M tends to yield increased sample diversity while keeping sampling costs low, whereas large values of M yield a greater proportion of samples which qualitatively resemble barycenters of face images. This also likely an instance of the phenomenon illustrated in Figure 8, in which model samples cluster around centroids of training samples for large values of M.
Table 3: Metrics for a- C-CFDM sample quality and generation time in latent space as function of M.
Value of M
Metric
CelebA
Inception score t
2.18 + 0.22
M = 2
KID +
0.091 + 0.0071
Sampling time (CPU)
41 ms
Inception score f
2.13 + 0.16
M 4
KID +
0.095 + 0.0077
Sampling time (CPU)
52 ms
Inception score f
2.03 + 0.17
M
KID +
0.098 + 0.0088
Sampling time (CPU)
97 ms 2.11 0.19
Inception score t
M = 8
KID +
0.095 + 0.0073
Sampling time (CPU)
368 ms
M = 2
M 4
M 10
M 20
5.-
Model somples ning samples
Model samples Training samples
Nodel samples Training samples
Model samples Trairing samples
132
1.2
15-
6.-
-35
-05
-13
-18
-32
-3.0
-1.
-15
2/2
-25
-3.5
6.2
M
50
M
100
M
500
M
1000
Model samples Tra ning samples
Model samples
Model samples
Model samples Training samples
Training samples
Training samples
-12-
-25
-03
-05
-13
-10
Figure 8: o-CFDM samples (in red) generated given two training points (in blue) for various M.
E Comparison of Gaussian and Gumbel noise for latent space sampling
In this appendix, we briefly illustrate our method's robustness to the distribution of noise used for smoothing the closed-form score. We use our oCCFDM sample CelebA images in latent space using the same hyperparameters as described in Section C.2, and compare the effects of smoothing the closed-form score with Gaussian noise and with Gumbel noise whose first two moments match those of the Gaussian noise. We
27

---

Published in Transactions on Machine Learning Research (05/2025)
10
Figure 9: d-CFDM samples (in red) generated given 500 training samples from the checkerboard distribution (in blue) for various M.
depict grids of decoded samples in Figure 11 and report sample quality metrics in Table Our method is robust the distribution of smoothing noise, with Gaussian and Gumbel noise resulting in similar sample quality metrics and qualitatively similar samples.
Table 4: Metrics for c-CDDM sample quality and generation time in latent space when smoothing with Gaussian and Gumbel noise with matched mean and covariance.
Noise distribution
Metric
CelebA 2.18 + 0.22
Inception score
Gaussian
KID & Sampling time (CPU)
0.091 + 0.0071
41 ms
Inception score
2.22 + 0.19
Gumbel
KID &
0.092 + 0.0082
Sampling time (CPU)
50 ms
F
Adding noise to the velocity field does not induce generalization
The proof of Theorem 5.1 shows that in the limit of small step sizes, a-CFDM outputs barycenters of training samples in its final sampling iteration. This is true regardless of the position of the second-to-last iterate ø5-1- Consequently, adding noise to the velocity field at each sampling step, as is typical for the Marrea-aaruyaaa scheme to simulate stochastic differential equations (SDEs), does not fundamentally change the result of Theorem 5.1: "SDE sampling" for c-CFDM would still output barycenters of training samples, provided the noise variance vanishes as + (If the noise variance does not vanish at the final sampling iteration, then the sampler will return noisy samples
noise remains present in the final iteration.)
Furthermore, adding noise to unsmoothed CFDM's velocity field is insulficient to induce generaliza- tion. Theorem 5.1 also shows that the output of our sampler is of the form S agt (zs-1). This would not change if the velocity field were augmented with noise that vanishes in the final sampling it- eration; only =s-1 would change. When the smoothing parameter 0, k 2ي1 (2s-1) ks (2s-1)
28

---

Published in Transactions on Machine Learning Research (05/2025)
(a) M 2
(b) M 4
(c) 6
(d) M
Figure 10: o-CFDM samples drawn in latent space with M E (2,4
ΣN softmax (-11-- nearly O, which implies that the unsmoothed CFDM outputs a training sample in its final iteration, regardless of the penultimate sample æ5-1. It follows that augmenting the velocity field it of an unsmoothed CFDM with noise does not induce generalization.
2 )₂ S x4 For sufficiently small step sizes, the temperature of this softmax is
Sz1n,
We demonstrate this empirically in Figure 12, in which we sample from an unsmoothed CFDM constructed from 500 empirical samples from the 2D checkerboard distribution, and from the same CFDM whose velocity field at time t is augmented with Gaussian noise with covariance V0.1(1 10t-t). t)I. The step size in these experiments is 10-2. Adding noise to the velocity field does not induce generalization, and the model samples are numerically identical to training samples in each case: the chamfer distance between the noiseless CFDM's model samples and the training samples 6.41 x 10-4, and the chamfer distance between the noisy CFDM's model samples and the training samples is 6.59 10-4
G Impact of step size on generalization
In this section, we demonstrate empirically that an unsmoothed CFDM memorizes its training data, even when sampled with relatively few iterations.
29

---

Published in Transactions on Machine Learning Research (05/2025)
(a) Gaussian noise
(b) Gumbel noise
Figure 11: a o-CFDM samples drawn in latent space using Gaussian noise (left) and Gumbel noise (right) with matched means and covariances to smooth the closed-form score.
Unsmoothedd samples velocity Unsmoothed samples noisy velocity
(a) Noiscless velocity field
(b) Noisy velocity field
Figure 12: Augmenting an unsmoothed CFDM's velocity field with Gaussian noise that vanishes in the final step does not induce generalization.
We first show that sampling an unsmoothed CFDM with large step sizes may in principle lead generalization.
To do so, employ similar arguments in the proof of Theorem 5.1 in Appendix B.2. In particular, if
= 0, then the smoothed velocity field (2,(2) can be expressed as (2) = vr(2) = 1 (2 (1 2). By expanding the formula. for the final Euler step using this expression for vx(2) and fs-1 S-1 in the same way in Appendix B.2 but substituting the softmax weights from the definition of ke(z) in equation 2, one sees that the final iterate 29 will be of the form
1-t
2 28 softmax --1== 5-
Fz.
2
If the number of sampling iterations small, then the temperature of this softmax may be sufficiently large that as is non-trivial convex combination of training samples. However, demonstrate empirically
30

---

Published in Transactions on Machine Learning Research (05/2025)
that this does not occur in practice, for relatively small S. We construct unsmoothed CFDM using 500 samples from 100-dimensional standard Gaussian distribution to mimic the high-dimensional data that is typical of real-world applications, sample it with a number of iterations ranging from S = 1 to S = 100, and compute the chamfer distance between the model samples and the training samples for each value of S. We depict the result. of this experiment in Figure 13. An unsmoothed CFDM memorizes its training data when sampled with as few as iterations, demonstrating that in practice, large step sizes alone are insufficient cause CFDM to generalize.
Chamfer distance vs number of sampling steps
Chamfer distance
10 Number sampling steps
10'
Figure 13: An unsmoothed CFDM memorizes its training data when sampled with as few as 5 iterations.
H Comparing neural SGMs to o-CFDMs.
In this section, we present neural SGMs and that of our proposed o-CFDM. We first show that in low-dimensional problems, may approximate the velocity field of neural SGM with that of c-CFDM for an appropriate choice of a. We then show that this is no longer the case for dign-dimensional image datasets such as MNIST with the velocity field uy(2) parametrized by a Unet. Our strategy will be to extract (x(2) from a nourally-paramottized (2 and show that contrast to a o-CDDM, the neural (z) does not output convex combinations of training samples for s
preliminary findings the relationship between the inductive bias of
H.1 2D datasets
For this experiment, we train a simple 3-layer MLP on the flow-matching objective from (Liu et al., 2023), whose thcorctical optimum is attained by the velocity field (2(2) := (1- log (t).. (Recall that for all 0 L < 1, Pt is a mixture of Gaussians centered at rescaled training samples with common covariance (1 1)) Our MLP has 64 neurons in each hidden layer and uses Softplus activations. We train this MLP for 20k epochs at learning rate of 10-3 using AdamW, and take the target distributions to be the empirical distribution over fixed sets of 500 samples from the 2D "Checkerboard" dataset used throughout this paper and a 2D "spirals" dataset, respectively.
We then sweep over a E [0, 2] and construct a o-CFDM on the same 500 training samples for each value of a. For each value of o, we then sweep over E (0, 1), draw batches of samples from PF, and compute the average squared Taz distance between the neural velocity field ot and the velocity field Ua defined in equation 6. We
31

---

Published in Transactions on Machine Learning Research (05/2025)
report the average squared Lg distance across t for each value of in Figure 14. The unsmoothed CFDM's velocity field oi is poor approximation to the neural SGM's velocity field, indicating that the neural SGM does not learn the closed-form score for its training set. However, smoothing the CFDM siguificantly improves the quality of the approximation, with o = 0.45 achieving an 88.8% reduction in squarod Lg error against the neural velocity field relative to the unsmoothed velocity field on the checkerboard dataset, and a 0.15 achieving an 81. 81.1% reduction squared L2 error on the spirals dataset.
Average squared
vs. velocity fields
gAverage squared evvor velecity
Optionl siyma
auue er punnn
12 error
Optimal sigma
0.50
0.75
1,75
2,00
0.00
0.25
0.50
1.50
1.75 2.00
Skgma
Sagma
(a) "Checkerboard" datasett
(b) "Spirals" dataset
Figure 14: An insmoothed CFDM's velocity field is a poor approximation to a neural SGM trained on the same dataset, but smoothing it significantly improves the quality of the approximation.
In Figure 15, we compare samples crawn from the neural SGM (in red) to samples drawn from a o-CFDM with the optimal values of a = 0.45 and o 0.15 for the checkerboard and spirals datasets, respectively. While each distribution's samples are qualitatively similar, we note in particular that the o-CFDM places less mass near extreme points of the support of the training data. We observed a similar phenomenon Appendix D and conjectured that it results from /े Em= ky(x + oem) converging towards deterministic quantity for sufficiently large values of M. We further conjecture that this phenomenon may partially explain why neural SGMs generalize better dinn--dmensiioal image generation problems, they place more mass near extreme points of the data distribution and less mass on barycenters of training samples, which typically do not resemble natural images.
Finally, in Figure 16, we compare the velocity fields a neural SGM and an appropriately-smoode for the checkerboard and spirals datascts at thrce times: t e {0.1, 0.9}. We normalize the velocity fields in the top row of each subfigure to facilitate a comparison of the direction of each velocity field, and depict the difference of the non-normalized velocity fields in the bottom rows. Smoothing CFDM yields velocity field that accurately approximates the corresponding neural SGM's velocity field for t close to 0, but the accuracy of this approximation deteriorates as t +-> 1.
o-CFDM
H.2 MNIST
In this section, we will show that if one parametrizos by Unet and trains it on high-dimensional image dataset such as MNIST, the resulting model does not. behave like a o-CFDM. Because sample estimates of the squared L2 distance between the neural i and a o-CFDM's Uo are noisy in high dimensions, we do not employ the stratogy from the previous section compare neural SGM and our o-CFDM, but instead study the extent to which neural SGM points towards convex combinations of training samples in image generation problems.
32

---

Published in Transactions on Machine Learning Research (05/2025)
Noural va SCFDM samples
Neural vs SCFDM samples
Noural modal sureplis SCSDM cameles.
Neural modal samples SCRDM sauples
(a) "Checkerboard" datasot, a = 0.45
(b) "Two Spirals" dataset, o = 0.15
Figure 15: A C-CFDM generates samples that are qualitatively similar to a neural SGM on low-dimensional datasets, but sends less mass near extreme points of the data distribution.
(ogg where V Vlog = (1-1)² (hx(2) 2)), one may extract the corresponding k2 by substituting the formula for V log (z) into the formula for 1t and rearranging, Equation and Proposition 4.1 show that if (z) is extracted from either of the unsmoothed or smoothod scores 8,, (a = and a > 0, resp.); then it must output convex combinations of rescaled training samples for any 2: (=, = Ei uz(2)tz; with wx(2) 2 0 and Σ wz(z) 1. We will show that for t 1, the k2 function extracted from a neural SGM's velocity field vi is not well-approximated by convex combinations of rescaled training samples. This will imply that unlike a o-CFDM, a neural SGM's score function does not point towards convex combinations of training data.
Given a velocity field 0e(2)= ==
-
To this end, we train a Unet on the flow-matching objective from (Liu et al., 2023), whose theoretical optimum is attained by the velocity field vx(2) == [ (1 - V)V log (=)). The training set consists of the 60k images of handwritten digits from the MNIST train partition. We train for 10 epochs at a learning rate of 10-4 using AdamW.
We then draw a batch of 512 samples Iy from the MNIST test partition, fix a value of t E 1), and compute noisy samples of the form žt,x = Lxx + (1 - l)e, where e ~ (0, We compute the neural SGM's kz(2,,)) for each noisy sample, and use projected gradient. descent to regress each (=,,,, on the set of rescaled training samples tæ; subject to the constraint that the weights lie in the probability simplex. If the neural SGM is well-approximated by a o-CFDM, then we would expect the MSE of this regression to be close to O.
In the left panel of Figure 17, we depict the average MSE of the regression of each (=(,.))) onto the convex hull of the training samples I; as a function of t. (We rescale (z(,),)) by 1 to enable a direct comparison to the convex hull of the training samples z;; otherwise, the MSE values for small t would be small simply because the data has been scaled by L.) For small values of I, the function th, extracted from neural SGM is well-approximated by a convex combination of training samples F,, as one would expect for a c-CDDM. However, the quality of this approximation deteriorates as 1. This indicates that by pointing towards the convex hull of the training data, neural SGM behaves like a c-CFDM for t - 0, but that this behavior vanishes as t - 1.
I a neural SGM fails to behave like a c-CFDM for ( + 1, then how does it instead behave? The right panel of Figure 17 indicates that for + 1, the h, extracted from a neural SGM approximates an optimal denoiser for test data. In particular, by computing kz(2,) for a noisy test sample 2, = fx + (1 - t)c, one recovers the rescaled test sample fr. Because test samples often lie outside the convex hull of neural SGM's training data, this behavior cannot be replicated by c-CFDM. This provides evidence of a neural SGM's generalization
33

---

Published in Transactions on Machine Learning Research (05/2025)
capabilities by showing that its score points towards regions of the support of the target distribution that are outside the convex hull of the training data.
We further illustrate this phenomenon in Figure 18. When t 0.2, the output of the neural SGM's kx(z) on noisy data is well-approximated by a convex combination of training samples, as the theory of o-CFDMs predicts. In contrast, when & = 0.8, a neural SGM's k<(2) behaves like an optimal denoiser for test data, mapping noisy test samples their clean counterparts. While comprehensive study of the generalization of neural SGMs is out of scope for this work, we point to concurrent work such as Kamb & Ganguli (2024), which studies closed-form solutions to the scote-matching problem under locality and equivariance constraints to better understand the generalization of SGMs parametrized by convolutional architectures.
34

---

Published in Transactions on Machine Learning Research (05/2025)
Nocuollaed sciecity 1w0.]
Notrzliee velocity
Nocualaee velecity 1m0.0
neural velecty t-O.1
Teasde
(a) "Checkerboard" dataset, 0.45
Nectraloond veluctty falds, 2w0)]
Normaltzed veluctty Belds, t-0.3
Nommltooi velocity Belds, tHO.
#
ot * 1
of 2
SUPUN minu noundl velucity Goldl L-0.1
SUPUM nines nound velcctt fulld p-0l
SCHUM minus cound velusity fiiid, 1-0.3
W. v
"Two Spirals" dataset, 0.15
Figure 16: Smoothing CFDM yields a velocity field that accurately approximates neural SGM's velocity field for small times, but the accuracy of the approximation deteriorates as 1.
35

---

Published in Transactions on Machine Learning Research (05/2025)
Avg MSE between neural ((t(z) and wileix
Ang MSE betwees neural Hetttx (1 t14) and
230
200
200
150
130
100
73
(a) A neural SGM's by is close to the convex hull of the training data for small t, but not for t + 1.
(b) A neural SGM's ky behaves like an
optimal denoiser for test data as / -> 1.
Figure 17: A neural SGM's k2 bchaves like o-CFDM's ky for t - 0. However, as t - 1, it instead behaves like an optimal denoiser for test data.
Test images
Noisy test images (t 0.2)
kzlz)
Proj of kz(z) onto convex hull
/
0
8
8
/
6 7 8
/
6
7
8
/
2
5 4 9
0
7
8.
5 6 5
2
7 2
5
/ 9
4 4
0
9 4
0
9
4
6
0
/
3
0
3
0
5
/
Test images
Noisy test images 0.8)
kzlz)
Proj of kz(z) onto convex hull
/
0
8
8
0
8
8
/
0
8 8
/
8
8
/
550 0
/
2
5
0
2 4 9 1
/
5 4 9
0
/
2
5
O
/
4
4 0
/
4
4
0
/
0
/ 4
9
0
9
4
9
/
9
4
9
/
/
9 4
9
/
Figure 18: Row shows that for small t, neural SGM's kz(2) outputs convex combinations of training samples, one expects for o-CFDM. In contrast, Row 2 shows that for large f, neural SGM's kx behaves like an optimal denoiser for test data, mapping noisy test samples to their clean counterparts
36


Reviews:
[{"review_id": "sJzL8pCSxt", "reviewer": "Reviewer_9gc6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths** \n\n1. The generation ability is an essential in generative modeling. While most of the existing works study the generation ability of the popular diffusion models, the authors bring a new perspective, to find new algorithms with comparable generation ability. In this paper, the smooth CFDM is motivated from properties of diffusion models. But it is much more simpler as it doesn't require to train any neural networks. \n\n2. The authors discuss both the theoretical and experimental properties of the smoothed CFDM, which could benefit broader audiences that are interested in different aspects of generative modeling.\n\n**Weaknesses**\n\nI hope there could be a separate section providing a more complete comparison to diffusion models from both the analytical and empirical perspectives.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a training-free generation algorithm, namely the smoothed closed-form diffusion models (CFDM). The idea is to utilize the simple closed-form score functions for the empirical distributions in diffusion models, adding noises manually when evaluating the scores, hence guarantee the generation ability of the algorithm. The authors focusing on analyzing the generation ability of the algorithm and implementing the algorithm more efficiently. More specifically, they\n\n1. show the manual noises (M many) encourage the score function to pointing towards the barycenters of M-tuples of training samples. Therefore, the algorithm generates new samples.\n\n2. show the support of the generated samples tends to the barycenters of the M-tuples of the training samples when the step-size tends to zero.\n\n3. apply the nearest-neighbor-based estimator to approximate the score, which save computational cost while preserving the sample accuracy.\n\nLast, they experimentally test their algorithms by checking the generation ability, ablation and computational trade-offs and apply it to high-dimensional image generation tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. the notation $x_{i(k,m)}$ for $M$-tuples is a little confusing. Why does the index $i$ mean? A clarification of the notation would be helpful when it is first introduced.\n\n2. what is the trade-off between sample quality and generation ability in smoothed CFMD? How would it depend on parameters $M,\\sigma$? Can this trade-off be theoretically quantified (at least for some empirical data distributions)?\n\n3. When we apply the algorithm in practice? How do we choose the optimal $\\sigma$? What statistics should we look at when choosing $\\sigma$?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DlWt8tCMte", "reviewer": "Reviewer_hJJF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper builds on a good (and somewhat well known) observation that the generalization properties of diffusion models must be induced by the score error (as other sources of error are controlled, and without the score error, the model should memorize the data). Based on this, it is a good idea to try to come up with a structured way of perturbing the score to establish generalization.\n\nWhat I see as a weakness, however, the whole execution of the paper. The authors try to come up with a new idea/new sampling method (fundamentally) by perturbing the score. Obviously, the quality of the samples will not match modern diffusion models. I would rather like to see this idea as an investigation of the inductive bias of the modern diffusion models. See below more on this point.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This is an interesting paper, attempting to replicate the generalization inducing error of the score in diffusion models by using smoothing methods. This does result in a training-free diffusion model. A simple estimator is proposed and experimental results are provided.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It is clear to me that the introduced method will not fundamentally outperform modern score based models. However, I will put this as a question below to allow answers from authors. Other suggestions will come after that.\n\n1) The samples, say in Figure 7(c), should be compared to properly trained (latent) diffusion models. An interesting question here would be about a fair comparison. How big should the score network of a standard trained SGM be - for a fair comparison to your method? Even if the results are negative (that is SGM will clearly beat your method), include it. See my point below for how to actually change a bit the direction of this paper.\n\n2) The comparison to DDPM is on the butterfly dataset. Why was this not done on faces or something more standard - something that is easier to assess?\n\n3) What would be interesting for me is the following: It is clear that standard SGMs induce an error on the score, compared to the empirical score (which would result in memorizing). However, we don't know either that whether this estimator converges to the ``optimal score''. It would be interesting, even for low-dimensional examples, compute the exact score and trained score and compare the bias in the trained score to your approach. Perhaps some structural information can be used here to adjust your estimator, $\\sigma$, or the kernel. This would also contribute to a broader understanding of why these models generalize.", "broader_impact_concerns": "Not a particular concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HdBbkbvVFi", "reviewer": "Reviewer_C91g", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "$\\textbf{Strengths:}$\n\n- The article is well-written and well illustrated relying on both theoretical and numerical arguments. The paper is easy to read and the structure is clear. \n- Unlike most SGMs-related works that emphasize on learned approximations of the score function, this work focuses the empirical distribution and the exact closed-form score it implies. This perspective of research is interesting and provides other ways for analyzing memorization vs. generalization in diffusion models.\n- The authors acknowledged that their method could scale poorly when the number of training points is large, as the complexity is $\\mathcal{O}(N)$, but they provide solution to mitigate this effect. Their numerical experiments are detailed and convincing. Notably, $\\sigma$-CFDM is evaluated in terms of sampling quality (Sections 6.3 and 6.4) and compared to DDPM. Additionally, the study includes an analysis of its generalization properties (Section 6.1) and Theorem 5.1.\n\n$\\textbf{Weaknesses:}$\n\n- While focusing on ODE sampling can be clearer from an identifiability standpoint, SDE-based approaches often yield better sample quality in practice. Also I would be curious to know if you tried it and if you have reason to believe whether, the continuous noise, inherent to the discretization scheme of SDE such as Euler-Maruyama or Exponential Integrator schemes could work better. It is sometimes said that the noise injection compensate for the error in the discretization scheme on average but maybe they can promote generalization in this closed-form context.\n- The paper uses Gumbel noise in Proposition 5.3 to derive an analytical result for the one-step distribution. However, the rationale for noise distributions in actual experiments is unclear—are they Gaussian, Gumbel, or something else? Further discussion about how the choice of noise might impact smoothness or sample diversity would be valuable.\n- I believe that the discretization error is not clearly taken into account. For example when the authors say \"SGM memorizes its training data and does not generate novel samples\" I guess this is theoretical, as real implementations use a numerical solver with a finite step size. This can introduce unavoidable discretization error, which may indirectly help with generalization.\n- The paper states that the model samples converge towards barycenters of $M$-tuples of training points, which suggests that the parameter $M$ could be critical in controlling the interpolation behavior between training samples and the degree of generalization. However, I did not find a clear indication of how $M$ is set in the experiments, nor a discussion on its theoretical impact. Perhaps it would be beneficial to see an analysis of the sensitivity of the results to the choice of $M$.\n\n$\\textbf{Minor details:}$\n- I believe a factor 2 is missing below equation (25) when upper bounding the Lipchitz constant L.\n- In equation 23, the variable of integration is missing.\n- I believe an exponential is missing in the definition of $\\tilde \\beta(T)$ in equation (33).\n- Does the hypothesis in page 22 : $\\left\\| z - tx_i\\right\\| \\leq A$ for all $z$ and $tx_i$, imply a restriction to a compact set ?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces smoothed closed‐form diffusion models ($\\sigma$-CFDMs)—a training‐free approach to score-based generative models (SGMs). In particular, the article studies the impact of sampling with SGMs without training a deep neural network to approximate potentially complex and non-linear score functions $\\nabla \\log \\rho_t$ (following the article notations) associated with a sequence of a perturbed data distributions through Gaussian noise.  While neural approximation methods such as conditional score matching or sliced score matching have demonstrated strong numerical performance, they remain computationally expensive and, to some extent, opaque.\n\nOn the contrary, for any finite dataset, the exact score functions can be written in closed form when the training distribution is modeled as an empirical distribution (i.e. a sum of Dirac masses over the training dataset). This approach, called CFDM, provides an exact solution to the so-called, score matching problem.\n\nHowever, CFDMs lack the ability to promote generalization as $\\rho_t$ is merely a mixture of Gaussian random variables centered in the training data point with shrinking variance as $t\\to 1$. In the toy example Figure 1 (a) the authors illustrate that the algorithm only outputs copies of the training dataset when sampled with CFDMs. To address this limitation and promote generalization, the authors propose using a smoothed version of the empirical score function rather than the exact empirical score itself.  This is achieved by convolving the score function with centered random noise $\\epsilon$, scaled by a parameter $\\sigma > 0$ (i.e. in the Gaussian case $\\epsilon \\sim \\mathcal{N}(0,\\sigma^2 I_d))$. When, $\\rho_t$ is defined as the density of the same flow as in the article, it yields\n\\begin{align*}\n    s_{\\sigma,t} (z) & := \\mathbb{E} \\left[ \\nabla \\log \\rho_t(z + \\epsilon) \\right] = \\frac{1}{(1-t)^2}\\left( \\mathbb{E} \\left[  k_t(z + \\epsilon )\\right] - z \\right) \\,\n\\end{align*}\nThe remaining expectation over $\\epsilon$ is replaced by a Monte Carlo estimator over $M$ samples, yielding\n\\begin{align*}\n    s_{\\sigma,t}(z)  & =\\frac{1}{(1-t)^2} \\left( \\frac{1}{M} \\sum_{m=1}^M \\sum_{i=1}^N \\text{softmax} \\left( -\\frac{\\left\\| z - tX \\right\\|^2 -2 \\sigma t \\langle \\epsilon^{(m)}, x_i \\rangle }{2(1-t)^2}\\right)_i tx_i  - z  \\right) \\,\n\\end{align*}\n\nReplacing the score function $\\nabla \\log \\rho_t$ with $s_{\\sigma,t}$ and discretizing the associated velocity field yield a generative model called $\\sigma$-CFDM. The authors showed that in the infinite time limit the model output will converge towards barycenters of tuples of training points (Theorem 5.1). This could signal better generalization properties as $\\sigma$-CFDM sampler ends up interpolating among training points rather than reproducing them one-for-one. However, computing the smoothed score functions $s_{\\sigma,t}$ remain costly especially for large dataset. As a solutions the authors proposed to:\n- initialize samples directly from the un-smoothed CFDM at some $T>0$ and thus skipping the costlier evaluation of the low-$t$ regime for the smoothed scores.\n- replace the full sum over all training points by focusing on nearest neighbors of $z$ plus a small random subsample.\n\nFinally, the authors assess numerically the generalization quality for various $\\sigma$, computational trade‑offs associated with the fast sampling schemes and the applicability of $\\sigma$-CFDMS to image generation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It might be beneficial to clarify, address, or justify the points mentioned in the $\\textbf{Weaknesses}$ section above.", "broader_impact_concerns": "There is not any ethical concerns with the submission.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The discussion with the reviewers led the authors to improve some of the technical claims and, above all, to detail their experimental contributions in various directions by extending the appendices. They also added  some preliminary experimental results comparing the inductive bias of some neural SGMs and of their algorithms. The discussion on the relationship between other SGMs and the proposed approach was a problematic point in the initial version, the revision proposed additional experimental clarifications and has also expanded the various numerical perspectives offered by the methodology.

As stated by the authors, a very active area of research focuses on conditional generation (not only for images). In this perspective, many solutions have been proposed to combine pre-trained score functions with (Sequential or  Markov Chain) Monte Carlo methods to solve inverse problems or conditional sampling problems. A valuable perspective would be to compare the performance of the proposed algorithm, which does not require any training,  with pre-trained scores on posterior sampling tasks.
Decision: Accept (poster)