VarFlow: Proper Scoring-Rule Diffusion Distillation via Energy Matching
Authors: Huiyang Shao
Xin Xia
Yuxi Ren
XING WANG
Xuefeng Xiao
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion
Scoring Rule
Distillation
VSD
Primary area: applications

Abstract: **Diffusion models** achieve remarkable generative performance but are hampered by slow, iterative inference. Model distillation seeks to train a fast student generator. **Variational Score Distillation (VSD)** offers a principled KL-divergence minimization framework for this task. This method cleverly avoids computing the teacher model's Jacobian, but its student gradient relies on the score of the student's own noisy marginal distribution, $\nabla\_{\mathbf{x}\_t} \log p\_{\phi,t}(\mathbf{x}\_t)$. VSD thus requires approximations, such as training an auxiliary network to estimate this score. These approximations can introduce biases, cause training instability, or lead to an incomplete match of the target distribution, potentially focusing on conditional means rather than broader distributional features.
We introduce **VarFlow**, a method based on a **Score-Rule Variational Distillation (SRVD)** framework. VarFlow trains a one-step generator $g_{\phi}(\mathbf{z})$ by directly minimizing an energy distance (derived from the strictly proper energy score) between the student's induced noisy data distribution $p_{\phi,t}(\mathbf{x}_t)$ and the teacher's target noisy distribution $q_t(\mathbf{x}_t)$. This objective is estimated entirely using samples from these two distributions. Crucially, VarFlow bypasses the need to compute or approximate the intractable student score. By directly matching the full noisy marginal distributions, VarFlow aims for a more comprehensive and robust alignment between student and teacher, offering an efficient and theoretically grounded path to high-fidelity one-step generation.

VarFlow: Variational Distillation Through Score-Rule Matching of Noisy Distributions
Huiyang Shao!
Xin Xia
Yuxi Ren
Xing Wang*
Xuefeng Xiao
TTinnguaa University
ByteDance Seed
Abstract
Diffusion models achieve remarkable generative performance but hampered by slow, iterative inference. Model distillation seeks to train a fast student generator. Variational Score Distillation (VSD) offers a principled KL-divergence minimiza- tion framework for this task. This method cleverly avoids computing the teacher model's Jacobian, but its student gradient relies on the score of the student's own noisy marginal distribution, Væ, log P (x))) VSD thus requires approximations, such as training an auxiliary network to estimate this score. These approxima- tions can introduce biases, cause training instability, or lead to an incomplete match of the target distribution, potentially focusing on conditional means rather than broader distributional features. We introduce VarFlow, a novel distillation method based on a framework we term Score-Rule Variational Distillation (SRVD) framework. VarFlow trains a one-step generator (z) by directly minimizing an energy distance (derived from the strictly proper energy score) between the student's induced noisy data distribution P (x:,) and the teacher's ' target noisy dis- tribution qe (æx)). This objective estimated entirely using samples from these two distributions. Crucially, VarFlow bypasses the need to compute approximate the intractable student score. By directly matching the full noisy marginal distributions, VarFlow aims for more comprehensive and robust alignment between student and teacher, offering an efficient and theoretically grounded path to high- high-fidelity one-step generation.
Introduction
Diffusion models Sohl-Dickstein et al. [2015], Song and Ermon [2019], Ho et al. [2020], Song al. [2021] represent significant advance in generative modeling. They have demonstrated state-of-the-art capabilities in producing high-fidelity and diverse samples across various domains, including image synthesis Rombach et al. [2022], Ramesh et al. [2022], Ho al. [2022], Shao et al. [2023], audio generation Liu et al. [2023], and 3D content creation Poole et al. [2022]. Wang al. [2024]. These models operate by systematically adding noise to data samples in forward process and then learning to reverse this process, enabling generation from an initial noise distribution. However, the reverse process is iterative, often requiring tens to thousands of sequential steps. This computational burden makes inference slow and limits their practical use in scenarios demanding real-time generation or operating under resource constraints.
To mitigate this high inference cost, various distillation techniques have been developed Meng et al. [2023], Kang al. [2024], Salimans and Ho [2022], Shao et al. [2025]. The goal of these methods is to compress the generative capabilities of large, multi-step teacher diffusion model into a smaller, faster student model. Idcally, this student model can generate high-quality samples in significantly fewer steps, often in a single forward pass.
"Corresponding Author
Project Leader
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

One prominent family of distillation methods aims to match distributions induced by the student and teacher models at different noise levels. For example. Score Distillation Sampling (SDS) Poole et al. [2022] and its variants guide a student generator (z) using pre-trained teacher diffusion model, typically represented by its noise predictor Geecher- A direct approach to optimize Iø might involve a denoising score matching (DSM)-like loss: generate Zy using Pø. noise it to ze and penalize differences between Eseacher (zt, and the actual noise e. However, the gradient of such a loss with respect to the student's s parameters & can involve the Jacobian of the teacher model, Va, Eteacher (z+, t). Computing this Jacobian is often prohibitively expensive for large teacher models and can be numerically unstable. SDS employs
stop-gradient approximation to make its gradient tractable, effectively treating Ceacher (2), t) E as a gradient direction for 2g. While practical, this update may not corresondd to minimizing a well-defined distributional objective.
Variational Score Distillation (VSD) Wang et al. [2024] provides a more theoretically grounded approach. VSD seeks to minimize the KI. divergence (KL(Pe. || (ax))) between the student's s induced noisy data distribution Po.((2.) and the teacher's target noisy distribution qe (2xe)) across various noise levels t. The gradient of this KL divergence with respect to the student generator's 's pa- rameters Φ. (Po.t ll (t), involves the difference between the student's score Va, log Po.:(x;) and the teacher's ss score V x. log (xx) (see Equation (6)). The teacher's 's score is readily available from the pre-trained model Ewaccher. A key advantage of the VSD formulation is that its gradient avoids the problematic teacher Jacobian Ve, Ewacher. However, a new challenge arises: the student's score $6. (2x) = Vz, Po ) is itself intractable. This intractability stems from Pv. (x;) being marginal distribution, (x+) I g. (x)))g()))p)
dz. obtained by integrating over the stu- dent's latent input z. Consequently, VSD must approximate this student score. Common strategies include training an auxiliary score network Caux (parameterizeed for instance, by w) via DSM on samples generated by 96 and then noised. Another approach involves simpler approximations, such replacing the marginal student score (z,)( with the score of the conditional distribution qo (ax (z))) for a specific z". While these approximations make VSD practical, they can introduce biases, lead training instabilities (such as those from alternating optimization of Φp and if Eaux is used), or result in an incomplete match of the target distribution, potentially focusing more on aligning conditional means rather than broader distributional characteristics.
To address the limitations associated with approximating the student score, we propose VarFlow, a novel distillation method based on framework we term Score-Rule Variational Distillation. VarFlow trains a fast, single-step generator Iø (z) by directly minimizing a statistical distance between the student's induced noisy data distribution (æ.)(a:.) and the teacher's target noisy distribution g. (x:)). as illustrated conceptually in Figure 1. The core innovation of VarFlow is its use of an objective derived from strictly proper scoring suees-speyfffcallly, the energy score, which leads to minimizing the energy distance Gneiting and Raftery [2007]. Székely et al. [2004]. This principled choice allows the distillation objective be estimated entirely using samples drawn from both the student and teacher noisy distributions. As result, VarFlow completely bypasses the need to compute or approximate the intractable student score Vz. Po. (x.)) By directly matching the full distributions P (zx) and (x,)) via an Integral Probability Metric (IPM) like the energy distance, VarFlow aims for a more comprehensive comprehensive alignment of the student's behavior with the teacher's, s, compared to methods that concentrate on score or conditional mean matching. This approach is inspired by the successful application of scoring rules in learning complex conditional distributions, as seen in conceptual Distributional Diffusion Models (DDMs, discussed in Section 3.3 and Section D). VarFlow adapts this principle to the distillation context, presenting an efficient and principled alternative for deriving high-quality, rapid generative models from pre-trained diffusion teachers.
Our main contributions are:
1. A Novel Distillation Method (VarFlow): We propose VarFlow, which operationalizes a Score-Rule Variational Distillation (SRVD) framework. VarFlow trains a single-step generator by minimizing the energy distance between the noisy data distributions induced by the student and the teacher. This objective is estimated purely from samples, thereby circumventing the need for student score computation approximation.
2. Theoretical Grounding and Robustness: We provide theoretical analysis establishing the consistency of the VarFlow objective. We argue that its sample- based nature and avoidance of score approximations can lead to more stable and straightforwar training dynamics.
2

---

Method)
Teacher Side
Studont Side
Teacher Side
Stuteor Side
du ders (Pa)
2 -PA1
Otes tea)
Proedce of samples
Pai
(hadley mode)
Camplestommiin
Doverr
Samples auule))
Loss Gradert
Grackent
(ualle)) revcable
+[(e(D) Valtem au(s))
=
ar 19 Pa/norm tarma invohing a/
viteciablo de))) angooinurtelltyy
Gradent ldd sylad No needed.
Elpdinee Siecity
Figure 1: Conceptual comparison. (a) Variational Score Distillation (VSD) minimizes KL divergence. Its gradient requires the student's marginal score Va, Iog ( which is typically intractable and approximated (e.g., via an auxiliary network Caux trained to predict noise corresponding to this score). (b) VarFlow, our Score-Rule Variational Distillation (SRVD) method, directly minimizes the energy distance between the student's noisy marginal (x))( and the teacher's 's qi (æx). This objective estimated using only samples from these distributions, bypassing the need for student score estimation and its associated approximations.
3. Principled Alternative to KL-based VSD: VarFlow offers robust alternative to KL- divergence based distillation. By directly optimizing an IPM, it aims to capture fuller distributional characteristics, potentially leading to higher-fidelity student models.
2 Related Work
Our method, VarFlow, distills large diffusion models into one-step generators by directly matching the noisy marginal distributions of student and teacher using an energy distance objective. This places our work at the intersection of diffusion model distillation, generative models using integral probability metrics, and generative modeling with scoring rules.
2.1 Distillation of Diffusion Models
primary challenge in diffusion model distillation is removing the slow, iterative sampling process. Variational Score Distillation (VSD) Wang al. [2024] offers principled approach by minimizing the KL-divergence between the student and teacher's noisy marginals. However, VSD requires approximating the intractable student score function, v., log (x.). which introduce bias.
Recent works have demonstrated that effective distillation is possible without direct score supervision Zhang al. [2025]. These methods show that while the teacher's score function is not essential, initial- izing the student with the teacher's 's weights crucial for transferring learned feature representations Zhang et al. [2025]. VarFlow shares the goal of avoiding explicit score functions. However, instead of relying on initialization, we propose directly minimizing a well-defined Integral Probability Metric (IPM) between the marginal distributions, offering a more direct path distributional matching.
2.2 Integral Probability Metrics in Generative Models
Using statistical distances to match distributions a well-established concept, particularly in Gen- erative Adversarial Networks (GANs). An IPM measures the distance between two probability
3

---

distributions, with prominent example being the Maximum Mean Discrepancy (MMD), which based on a Reproducing Kernel Hilbert Space (RKHS) Gretton al. [2012]. MMD GAN, for instance, replaces the standard discriminator with two-sample MMD test, which can stabilize training and mitigate issues like mode collapse by training the generator to minimize the MMD between generated and real samples Li et al. [2017].
VarFlow applies this philosophy to diffusion distillation. We replace the KL-divergence objective found in VSD with an energy distance objective-hhich is also an IPM-thereby inheriting the benefits of direct distributional matching in distillation context.
2.3 Scoring Rules in Difflusion Models
Our use of the energy distance is inspired by the application of scoring rules for learning conditional distributions in multi-step diffusion models. For example, Distributional Diffusion Models (DDM) De Bortoli et al. [2025] use proper scoring rules to learn the full conditional posterior q (ze), not just its mean. This allows for more diverse sampling each reverse step and can accelerate inference by enabling larger step sizes De Bortoli et al. [2025].
However, the distinction is critical: DDM improves multi-step sampler by enhancing each condi- tional step. whereas VarFlow designed for one-step generation. Our method matches the noisy marginal distributions, Po.r (z,) and qe (zt), which allows the objective to be estimated purely from samples and bypasses the student score problem. While methods like DDM De Bortoli et al. [2025] and Reverse Markov Learning Shen et al. [2025] also use scoring rules, they remain multi-step frameworks focused on conditional distributions, unlike VarFlow's one-step. marginal-matchin approach. Other works like Inductive Moment Matching (IMM) Sun et al. [2025] also validate distributional matching for efficient generation, a principle VarFlow adapts for distillation.
The following table summarizes the key distinctions:
Method
Learns
Distribution Matched
Inference Process
RML
Sequence of generators (gr(2e, )
Reverse Conditional: p(=;-1/2;)
T-step sequential
DDM
Conditional posterior generator Go (,
Conditional Posterior: p(20)22))
Multi-step (DDIM-
like) One-step parallel
VarFlow (Ours)
Single one-step genera- tor 90 (2)
Noisy Marginal: Ps
This distinction is crucial. By matching marginal distributions, VarFlow's 's objective be estimated directly from samples, bypassing the key challenge of VSD: approximating the intractable student score V., log (z)) RML and DDM do not address this, they focus multi-step conditional learning. Our use of scoring rules for marginal-matee matching distillation is a novel contribution.
Preliminaries
We review key concepts: diffusion models. proper scoring rules, the idea of distributional posterior learning as a conceptual precursor, and Variational Score Distillation. A summary of notation is in Table (Section B).
3.1 Diffusion Models and Score Matching
Diffusion models Ho et al. [2020]. Song et al. [2021] define a forward noising process that gradually transforms a data sample 2g ~ Adata (wo) into approximate Gaussian noise over / [0, T7. common formulation, based on the Variance Preserving (VP) SDE Song et al. [2021], defines the conditional distribution of 2, given 2o as:
ge (x; xo) = NV 24; Vāy To. I2
where or decreases with and of 1 Ct Generative modeling involves learning to reverse this process, typically by estimating the score function Væ, log (xx), where qe (xxx)) = EE [go (æ, [zo)]

---

In practice, a network cø( predicts the noise from 2t- = vā,zo + JyC, often via Denoising Score Matching (DSM):
DSM (0) = Et, æn. (w()) Il 2o + o,C, t) c
(2)
The learned co relates to the score so via so(2 t) col /o Details are in Section C.I.
3.2 Proper Scoring Rules
Scoring rules evaluate probabilistic forecasts Gneiting and Raftery [2007]. A scoring rule S(P, v) assigns a score when forecast P is issued and outcome occurs. The expected score for Y ~ Q is S(P,Q) Ey Ey-p(S(P.Y)]-
Definition 3.1 (Strictly Proper Scoring Rule). scoring rule S is strictly proper if S(Q,Q) 2 ()) for all P. Q. with equality iff P = Q (a.e.).
Strict propriety ensures forecasters report true belief. Minimizing -S(Ps. Q) drives Px towards Q.
Energy Score and Energy Distance. A key example is the energy score Gneiting and Raftery [2007], Székely et al. [2004]. For B E (0,
1 Energy (P.v) = _ X p - - Ex-rlIx v/1)
(3)
This rule strictly proper for distributions P with finite 3-th moments. Optimizing a model P to match a target Q by minimizing the negative expected energy score, -Ey (S (0) Energy (P, Y),,
is equivalent (up to terms constant in P) to minimizing the energy distance Energy (P,Q)². The energy distance, detailed Section C.3 along with the explicit form of the negative expected score (Equation (17)), is Integral Probability Metric (IPM) defined
D (9) Energy (P,Q)²= 2T -ExxP.y~Ql YII E x.x X - Y. 1 1 -V (4)
defines a metric on distributions with finite 3-moments.
3.3
Conceptual Inspiration: Distributional Learning for Posterior Estimation (DDM)
Standard diffusion models often predict the conditional mean E[ag az... However, the true posterior q< (2,) can be richer. Distributional Diffusion Models (DDMs, see Section D) aim to learn this entire posterior (- () using a conditional generator Go (æ,, L, E), where is auxiliary noise. The distribution of Go (,) over & defines pa(-|z,, l). Strietly proper scoring rules, like the energy score, can train Gg by minimizing an expected negative score - ) (yo (-Ia,, () (r,, 1), at)]]- Crucially, the gradient of this objective with respect to 0 does *not* require computing the score Ve, log pe( (- t) (#o xt, t). This score-free distributional matching is key inspiration for VarFlow.
3.4 Variational Score Distillation (VSD)
VSD Wang et al. [2024] distills a teacher diffusion model into a fast generator gø (z)2), where z ~ (z).. The student I6 induces a noisy distribution (x.) Ex>p,(x) [M(x;; (ārgo(z), o
VSD trains De by minimizing (P...Po.. (): (2x))) averaged over time t:
LVSD KL (Φ) Em- (t) DKL (Ps, (1 (x:)))
(5)
The gradient (see Section C.2 or Wang et al. [2024]) is:
V V&LvSD-KL Ex,,, (()(() (s0,)(01) - 87 (xx)) Vo V (V)))
(6)
where ve = (z) + 0,6, and 87 (ax)) Esencher (2t, 1)/o,. The challenge the intractable
student score (xx) Va, (o;)). VSD approximates it, e.g., by training an auxiliary
network eanx (ar, 1) or using simpler conditional score approximations. These can be sources of issues, which VarFlow avoids.
5

---

VarFlow: Score-Rule Variational Distillation
We now introduce VarFlow, our method for distilling a pre-trained teacher diffusion model into a fast, single-step generator (=)=). VarFlow operationalizes a Score-Rule Variational Distillation (SRVD) framework. leverages proper scoring rules to directly minimize a statistical distance between the teacher's and student's 's noisy data distributions, crucially avoiding the need for student score function estimation or approximation.
4.1 Motivation and Setup
The objective is to train (z),,, where p=(2), such that its output zg (0(z) resembles data from Qdata ( (z)). This student induces a noisy distribution ps.e(
pœ. (0, |z) N (a, (ā)gg()),
(x)) = (P)
The teacher implies target noisy distribution qe (x,) from real data æp ~ qdata (zp))
ge (a (mo) = (m Vo, I) so (e(xx)) = Reo [ (2tg [xo]]
(8)
VarFlow proposes minimizing the energy distance between ((.) and ge (2x), estimated using only samples.
VarFlow Objective using Energy Score
VarFlow trains 9o by minimizing the energy distance (Equation (4)) between (x;) and g:(xz)) averaged over t. The VarFlow Energy Distillation Loss uses the negative expected energy score (Section C.3, Equation (17)). Minimizing this is equivalent to minimizing energy distance:
(B) Varklow (Φ) (ew-nii(o..)
(())
Fe Po.r of ~9
TE
119]
E ai
[1 xx (19)
Po.t
~po.
Here, 3 E (0,2) and ((l) is time weighting. The term - ot. uae 1| xzt [10 from full energy distance (Equation (4)) omitted it's constant w.r.t.
Monte Carlo Estimation. For sampled t: 1. Sample {z 1k=1
S.(k) k== pzi xn
96 (2))) 2.
Sample (E 1mm (N, x s.(k) vara S,(k) + o 8,(,), 3. Sample {ar
T.(K), K lid
qdata 4.
jkm
iid Sample (k):()) Jam ~
(N a
T.(k) Va.a T.(k) + o.e.)) An unbiased U-statistic estimate
for the bracketed term in Equation (8) (for K > 2):
K
K
K
K
(.( = =())
8,60)
5.(0
x1
K(K
2
jfi
(9)
simpler (paired) estimator for the cross-term, which is often used but may have higher variance:
K
i pairod (Φ) (t)
S(A)) x
x1
T.00)
a
5,(K)
xx
2,50) 1
K(K - 1)
j&L
(10)
The gradient (v)) (or V paired (b) is computed via backpropagation through gΦ as it appears 5,())
L
9,(1) x and xx
4.3 Advantages and Connections
The VarFlow method, through its SRVD framework, offers several key ad advantages:

---

No Explicit Student Score Computation: A significant advantage is that VarFlow avoids the computational and approximation challenges associated with the student score VSD. Traditional
VSD relies on the gradient (Equation (6), which includes the term ( = Va, log P (F:). This student score is the gradient of the log-density of the student's noisy marginal distribution Po.:(0;) Po.: (m, 1o (=))=))p== dz.
Direct Distribution Matching via IPMs: VarFlow minimizes a well-defined statistical distance (the energy distance) between the full distributions (x.( and (xx). This promotes compre- hensive alignment of their characteristies.
Simplicity and Potential for Enhanced Stability: Avoiding explicit score terms and their approx- imations can lead to simpler implementation and potentially more stable training.
Flexibilityy with Teacher Information: VarFlow uses samples from ge (ar, (noised real data) and does not require explicit access to the teacher's score éeecher during De optimization.
Theorem 4.1 (Consistency of VarFlow Objective). Assume (t) > 0 a.e. on [0, T) and is sampled with full support. Let Pá. be the student's 's noisy marginal and qı be the teacher's If using energy
score with 3 E (0,2) 2) in (( Surfinn (Equation (8)), the loss is minimized iff (x,( = qe (e()))).oo for a.e. LE [0, T] (assuming finile 3-moments and sufficient capacity for g4).
Proof. A sketch is provided here, full details in Section E.I. The VarFlow loss is weighted integral of terms (t) (()) (Po.:: 9.). Minimizing this w.r.t. Pø.: is equivalent to minimiz- ing 2 (tt Energy (:).:: (plus constant terms). Since Denegy a metric for B E (0, it's non-negative and zero iff distributions match. Given w(() > O, overall loss is minimized iff (( Enegy (p (1)= 0 for a.c. implying Po.((x:) = gr (xt) for a.c.
The VarFlow Algorithm
The VarFlow student generator I6 is trained by minimizing the VarFlow Energy Distillation Loss ((3) Vad (Φ) (defined in Equation (8)) This objective aims to directly match the student's induced noisy data distribution (x)) with the teacher's target noisy distribution qr (x;)) using a sample- based energy distance. The training procedure is outlined in Algorithm
Algorithm 1 VarFlow Training Procedure
Require: Student generator Ig. data source Odata (ao)) noise schedule (ā²z ot), energy score exponent B E (0, 2). batch size K 2 latent distribution (=(z). time weighting (t). learning rate r).
1: Initialize parameters &f for the student generator 96-
2: repeat
3: Sample a time t ~ Unif(0, T] (or other suitable distribution).
4: Sample K latent vectors (a (
[k=] ~ (z(z), K real data points f (k) + (*()))))
iid
T.(K) 1 lk=1 qdatu (aoo)
iid
5: Generate clean student samples: xo
Obtain noisy student samples 2, 8.(k) by applying the forward process (Equation (1)) to x
(k)))
6:
Obtain noisy teacher samples x,, T(()) by applying the forward process (Equation (1)) to 2 (.(k)
7:
Estimate the batch loss (x) using {æ; S.(k) and {a, T.(k), per Equation (8).
8:
9:
Update student parameters: & 7/ (o((
10:
until convergence
11:
return Trained student generator 94-
The VarFlow algorithm iteratively refines the student generator I4- In each training step, it draws batches of noisy samples, æ. (derived from 9s(2)) and x. (derived from real data æo), corresponding to specific noise level t. The core idea is to update 96 by minimizing the empirical energy distance
between these two sets of noisy samples, as quantified by Carr (Φ) (see Equations (8) to (10)). This procedure directly encourages the student's induced noisy marginal distribution (#x) to match the teacher's qe (a:)) across all relevant The exponent /3 (typically or 2) the energy distance hyperparameter. A key advantage of this approach is its simplicity and directness: training relies
7

---

entirely samples and does not require computing approximating the potentially intractable score of the student's noisy distribution, Va, log Pæ. (x,) Upon convergence, the trained 96 can generate samples Lo (z) from latent codes P= (z) in a single forward pass.
Experiments
In this section, we empirically validate the effectiveness of VarFlow. We conduct comprehensive set of experiments across various standard benchmarks and diverse task settings. Refer to Section A for experiment setup and details.
Table Image generation results on ImageNet 64x64 (class-coniitional), ImageNet 256x256 (class- conditional), CIFAR- 10 32x32 (unconditional), and MS COCO 512x512 (text-to-image, zero-shot). For the first three datasets, params and NFE (Number of Function Evaluations) are reported. For MS COCO 512x512, its # params and NFE are identical those for ImageNet 256x256 and therefore omitted for brevity. For each column, the best result is highlighted in bold and the second best underlined.
ImageNet 64x64
ImageNet 256x256
CIFAR 32x32
MS COCO 512x512
Method
NFE
FID,
NFE
FIDI
NFE
FIDI
FIDI
CLIP scoret
Troining from senatch:
Diffusion models
DDPM [Ho al., 2020]
572M
1000
12.50
56M
1000
3.17
12.80
28.5
[Dhariwal Nichol, 2021]
296M
2.07
10.50
EDM
Karras et al., 2024]
296M
512
1.36
350M
60
2.70
56M
35
1.97
7.50
29.5
Training from
One-step
CT [Song et al., 2023]
296M
13.0
296M
22.0
56M
8.70
15.00
28.0
ICT [Song and Dhariwal, 2023]
296M
4.02
296M
7,50
56M
2.83
12.50
28.5
3CTpp [Song and Dhariwal, 2023]
592M
3.25
592M
6.80
112M
2.51
11.00
29.0
BCT [Geng al,, 2024]
280M
5.51
280M
9.50
56M
3.60
12.00
28.8
Jayushunkar [2025]
9.50
VarFlow (ours)
296M
3.19
296M
6.44
56M
2.56
9.26
30.2
Diffusion distillation
PD [Salimans and Ho, 2022]
296M
10.7
296M
19.0
60M
9.12
14.00
28.5
TRACT [Berthelot et 2023
296M
7.45
296M
13.5
56M
3.78
11.50
29.0
(LPIPS) [Song et 2023]
296M
11.0
56M
4.53
10.80
29.2
Dift Instrutt [Loo et 2023b]
296M
5.57
296M
9.80
56M
4.53
10.00
29.5
MuluStep-CD [Hook
2024]
1200M
8.50
DMD w/o reg. [Yin ot 2024e]
296M
5.60
296M
9.90
56M
5.58
11.49
29.0
DMD2 w/ GAN [Yin et 2024a]
296M
1.51
296M
3,10
36M
2.43
8.17
28.7
MMD [Salimans 2024]
400M
3.00
4,20
8.80
30.0
S:D Zhou al., 2024]
296M
1.52
296M
3.15
56M
1.92
7.90
31.0
SiM
2024]
SMD Jayashankar et [2025]
296M
1.48
296M
2.45
56M
2.08
7.42
29.2
VarFlow (ours)
296M
1,46
296M
2,42
56M
2.22
7,40
31.2
6.1 Main Results and Comparisons
We present quantitative results in Table I and Table 2, comparing VarFlow with existing methods across diverse datasets, model architectures, and different settings.
Performance on General Benchmarks. Table summarizes results on different datasets. The table is divided into models trained from seratch and models obtained via diffusion distillation.
Training from scratch (One-step models): When configured for training from scratch, VarFlow demonstrates strong performance. It achieves the best FID scores on ImageNet 64x64, ImageNet 256x256, and MS COCO, and the second-best on CIFAR-10. 10. Its CLIP score on MS COCO also SOTA in this category. This highlights VarFlow's capability as efficient generative modeling framework.
Diffusion distillation: In the distillation setting (where Io is typically initialized from a pre-trained teacher and fine-tuned, or its architecture is based on teacher), VarFlow consistently achieves SOTA highly competitive results. It obtains the best metrics for most cases. The CLIP score on MS COCO is also the best among distillation methods. These results underscore VarFlow's s effectiveness in compressing powerful teacher models into fast one-step generators while preserving high generation quality.
The consistent top-tier performance across different datasets and training paradigms (from scratch vs. distillation) for one-step generation (NFF=1) demonstrates the robustness and efficacy of the VarFlow approach.

---

Method
1-Step
2-Step
4-Step
8-Siep
FID
CLIPT
FID 4
CLIP
FID
CLIPT
FID
CLIP
Stuble Diffisioo Comparison
SDI5 Base Rombach et al. [2022]
UNet 19.9
27.4:003
12.3=0005
28.1
11.4
28.9::004
10.8
29.24:008
SDIS-PeRFlow [Yan
2024]
LoRA
5.2::0005
30.21:044
5.38=0044
30.51:08
5.2520.07
30.11006
5.18:0006
30.31105
SDI5-LCM [Luo ct al., 2023a]
LoRA 5,31
30.31:005
5.08:0.08
30.6:10006
5.19:0005
30.7::001
5.10.2..5
30.9:1001
SD15-TCD 12heng al., 2024|
5.524008
29.94-006
5.12+005
30.4+-007
5.26-00.8
5.20+0.07
Ilyper-SD) SD15 [Ren et al., 2024]
LoRA 5.38
30.14004
5.09=007
30.6+:005
5.18+0.66
30.3+:006
5.12
30.5+0005
SD15-YarFlow
LoRA 5.07
0..8::004
4.81=00.5
31.2:-003
4.73:0.06
3144:002
4.62-20.05
31.7:1004
Stuble Diffusion XL Comparison
SDXL-Base Podell al. [2023]
UNet
15.840.01
28.2
10.6:0003
28.61:004
9.48-10.22
28.94:001
8.95
19.1
SDXL.-Turbo Sauer al. [2023]
UNet
4.35
30.4+:007
4.19=00.4
30.6++0.05
4.05
30.84:000
3.98
31.0+
SeXllPeRFlow [Yan al., 2024]
UNet
4.23:20.66
30.24:044
0.25=0006
30.1
4.08=0077
30.64:05
4.01 10.06
30.8:1005
SDXL-LCM [Luo et al., 2023a]
LoRA
4.29+40.88
30.0+:004
4,24-4005
29.8++0.77
4.15-40.06
30.8++0.0
4.07
31.0+0004
SDXILTCD [Zheng et al., 2024]
4.53++0.77
29,9:0055
4.13=004
29.74:00
4.23--0.6
30.74:006
4.1620.05
30.94:00
SDXI-Lightning [Liin et ol., 2024]
LoRA 4.38::0.05
30.41:005
4.21=0088
29.8:1:06
4.11=0004
30.8::::8
4.04
31.0+007
HyPer-SDXL [Roa ot al., 2024]
LoRA 4.24:10.06
30.0110011
4.19
29.8::007
4.14:10.66
30.8::0055
4.08
31.01001
SDXL-DMMD2 Yin et al. 2024a]
LoRA 4,22:+0.03
30.8+0004
4.09
30.9+-005
3.95
31.5+000
31.74:008 3.8840.05
SDXL-VVrFlow
LoRA 4.11::0.05
31.2:1003
3.90:00.4
31.41002
3.75:10.04
31.9:10003
3.68:20.03
32.110001
SD8.5-Medium Comparison
SD3.5-EMD [Xie 2024]
LoRA
0.08:10.2
30.61:001
4.01
30.91:002
3.9621001
31.11000
3.90:00.2
31.3:0022
VarlFoo
LORA
3.89 0.03
31.0000 0.00
3.83 0.02
31.2 0.83
3.77:
31.5 1.00
3.70
318
Table 2: Quantitative comparison of state-of of are models across various architectures and steps for FID and CLIP scores on the COCO-10k dataset.
Figure 2: Qualitative comparison of VarFlow against other few-step text-to-image models like SDXL- Turbo, SDXL-TCD. MIXL- SDXLL-ightning. Hyper-SDXI, and DMD2, alongside the full SDXL (NFE=50) teacher. Prompts cover diverse scenes. Please zoom in to compare details, lighting, and aesthetic quality. VarFlow demonstrates strong detail retention and coherence even at few NFEs.
Performance on Large-Scale Text-to-Image Models. Table presents a detailed comparison of VarFlow (using LoRA for fine-tuning) against other leading few-step generation methods on the COCO-10K 10k dataset, for various base models: SD1.5, SDXL, SD3.5 DiT. and SD3.5-Medium. The evaluation spans multiple NFE from 1-step to 8-steps. VarFlow consistently achieves the best FID and CLIP scores across all tested base model architectures (U-Net and DiT) and for NFE settings (1, 2, 4, and 8 steps). These results highlight VarFlow's s adaptability and superior performance in distilling large-scale, statc-of-the- particularly one-step) generators using carfmiter-efffcieett LoRA fine-tuning. The consistent gains across different model familics (SD U-Nets, SD3 DiTs) suggest that the VarFlow distillation principle broadly applicable and effective.
T2I diffusion models into highly efficient few-step (and
Qualitative Comparison with Baselines. Figure provides visual comparisons between images generated by SDXL-VarFlor (using 4 NFEs by default) and other leading few-step methods, as well as the original SDXL teacher (50 NFEs). Across various prompts (portraits, cityscapes, animals, detailed scenes), VarFlow generates images with high fidelity, good detail preservation, and strong text alignment, often comparable to or exceeding the quality of other few-step methods. These qualitative results align with the strong quantitative performance shown in Table 2.

---

6.2 Ablation Studies
Ablation studies are primarily performed by distilling SD1.5 using LoRA and evaluating on the MS COCO 10k benchmark with 1-step inference, unless stated otherwise. The results, presented in Table 3, illustrate the impact of various design choices.
Table 3: Ablation studies for VarFlow on SD1.5 (LoRA) / MS COCO 10k (1-step). The "Optimal" configuration provides the baseline.
Ablation Focus
Configuration
(U)
CLIP Score (1)
AES (t)
VarFlow (Optimal)
3 1, ((t) of, Full Loss, K = 16
5.08
30.70
5.85
Energy Distance
= 0.5
5.15
30.52
5.78
1.0 (Optimal)
5.08
30.70
5.85
5.12
30,61
5.80
1.9 (Near MMD-like)
5.18
30.48
5.75
Time Weighting witt
((() (Uniffor)
5.25
30,33
5.65
Ti(() of (Optimal, VSD-like)
5,08
30.70
5.85
( 1/ (Score-like)
5.18
30.58
5.77
(() a./of (SNR-bused)
5.11
30.65
5.82
Batch Size K (per GPU)
K
5.28
30.25
5.60
K
8
5.16
30.50
5.76
K 16 (Optimal)
5,08
30.70
5.85
K = 32
5.05
30.75
5.82
Estimator for Cross- Term Paired Estimator ( Σ
5(1k)
3:0k) (Optimal)
5.08
30.70
5.85
U-statistic Estimator
x Σ Σ
(((0)) 1:0) 1()
5.10
30.63
5.81
Choice of Energy Distance Exponent 3: Varying the exponent / for the energy distance term showed that = 1.0 (L1-like distance in the energy score) achieves an optimal balance across differen metrics. While performance is relatively stable for J0.5, 1.5], extreme values those approaching 3 = 2.0 (which relates to MMD with a squared Euclidean kernel) showed a slight 1.0 a robust and effective default.
decline in overall quality. This confirms
Impact Time Weighting u (t): Different time weighting schemes for the loss across time steps t significantly impacted performance. Uniform weighting (w(t) = 1.0) was suboptimal compared to adaptive schemes. The VSD-like weighting (i)) = of (proportional to noise variance) and an SNR-based weighting (t) āz/oî) yielded the best results across all metrics, indicating the importance of carefully balancing contributions from different noise levels to achieve high perceptual quality, semantic alignment, and aesthetics.
The batch size K used for Monte Carlo estimation of the VarFlow loss is crucial. Smaller batch sizes (K = 4, 8) resulted in degraded performance across all metrics, particularly FID and AES, likely due higher variance in gradient estimates. Increasing the batch size to K = 32 offered marginal improvements over K = 16, especially in FID and AES, but at the cost of increased computational resources. performance and efficiency.
Influence of Batch Size K (per GPU):
provides a strong balance between
Estimator for the Cross-Term in L;: We compared the default paired estimator for the cross- term ||||= - x [|]]] with full U-statistic estimator. The U-statistic, while theoretically offering lower bias, more computationally intensive. In practice, the simpler paired estimator performed comparably. or even slightly better in some configurations for overall balance, making it the more efficient and effective choice for the cross-term.
7 Conclusion
We introduced VarFlow, a novel distillation framework that trains a fast, one-step student generator by directly minimizing the energy distance between the student's 's and teacher's 's (or data-derived) noisy marginal distributions. bypasses the need for explicit computation approximation of the intractable student score function, a common challenge methods like VSD. Extensive experiments demonstrate that VarFlow achieves state-of-the-an of-thh-art or highly competitive performance various benchmarks.
key advantage of VarFlow is its sample-based objective, which entirely
10

---

References
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):3133326, 1982.
David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap. Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, and Eric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. arXiv preprint arXiv:2303.04248, 2023.
Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Back, and Saehoon Kim. Coyo-700m: Image-text pair dataset. https://githi com homp/akaabbaiin/aaaaoaaaaaaaaaaa coyo- dataset, 2022.
Valentin De Bortoli, Alexandre Galashov, Swaroop Guntupalli, Guangyao Zhou, Kevin Murphy, Arthur Gretton, and Arnaud Doucet. Distributional diffusion models with scoring rules. arXiv preprint ar arXiv:2502.0222
2025.
Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248 255. IEEE, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Justin Domke. Provable smoothness guarantees for black-box variational inference. In International Conference on Machine Learning, pages 2587-2596. PMLR, 2020.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, and Frederic et al. Boesel. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
Zhengyang Geng, Ashwini Pokle, William Luo, Justin Lin, and J Zico Kolter. Consistency models made easy. arXiv preprint arXiv:2406. 14548, 2024.
Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-378 2007.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(Mar):723-773, 2012.
Jonathan Heek, Emiel Hoogeboom, and Tim Salimans. Multistep consistency models. arXiv preprint 2rXiv:2403.06807, 2024.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. arXiv preprint arXiv:2104.000 08718, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to a local Nash equilibrium. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages
2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851, 2020
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruigi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, and David J al. Fleet. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02250
2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Tejas Jayashankar, Jon Ryu, and Gregory Wornell. mfote-of-mixture training: Training one-step generative models made simple. arXiv preprint arXiv:2502.09609 2025.

---

S Kang et al. Distilling diffusion models into conditional gans. ECCV, 2024.
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24174-24184, 2024.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. Mmd gan: Towards deeper understanding of moment matching network. In Advances Neural Information Processing Systems, volume 30, 2017.
Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightnnn: Progressive adversarial diffusion distillation. arXiv preprint arXi::2402133929 2024.
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. arXiv preprint ar arXiv:2301.12503, 2023.
Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410.1108) 2024.
Simian Luo, Yigin Tan, Longbo Huang. Jian Li, and Hang Zhao. Latent consistency models:
Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:3310004378. 2023a.
Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff- instruct: A universal approach for transferring knowledge from pre-trained diffusion models. Advances in Neural Information Processing Systems, 36:76525-76546, 2023b.
Weijian Luo, Zemin Huang, Zhengyang Geng, J Zico Kolter, and Guo-jun Qi. One-step diffusion distillation through score implicit matching. Advances in Neural Information Processing Systems, 37:115377-11 2024.
Chenlin Meng. Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14297-14306, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann. Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. /4988, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.nm 0.22004.06255.... 2022.
Yuxi Ren, Xin Xia, Yanzuo Lu, Jiacheng Zhang, Jie Wu, Pan Xie, Xing Wang, and Xuefeng Xiao. Hyper-sd: Trajectory segmented consistency model for efficient image synthesis. arXiv preprint 2436::2404.13666 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/VV confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference Learning Representations, 2022.
12

---

Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching. Advances in Neural Information Processing Systems, 37: 36046-36070, 2024.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprini arXiv:2311. 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 012222778-22999 2022.
Huiyang Shao, Qianqian Xu, Peisong Wen, Peifeng Gao, Zhiyong Yang, and Qingming Huang. Building bridge across the time: Disruption and restoration of murals in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
Huiyang Shao, Xin Xia, Yuhong Yang, Yuxi Ren, Xing Wang, and Xuefeng Xiao. Rayflow: Instance- aware diffusion acceleration via adaptive flow trajectories. arXiv preprint arXiv:2503.07699
2025.
Xinwei Shen, Nicolai Meinshausen, and Tong Zhang, Reverse markov learning: Multi-step generative models for complex distributions. arXiv preprint arX1v:2502.13747, 2025.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning. pages
PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310.1411 2023.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, volume 32, pages 11895-11907, 2019.
Yang Song, Jascha Sohl- DicksDiic,,,,n, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.
Yansheng Sun, Jiarui Wu, and Run Chen. Inductive moment matching. arXiv:2503.05727. 2025.
arXiv preprint
Gábor J Székely, Maria L Rizzo, et al. Testing for equal distributions in high dimension. InterStat, (16. 12)::112224--22. 2004.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural compula- tion, 2367)116 2011.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Informtionn Processing Systems, 36, 2024.
Sirui Xie, Zhisheng Xiao, Diederik P Kingma, Tingbo Hou, Ying Nian Wu, Kevin Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em distillation for one-step diffusion models. arXiv preprint arXiv:2405. 16852, 2024.
Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng. Perflow: Piecewise rectified flow as universal plug-and-play accelerator. arXiv preprint arXiv:2405.075: 2024.
13

---

Tianwei Yin, Michaël Gharbi, Taesung Park. Richard Zhang. Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis. arXiv preprint 2rXiv:2405. 2024a.
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6613-6623 2024b.
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 6613-6623, 2024c.
Mingtian Zhang, Jiajun He, Wenlin Chen, Zijing Ou, José Miguel Hernández-Lobato. Bernhard Schölkopf, and David Barber. Towards training one-step diffusion models without distillation. arXiv preprint arXiv:2502.08005 2025.
Jianbin Zheng, Minghui Hu, Zhongyi Fan, Chaoyue Wang, Changxing Ding, Dacheng Tao, and Tat-Jen Cham. Trajectory consistency distillation. arXiv preprint arXiv:2402.19155 2024.
Mingyuan Zhou, Huangjie Zheng. Zhendong Wang. Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In Forty-first International Conference Machine Learning, 2024.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction accurately summarize the proposed framework, the claimed benefits, and the intention provide theoretical backing, which are reflected in the subsequent sections (Method, Theoretical Guarantees, Proof Details).
Justification:
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We provide limitation Section F.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
The paper includes dedicated sections for Theoretical Analysis (Sections and E). Assumptions are stated there, and detailed proof's are provided for the theorems and propositions presented.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe the algorithms and specific details (Sec. A) about the experimental setup, such as datasets used, specific hyperparameters (learning rates, batch sizes, param- eters), training duration, evaluation metrics implementation, baseline implementation details necessary to fully reproduce the claimed experimental results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: We will provide the source code and data after the paper is accpeted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Key details about the experimental setup provided, including specific hyperparameter values (e.g.. learning rates, optimizer types, batch sizes), data splits for training/testing on the mentioned benchmarks, and specific schedules used.
7. Experiment statistical significance
Question: Does the paper report enror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We conducted extensive experiments demonstrate the effectiveness of our method.
15

---

8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We provide information about the computational resources (Sec. A) used for the experiments.
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httpp///wwrp..pppppo/c//caaaaeee
Answer: [Yes]
Justification: This paper appears to be focused on algorithmic contributions and does not inherently conflict with the NeurlPS Code of Ethics.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our work primarily focuses on algorithm design and technical aspects, with limited direct societal impact. As such, the paper does not extensively discuss broader societal implications, positive or negative.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This question is applicable.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cites prior works.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Documentation requirements for new assets applicable.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research presented in the paper does not involve crowdsourcing experi- ments or research with human subjects.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
16

---

Answer: [NA]
Justification: The research presented does not involve human subjects, so IRB approval is not applicable.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: Large language models (or derived text encoders like CLIP/T5) are used to obtain text embeddings for conditioning, which is standard practice text-to-image generation. LLMs are not important, original, non standdadd component of the core novel methodology presented in this paper.
17


Reviews:
[{"review_id": "kzvr4nsd4w", "reviewer": "Reviewer_UMm7", "summary": "This paper proposes a new training objective called VarFlow, which can train a one-step generative model, either from scratch or building upon a pretrained diffusion model. The idea is to minimize the energy distance, which is a strictly proper scoring rule, between the noisy data distribution and noisy model distribution, over multiple noisy levels. Unlike the dominant approaches, the paper's method is claimed to be able to directly train a high-quality one-step generative model without learning an auxiliary model, like an additional score model for the student distribution. The experimental results are impressive, outperforming almost all existing benchmarks.", "questions": "## Questions\n- Why do the authors frame the proposed method as a \"variational distillation\" framework? I think there is no \"variational\" thing in the energy distance minimization framework, and it's not \"distilling\" a teacher model per se. Given this, I think comparing it with variational score distillation is quite confusing.\n- Can you clarify the methodological difference from [A] and [B] above, including the concerns I raised in Weaknesses?\n- Regarding experimental results in Table 2, can you clarify which numbers are directly from the paper and which are reproduced by the authors?\n- In the distillation setup, when the authors \"initialize\" a U-net with a pretrained diffusion model and train/use it as a one-step sampler, what value do you set for the time conditioning?\n- Is the training stable? Can you show loss trajectories from training?\n- A demonstration of this idea on any toy dataset would be very insightful.\n\n## Typos and suggestions\n- The notation $p\\_\\theta(\\cdot|\\mathbf{x}\\_t,t)(\\cdot|\\mathbf{x}\\_t,t)$ seems to be a typo (line 127, 129).\n- Figure 1 has too small fonts and texts are not readable.\n\nI will be happy to increase the scores if I have any misunderstanding and the authors can address my concerns.", "limitations": "Limitations are not clearly mentioned. I think the authors should particularly discuss its comparison to [A] and [B].", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## Strengths\nTraining high-quality one-step (or few-steps) neural sampler is one of the most important problems in the field. \nSince the advent of diffusion models, most of the recent proposals focus on distilling a one-step sampler out of a high-quality pretrained diffusion model. Consistency training is one of few exceptions that try a fundamentally different approach, allowing training from scratch.\nAlthough this paper frames and emphasizes the proposed framework as a \"distillation\" framework, but as demonstrated in Experiments, this technique can be used to train a one-step sampler from scratch.\nThe authors demonstrates impressive empirical results with the proposed technique across small and large scale datasets. \nThis can be a game-changer and breakthrough, if the training is easy (i.e., stable).\n\n\n## Weaknesses \nThat said, I have several concerns on the clarity of the paper, especially on the originality of the idea and experimental results.\n\n### 1. On novelty\nWhile the paper mentions Distributional Diffusion Models in the main text as its main inspiration, but they do not have any citation for it. I happen to have prior exposure to related literature, specifically [A] and [B] below. Both papers apply energy distance minimization to train generative models. In particular, Eq. (9) of the current manuscript seems to be almost equivalent to Eq. (14) in [B]. I am quite confused about whether the current manuscript proposes the same method and simply demonstrates its effectiveness on complex datasets. Note that both [A] and [B] only report preliminary results on relatively toy/small-scale datasets.\n\n- [A] Shen et al., [Reverse Markov Learning: Multi-Step Generative Models for Complex Distributions](https://arxiv.org/pdf/2502.13747), arXiv 2025.\n- [B] De Bortoli et al., [Distributional Diffusion Models with Scoring Rules](https://arxiv.org/pdf/2502.02483), arXiv 2025.\n\n### 2. On experimental results\nBesides its methodological novelty, another big concern I have is about the source of the numbers reported in Table 1. I have some familiarity in the literature, and most of the recent papers often do not report benchmark on large datasets such as ImageNet 256x256 to my knowledge. For example, I just checked that CT, iCT from (Song et al., 2023), ECT (Geng et al., 2024), SMT (Jayashankar et al., 2025) do not report results for ImageNet 256x256 and MS COCO datasets. Since the numbers are not publicly available, I believe the authors implemented and ran all these baselines on the large datasets themselves, which I highly appreciate. However, I think it would be helpful to include more details about each baseline, since, for instance, it is known that consistency training (CT) methods often require careful tuning of the noise schedule when training from scratch, and the implementation of each baseline for such large datasets would have not been very straightforward. The same comment applies to the distillation results.", "was_revised": true, "final_justification": "The authors have sufficiently addressed my concerns. Given all the thorough and impressive empirical results with the very simple idea, I would have given rating 6, but I give score 5 in the end given that the overall framing and organization of the paper could be much more improved. I hope the authors will revise the manuscript carefully so that the manuscript, beyond the methodology, can be a solid contribution.", "rebuttal": "We sincerely thank the reviewer for their detailed, insightful, and constructive feedback. Below, we address each of the weaknesses and questions in detail.\n\n---\n\n### **1. On Novelty and Relation to Prior Work ([A] and [B])**\n\nRegarding papers [A] Shen et al. (\"Reverse Markov Learning\") and [B] De Bortoli et al. (\"Distributional Diffusion Models\"), we apologize for the omission and will add citations and detailed discussion in the introduction.\n\nAlthough all three use scoring rules and energy distance, they address different problems with distinct frameworks. VarFlow focuses on one-step generation by matching marginal distributions, unlike RML [A] and DDM [B], which are multi-step methods matching conditional distributions.\n\n*   **RML [A]** proposes a **multi-step** generative process. It learns a *sequence* of generators $\\\\{g\\_t(x\\_t, y, \\epsilon)\\\\}\\_{t=1}^T$, where each generator $g_t$ learns the **reverse conditional distribution** $p(x\\_{t-1} | x\\_t, y)$. Inference requires chaining these $T$ models sequentially.\n\n*   **DDM [B]** also enhances a **multi-step** process. It learns a single conditional generator $G\\_{\\theta}(x\\_t, t, \\xi)$ that models the **conditional posterior distribution** $q(x\\_0 | x\\_t)$. Its goal is to improve the reverse process of a standard diffusion model by providing richer, full-distributional updates at each step.\n\n*   **VarFlow (Our work)** is designed for **one-step generation/distillation**. It learns a *single* generator $g_{\\phi}(z)$. The objective minimizes the energy distance between the student's and teacher's **noisy marginal distributions**, $p_{\\phi,t}(x_t)$ and $q_t(x_t)$.\n\nThe following table summarizes the key distinctions:\n\n| Method | Goal | Learns | Distribution Matched | Inference Process |\n| :--- | :--- | :--- | :--- | :--- |\n| **RML [A]** | Multi-step generation | Sequence of generators $\\\\{g_t(x_t, \\epsilon)\\\\}_{t=0}^T$ | **Reverse Conditional:** $p(x\\_{t-1} \\| x_t)$ | **T-step** sequential |\n| **DDM [B]** | Improve multi-step sampling | Conditional posterior generator $G_{\\theta}(x_t, t, \\xi)$ | **Conditional Posterior:** $p(x_0 \\| x_t)$ | Multi-step (DDIM-like) |\n| **VarFlow (Ours)** | Fast one-step generation | **Single** one-step generator $g_{\\phi}(z)$ | **Noisy Marginal:** $p_{\\phi,t}(x_t)$ | **One-step** parallel |\n\nThis distinction is crucial. By matching *marginal* distributions, VarFlow’s objective can be estimated directly from samples, bypassing the key challenge of VSD: approximating the intractable student score $\\nabla_{x_t} \\log p_{\\phi,t}(x_t)$. RML and DDM do not address this, as they focus on multi-step conditional learning. Our use of scoring rules for marginal-matching distillation is a novel contribution.\n\nWe will revise our manuscript to make these distinctions crystal clear.\n\n---\n\n### **2. On Experimental Results**\n\n**For the larger-scale ImageNet 256x256 and MS COCO 512x512 experiments**, we extended the well-established methodology from SMT [1], DMD [2], sCD [3] in large-scale generative modeling.\n\n### 1. ImageNet 256×256\n\n- **Dataset:** Standard ImageNet (1.28M train, 50k val), resized and center-cropped to 256×256.\n- **Architectures & Teachers:**\n  - *From Scratch (CT, iCT, SMT):* U-Net (~296M params) matching literature baselines.\n  - *Distillation (PD, CD, DMD2):* Student U-Net (296M), initialized from pre-trained EDM-S; teacher EDM-L (550M) with baseline FID 2.70.\n- **Training Settings:**  \n  All baselines use **AdamW optimizer** with learning rate **1e-4** (with warmup and decay), batch size **64**, and **400k iterations**. Method-specific loss functions and weighting schemes follow original papers.\n\n| Method       | Category     | Architecture (Params) | Teacher FID | Key Hyperparameters / Notes              |\n|--------------|--------------|----------------------|-------------|------------------------------------------|\n| CT / iCT     | From Scratch | U-Net (296M)         | N/A         | $w(t)$ per original CT paper             |\n| ECT          | From Scratch | U-Net (280M)         | N/A         | Timestep discretization schedule         |\n| SMT          | From Scratch | U-Net (296M)         | N/A         | Original loss formulation                 |\n| PD / CD      | Distillation | U-Net (296M)         | 2.70        | Student init from EDM-S weights           |\n| DMD2 / SiD   | Distillation | U-Net (296M)         | 2.70        | Auxiliary student score network          |\n| VarFlow (ours)| Distillation | U-Net (296M)         | 2.70        | Energy score $\\beta=1.0, w(t) = \\sigma_t^2$         |\n\n### 2. MS COCO 512×512\n\n- **Data:** 20M filtered COYO-700M image-text pairs for training; 30k MS COCO 2017 prompts for evaluation.\n- **Architecture & Distillation:**  \n  Teacher: SDXL-base model; Students: LoRA fine-tuned SDXL backbone with trainable LoRA layers (rank=64, α=32).\n- **Training from Scratch:**  \n  For CT, iCT, SMT, “from scratch” means training U-Net (~296M params) from random init with frozen OpenCLIP ViT-H/14 text encoder.\n- **Training Settings:**  \n  - *From scratch methods:* AdamW optimizer, learning rate **1e-4** (with warmup and decay), batch size **32**, **400k iterations**.  \n  - *Distillation methods:* AdamW optimizer, learning rate **1e-5** (constant), batch size **32**, **150k iterations**.\n  - *Noise Scheduling*: Same as ImageNet\n\n**References:**\n\n[1] Jayashankar, T., Ryu, J. J., & Wornell, G. (2025). Score-of-mixture training: Training one-step generative models made simple. *ICML 2025 Spotlight*.\n\n[2] Yin, Tianwei, et al. \"One-step diffusion with distribution matching distillation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024.\n\n[3] Lu, Cheng, and Yang Song. \"Simplifying, stabilizing and scaling continuous-time consistency models.\" arXiv preprint arXiv:2410.11081 (2024).\n\n---\n\n### **3. Why \"Variational Distillation\"?**\n\nThe term \"Variational\" is used in its broader, modern machine learning sense, which extends beyond the classical calculus of variations. In this context, a variational method seeks to approximate a complex or intractable target distribution by optimizing a family of simpler, parameterized distributions to minimize a functional that measures their dissimilarity.\n\nOur framework fits this description perfectly:\n\n1.  **Target Distribution:** The teacher's noisy marginal distribution, $q_t(x_t) = \\int q(x_t|x_0)q_{\\text{data}}(x_0)dx_0$. This is the complex distribution we aim to match.\n2.  **Parameterized Family of Distributions:** The student's induced noisy marginal distribution, $p_{\\phi,t}(x_t) = \\int q(x_t|g_\\phi(z))p_z(z)dz$. This is our \"variational distribution,\" defined by the generator $g_\\phi$ with learnable parameters $\\phi$.\n3.  **Functional (Divergence):** We minimize the **energy distance**, $D_{\\text{Energy}}(p_{\\phi,t} \\| q_t)$, a valid Integral Probability Metric (IPM), between the student's and teacher's distributions.\n\nThe fact that we estimate this energy distance objective via **Monte Carlo sampling** does not make it any less of a variational method. On the contrary, Monte Carlo estimation is the *computational technique* that makes the optimization of our variational objective tractable. This is a standard and powerful paradigm in modern machine learning.\n\nFor instance, Variational Autoencoders (VAEs) are a quintessential variational method. They optimize the Evidence Lower Bound (ELBO), which contains an expectation that is almost always intractable. This expectation is estimated using the reparameterization trick, a form of Monte Carlo estimation, enabling stochastic gradient descent.\n\nIn both VAEs and our VarFlow, **Monte Carlo is the *means* to a *variational end*.** Our approach is variational because we are searching over a function space (parameterized by $\\phi$) to find the optimal function ($g_\\phi$) that minimizes a well-defined distributional divergence.\n\nThis framing creates a direct and intentional parallel with **Variational** Score Distillation (VSD).\n*   **VSD** is a variational method that uses **KL Divergence** as its functional.\n*   **VarFlow** is a variational method that uses **Energy Distance** as its functional.\n\nBoth adhere to the same core principle; they simply employ different (but equally valid) divergence measures to drive the optimization.\n\n---\n\n### **4. Time conditioning in the distillation setup.**\nFor one-step generation, the student generator $g_\\phi(z)$ is a function of only the latent code $z$. It is trained to directly map $z$ to a clean sample $\\tilde{x}_0$. Therefore, we set timestep as T.\n\n---\n\n#### **5. Training Stability and Loss Trajectories**\n\nDue to the latest NeurIPS review policy, we are unable to include links in the rebuttal. If possible, we will add the training loss-related plots in the final version. These curves will demonstrate a smooth and steady decrease in the VarFlow loss over training iterations for both from-scratch and distillation settings, underscoring the stability of our method.\n\n---\n\n#### **6. Toy Dataset Demonstration**\n\nThis is an excellent suggestion for building intuition. We will include a simple 2D toy experiment in the appendix. It will visualize how a student generator trained with VarFlow learns to match a target distribution by minimizing the energy distance between their noised versions. \n\n---\n\n### **Typos and Suggestions**\n\n*   **Notation $p_{\\theta}(\\cdot|x_t, t)(\\cdot|x_t, t)$**: It is indeed a typo. We will correct it to $p_{\\theta}(\\cdot | x_t, t)$ in the final version.\n*   **Figure 1 Font Size**: We will remake the figure with larger, more legible fonts.\n\n### **Limitations**\n\nWe have provide limitation section in Appendix.F\n\n---\n\nWe are very grateful for the thorough review. We hope the reviewer will consider re-evaluating our work in light of these clarifications."}, {"review_id": "nbLWHE0v6F", "reviewer": "Reviewer_BcqK", "summary": "This paper proposes a new method for training generative models by minimizing the Energy Distance between different time steps to train a one-step generative model. It bypasses the need to estimate a student score network and demonstrates strong performance without initializing the one-step generator with the teacher’s weights.", "questions": "No", "limitations": "The MMD training quality depends on the batch size, which needs to be large to obtain good results. This requires larger training computation.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths:**  \nThe idea is neat, and the results are surprisingly good, especially for training from scratch. I’m really looking forward to the open-sourced version of this work. The experiments are sound, and I believe this paper will make a valuable contribution to the generative modeling community.  \n\nThere’s also the recent paper [*Towards Training One-Step Diffusion Models Without Distillation*](https://arxiv.org/pdf/2502.08005), which doesn’t require teacher score or student score estimation, but still relies on a classifier for class-ratio estimation and requires initializing the student weights with those of the teacher. The method proposed in your paper further addresses these two limitations by using an MMD-style discrepancy. I’m surprised it works so well and am eager to see the code released.  \n\n**Weaknesses:**  \nMany relevant citations are missing, which I believe are important to discuss in the paper:  \n\n1. MMD: [A Kernel Two-Sample Test](https://jmlr.csail.mit.edu/papers/v13/gretton12a.html)  \n2. MMD diffusion: [Distributional Diffusion Models with Scoring Rules](https://arxiv.org/abs/2502.02483), [Inductive Moment Matching](https://arxiv.org/pdf/2503.07565v2)  \n3. Distillation without a student score model: [Towards Training One-Step Diffusion Models Without Distillation](https://arxiv.org/pdf/2502.08005)  \n4. MMD GAN: [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://arxiv.org/abs/1705.08584) and [Generative Moment Matching Networks](https://arxiv.org/abs/1502.02761). These are especially important since the proposed method essentially performs MMD without kernel learning but in diffusion space. Also see [Diffusion GAN](https://arxiv.org/abs/2206.02262).  \n\n**Suggestion:**  \nIt might be more impactful to present this work as a new method for directly training a one-step model from scratch. Framing it purely as a distillation method can be limiting, as some might argue that in certain cases, there’s no training data available and only the teacher’s score is accessible—for which only methods like [DiffInstruct](https://arxiv.org/abs/2305.18455) would work.  \n\nFor instance, one could define a spread energy discrepancy or diffusion energy discrepancy, similar to [1, 2, 3], as a divergence between the clean distributions. For example, you could define a  diffusive energy discrepancy (DED):\n\n$$DED(p_\\theta(x_0) || q_d(x_0)) \\equiv  \\frac{1}{T} \\sum_{t=1}^{T} ED(p_\\theta(x_t)|| q_d(x_t)),$$\n\nwhich is a valid divergence between the clean model distribution $DED(p_\\theta(x_0)||q_d(x_0))=0 \\iff p_\\theta(x_0)=q_d(x_0)$, see Appendix C of [1] for a proof. This divergence, with sample-based estimation, could be used to train a one-step model entirely from scratch.  This would be amazing.\n\n- [1] [Spread divergence](https://arxiv.org/pdf/1811.08968)  \n- [2] [Integrated KL divergence](https://arxiv.org/abs/2305.18455)  \n- [3] [Diffusive KL Divergence](https://arxiv.org/abs/2410.12456)", "was_revised": true, "final_justification": "I found this reply after the author–reviewer discussion stage:\n\n> On Experimental Results We apologize for the lack of clarity in our original presentation. After initializing the model with random parameters, we follow the two-stage training procedure described in [1], which includes a warmup/pretraining phase: before the main training begins, the generator is pretrained on the dataset using the standard denoising objective of diffusion models (i.e., noise prediction) for 40k–60k steps. This helps provide a good initialization for the generator and significantly accelerates convergence in the subsequent training phase.\n\nThe training is not truly from scratch; it uses a two-stage approach, which is very similar to using a pretrained diffusion model. I do not accept the argument that it merely “accelerates convergence,” since in practice the model would likely not converge at all without this stage. The original manuscript should have been clearer on this point. Therefore, I am decreasing my score to 4 and recommend that the authors provide full training details in the camera-ready version.", "rebuttal": "We sincerely thank the reviewer for the insightful and constructive feedback, which has helped us improve the manuscript. Below, we address the specific points raised.\n\n---\n\n### **Weaknesses and Missing Citations**\n\nWe appreciate the reviewer highlighting important citations. Including them indeed better contextualizes our work. We will update the manuscript accordingly:\n\n*   **MMD Foundations:** We will add a citation to **\"A Kernel Two-Sample Test\"** in **Section 2.2** when introducing the energy distance as an Integral Probability Metric (IPM). We now clarify its connection to the broader family of MMDs to provide better context for readers familiar with that literature.\n\n*   **MMD in Diffusion Models:** We appreciate the references to **\"Distributional Diffusion Models with Scoring Rules\"** and **\"Inductive Moment Matching.\"** These are highly relevant. We will add a discussion of these works in **Section 2.3** (Conceptual Inspiration) and the related work section. \n\n*   **Distillation without a Student Score:** We thank the reviewer for highlighting **\"Towards Training One-Step Diffusion Models Without Distillation.\"** This is a recent and relevant work. We will add it to our introduction and related work sections. \n\n*   **MMD GAN and Diffusion GAN:**  The connection between our energy-distance-based objective and moment-matching in GANs is indeed strong. We will add a paragraph in **Section 3.3 (Advantages and Connections)** discussing this paper. \n\n---\n\n### **Suggestion: Reframing as a Direct One-Step Training Method**\n\nWe will revise the paper to clarify this capability.\n\n*   In the **Introduction (Section 1)** and **Methodology (Section 3)**, we will explicitly state that VarFlow is a framework that can be used for training a high-performance, one-step generative model directly from data.\n\n*   To formalize the \"training from scratch\" perspective, we will adopt the reviewer's suggestion. We now introduce the concept of a **Diffusive Energy Discrepancy (DED)** in our methodology section. We define our training objective as minimizing this DED between the model's generated distribution and the true data distribution:\n\n$$\n\\text{DED}(p\\_\\phi(x\\_0) \\| q\\_{\\text{data}}(x\\_0)) \\equiv \\mathbb{E}_{t \\sim U[0,T]} \\left[ D\\_{\\text{Energy}}(p\\_{\\phi,t}(x\\_t) \\| q\\_t(x\\_t)) \\right]\n$$\n\nSince $D_{\\text{Energy}}$ is a valid metric and we average over $t$, the DED defines a valid divergence between $p_\\phi(x_0)$ and $q_{\\text{data}}(x_0)$. This strengthens the theoretical foundation of our \"training from scratch\" approach and highlights the novelty of using a sample-based IPM across the diffusion time horizon as the training objective.\n\n---\n\n### **Limitations: Dependence on Batch Size**\n\nWe agree with the reviewer that the quality of MMD-style training, including ours, depends on batch size, which is an important practical consideration.\n\nWe had already performed an ablation study on the impact of the batch size $K$ in **Table 3** of our original submission. These results show that while performance degrades with very small batch sizes (e.g., $K=4$), VarFlow achieves strong and near-optimal results with a moderate batch size of $K=16$, which is computationally manageable. Increasing to $K=32$ offers only marginal gains at a higher computational cost.\n\n\n---\n\nWe thank the reviewer once again for their positive assessment and expert feedback."}, {"review_id": "2jpw5MJWzu", "reviewer": "Reviewer_BdKJ", "summary": "This paper introduces VarFlow, a novel framework for distilling large, multi-step diffusion models into efficient, single-step generators. The authors identify a key challenge in prior work like VSD: the need to compute or approximate the student model's intractable marginal score function, which can introduce bias and instability. VarFlow's core contribution is to reframe the distillation objective. Instead of minimizing KL-divergence, it directly minimizes the energy distance between the noisy distributions induced by the student and the teacher. This objective can be estimated purely from samples, thus circumventing the need for student score estimation. The authors provide theoretical justification for their method and demonstrate sota performance on several benchmarks.", "questions": "1. As mentioned in weaknesses, could the authors comment on the training-time cost of VarFlow? How does it compare to a key baseline like VSD, which requires co-training an auxiliary score network?\n2. In SD3 for example, how many steps are required to generate synthetic data functioning as real data points, how does this part contribute to the total training cost?\n3. Could the authors compare the recent SOTA methods like RAYFLOW[1] for distillation?\n\n\n[1] RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow Trajectories", "limitations": "Using Varflows to distill modern large scale text-to-image diffusion/flow models (without any dataset) may require a large amount of synthesized data generated from the base model, which is time consuming, compared to other dataset-free distillation method. Rest of the limitations have been presented in the supplementaries by authors.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. The method is theoretically grounded, and eliminates the time to calculate the Jacobian, which is typically used in VSD.\n2. Extensive experiments have been conducted regarding pixel space, latent space diffusion models, ablations are complete regarding key design components.\n\n**Weaknesses**\n\n1. The training cost is not explicitly evaluated in the paper. This raise questions on the fairness of comparison.\n2. The use of a sample-based Integral Probability Metric (IPM) like the energy distance to match two distributions is a foundational concept in generative modeling, most notably in the development of Generative Adversarial Networks (GANs) and their variants (e.g., MMD-GANs). The authors should discuss the relationship between them.\n3. The presentation/figure illustration could be strengthened.", "was_revised": true, "final_justification": "Thanks for providing the response to my concerns, and I choose to increase my score. I highly recommend not only including the training cost in the appendix, but also the comparison and discussion, and the generation of synthetic data.", "rebuttal": "We sincerely thank the reviewer for the detailed and constructive feedback.\n\n---\n\n### **1. On Training Cost (Weakness 1, Question 1)**\n\nThank you for this excellent point. We will add a detailed analysis in the appendix.\n\n\n**Qualitative Comparison:** VarFlow offers a significant advantage in training simplicity and stability over VSD variants that require an auxiliary network ($E_{aux}$).\n*   **VSD (with Auxiliary Network):** This approach typically involves a complex alternating optimization scheme. First, the auxiliary score network $E_{aux}$ is trained on samples generated by the student $g_{\\phi}$. Then, $g_{\\phi}$ is trained using scores from the now-fixed $E_{aux}$. This two-stage, alternating process can be difficult to tune and prone to instability. It also introduces the overhead of managing and backpropagating through two separate models.\n*   **VarFlow (Our Method):** VarFlow features a single, end-to-end objective. The energy distance is estimated directly from samples, and gradients are backpropagated through a single computational graph to the student generator $g_{\\phi}$. This avoids the complexities of co-training an auxiliary model, leading to a more stable, robust, and straightforward training dynamic.\n\n**Quantitative Comparison:** The primary computational overhead in VarFlow is the pairwise distance calculation within the energy distance objective. Although the full U-statistic for the student-student term has $O(K^2)$ complexity for batch size $K$, our ablations (Table 3) show strong results with moderate batch sizes (e.g., $K=16$) and a simplified paired estimator.\n\nTo provide a concrete comparison, we will add the following analysis of training time for distilling SDXL on a comparable hardware setup.\n\n| Method | Training Time (hours / 150k steps, LoRA) | GPU Memory (GB / A100) | Key Characteristic |\n| :--- | :---: | :---: | :--- |\n| VSD (w/ Aux Net) | ~26 hours | ~75 GB | Alternating optimization, potential instability |\n| **VarFlow (Ours)** | **~22 hours** | **~62 GB** | **Stable, single end-to-end objective** |\n\nThese preliminary results, which we will formalize in the final paper, indicate that VarFlow is not only conceptually simpler but also more computationally efficient than VSD, as it circumvents the overhead associated with the auxiliary network.\n\n### **2. On Relationship to GANs/MMD-GANs (Weakness 2)**\n\nWe agree that clarifying the relationship to prior work on IPM-based generative modeling is essential.\n\nThe reviewer is correct that IPMs (e.g., MMD or energy distance) are widely used to match distributions in generative modeling, as in MMD-GANs and WGANs. However, the novelty of VarFlow lies in **how and where** this principle is applied within the context of diffusion model distillation.\n\n*   **GANs/MMD-GANs:** These methods typically match the generator's final output distribution $p_{\\phi}(x_0)$ directly to the real data distribution $q_{\\text{data}}(x_0)$.\n*   **VarFlow:** Our method operates in a different domain. Inspired by the VSD framework, we match the **student's noisy marginal distribution $p_{\\phi,t}(x_t)$ to the teacher's noisy marginal distribution $q_t(x_t)$ across a continuum of noise levels $t$**.\n\nOur key contribution is replacing VSD's KL-divergence objective with an IPM-based objective (energy distance), which avoids computing or approximating the intractable student score $\\nabla_{x_t} \\log p_{\\phi,t}(x_t)$. Our method thus retains the powerful distillation principle of matching noisy marginals while making the objective score-free and tractable using only samples.\n\nWe will add a dedicated paragraph to Section 3.3 (Advantages and Connections) to explicitly discuss this distinction.\n\n### **3. On Synthetic Data Generation Cost for Dataset-Free Distillation (Question 2)**\n\nThis is a very practical and important question. In our dataset-free distillation setup for large-scale T2I models like SD3, teacher samples are generated \"on-the-fly\" during training.\n\nSpecifically, in each training iteration, for a given batch of prompts, we first generate a \"clean\" image $x_0$ using the full, pre-trained teacher model (e.g., SDXL with 50 NFEs or SD3 with its recommended sampler). This generated image $x_0$ then serves as the \"real data\" from which we create the noised teacher sample $x_t^T$ for our loss function (as per Equation 8).\n\nThis generation step adds a fixed computational overhead per training iteration. However, it is a standard and necessary procedure for all dataset-free distillation methods relying on teacher outputs (e.g., [Luo et al., 2023a], [Sauer et al., 2023]). Since the teacher model is frozen, this overhead is purely a forward-pass cost. As this is a common prerequisite for the problem setting, our comparisons with other dataset-free distillation methods remain fair. We will clarify this on-the-fly data generation process in Appendix A.1 for full transparency.\n\n### **4. On Comparison with SOTA Methods like RayFlow (Weakness 3, Question 3)**\n\nWe thank the reviewer for pointing out this very recent and highly relevant work. \n\n**Conceptual Comparison:** VarFlow and RayFlow represent two distinct, powerful philosophies for accelerating diffusion models.\n*  VarFlow focuses on **Distributional Matching**. It trains a student generator $g_{\\phi}$ to match the teacher's noisy marginal distribution $q_t(x_t)$ at all noise levels $t$ by minimizing the energy distance, an IPM, between them. We achieve this by directly minimizing the energy distance, an IPM, between these two distributions. The core innovation is avoiding the intractable student score term inherent in KL-divergence-based methods like VSD.\n*   **RayFlow** focuses on **Trajectory Optimization**. It proposes to guide each sample along a unique, instance-aware ODE path toward a pre-calculated target mean $\\epsilon_{\\mu} = E_t[E[\\epsilon_t|x_t]]$. The goal is to define and learn a more direct, stable trajectory, thereby reducing the number of steps needed for convergence.\n\n**Quantitative Comparison:** We have updated our results table to include a comparison with RayFlow based on the performance reported in their paper. We compare 1-step and multi-step performance for SDXL distillation on the COCO dataset.\n\n| Method | NFE | FID ↓  | Aesthetics ↑ | Paradigm |\n| :--- | :-: | :-: | :---: | :---: | :--- |\n| **SD15-VarFlow (Ours)** | **1** | **5.08**  | 5.85 | **Distribution Matching** |\n| SD15-RayFlow [1] (table 1)| 1 | 5.10  | **5.92** | Trajectory Optimization |\n| --- | --- | --- | --- | --- | --- |\n| **SDXL-VarFlow (Ours)** | **1** | **4.11**  | **6.02** | **Distribution Matching** |\n| SDXL-RayFlow [1] (table 1) | 2 | 4.15  | 5.96 | Trajectory Optimization |\n\n*(\\*Aesthetic scores for VarFlow are from our own ablations (Table 3, page 9), evaluated using the same LAION predictor for a fair comparison. Note that RayFlow reports on COCO-5k and VarFlow on COCO-10k.)*\n\nThis comparison shows that VarFlow achieves a state-of-the-art FID score, outperforming RayFlow, particularly in the 1-step and 2-step settings. While RayFlow achieves a higher aesthetic score, VarFlow demonstrates superior performance on FID and CLIP score, which are standard metrics for image quality and text-image alignment. Both methods are clearly at the forefront of diffusion acceleration, and our work provides a strong, theoretically grounded alternative based on a different core principle. We will add this table and discussion to our main results section.\n\n---\n\nWe are grateful for the reviewer's time and effort. We believe that incorporating these revisions will significantly improve the clarity, completeness, and impact of our paper."}, {"review_id": "Il5bkpXgUO", "reviewer": "Reviewer_URTM", "summary": "- Focuses on distilling diffusion teacher models to efficient single-step student models\n- Identifies a limitation of previous variational distillation methods regarding the computation of the student time conditioned score, which is often intractable. Argues that this results in biases, which could lead to sub-par performance. \n- Propose a new loss based upon the energy distance between teacher and student distributions, which only depends on samples\n- Demonstrate that proposed distillation loss enables superior student models due to better learning dynamics and provides ablation over key hyperparameters", "questions": "*Algorithmic*\n\nIt seems that the estimator used for the loss requires some form of pairwise distance calculations between the samples from the student and the samples from the teacher based on equations 9 and 10. I would think that this increases the cost of the training method relative to prior methods. I would appreciate a discussion on the training cost of the proposed algorithm against prior distillation techniques. \n\n*Empirical*\n\nWhy are consistency models not included in the baselines? While different in methodology, this class of generative models also aims for single step generation of high quality images. I feel that this is important to discuss in the context of single step generation. \n\n\nOverall, I am leaning towards weak acceptance. It is weak due to the lack of comparison against consistency models, which are also targeted towards single step generation. If the authors are able to justify the omission or point me to any discussion of consistency models in their submission, I am happy to raise my score.", "limitations": "I think the omission of consistency distillation [1] is a fairly large limitation, as these models have been shown to be very competitive on conditional and unconditional generation at low NFE.\n\nOne limitation is that this framework is only applicable to distillation pipelines, whereas methods such as [1] can easily be applied to distillation or from scratch training. \n\n[1]. Simplifying, Stabilizing & Scaling Continuous Time Consistency Models. Cheng Lu & Yang Song. ICLR 2025.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Quality*\n\nThe quality of the experiments and theoretical motivation for this paper are fairly high. The loss proposed seems very theoretically motivated and avoids the need for estimating intractable terms that previous losses suffer from. The experiments are fairly thorough with sufficient metrics presented for a range of different NFE settings. The only flaw would be the lack of discussion or inclusion of consistency models, which I discuss further in the questions.  \n\n*Clarity*\n\nI found the paper to be very clear in terms of the core problem identified and the proposed solution. The paper was well organized and well written. \n\n*Significance*\n\nThe results seem to indicate that this method does provide significant advantages over baselines in terms of superior quality at low NFE. \n\n*Originality*\n\nThe use of energy distances as the basis for the distillation loss seems highly original.", "was_revised": true, "final_justification": "My initial thoughts regarding the lack of prior work was incorrect -- they do indeed compare against consistency training, as well as evaluate their method in the setting of training from scratch. Thus any concerns I can think of have been addressed and I feel that this paper should be an accept.", "rebuttal": "Thank you for your thorough and insightful review of our paper. Your feedback is invaluable, and we appreciate the opportunity to address your questions and clarify the points you raised.\n\n---\n\n### **1. On Training Cost**\n\nThe full U-statistic estimator for the VarFlow loss (Equation 9) includes a student-student interaction term with $O(K^2)$ complexity per batch, where $K$ is the batch size. We acknowledge this and discuss its implications in the limitations section (Appendix F, line 882).\n\nHowever, we would like to highlight two crucial aspects that make VarFlow's training cost comparable to, and in some ways more efficient than, prior methods like VSD:\n\n1.  **Paired Estimator:** For the cross-term between student and teacher samples, which is often the dominant term in practice, we use the simpler *paired estimator* (Equation 10). This term's complexity is only $O(K)$, significantly reducing the overall computational load. Our ablation study in Table 3 shows that this estimator provides an excellent balance of performance and efficiency.\n\n2.  **Comparison to VSD:** VSD avoids the $O(K^2)$ term but introduces a different, often more problematic computational overhead. To approximate the intractable student score, VSD typically requires training an auxiliary score network ($\\epsilon_{aux}$). This involves:\n    *   An additional network, increasing memory usage.\n    *   An alternating optimization scheme (training the generator $g_\\phi$ and the auxiliary network $\\epsilon_{aux}$ in turns), which can complicate training dynamics and lead to instability.\n    *   The cost of running forward and backward passes through this auxiliary network at every training step.\n\nIn contrast, VarFlow's training is direct and end-to-end. It requires only forward passes through the student generator $g_\\phi$ and no auxiliary networks or complex optimization schemes. Therefore, while our student-student term is theoretically quadratic, the overall practical training cost is highly competitive, and the simplified training dynamics represent a significant advantage.\n\n**Quantitative Comparison:** The primary computational overhead in VarFlow is the pairwise distance calculation within the energy distance objective. Although the full U-statistic for the student-student term has $O(K^2)$ complexity for batch size $K$, our ablations (Table 3) show strong results with moderate batch sizes (e.g., $K=16$) and a simplified paired estimator.\n\nTo provide a concrete comparison, we will add the following analysis of training time for distilling SDXL on a comparable hardware setup.\n\n| Method | Training Time (hours / 150k steps) | GPU Memory (GB / A100) | Key Characteristic |\n| :--- | :---: | :---: | :--- |\n| VSD (w/ Aux Net) | ~26 hours | ~75 GB | Alternating optimization, potential instability |\n| **VarFlow (Ours)** | **~22 hours** | **~62 GB** | **Stable, single end-to-end objective** |\n\nWe will add a dedicated subsection in the Appendix to provide a more explicit head-to-head comparison of the computational costs and training pipeline complexities of VarFlow and VSD, making these trade-offs clearer.\n\n---\n\n### **2. On Comparison with Consistency Models**\n\nWe agree that Consistency Models are a vital benchmark for few-step generation. We have included several key consistency-based methods in our original submission, as they form a cornerstone of the one-step generation landscape.\n\n*   In **Table 1**, under \"Training from scratch,\" we compare against CT [Song et al., 2023], iCT [Song and Dhariwal, 2023], iCT-deep [Song and Dhariwal, 2023], ECT [Geng et al., 2024], SMT Jayashankar et al. [2025].\n*   In the \"Diffusion distillation\" section of **Table 1**, we compare against SMD Jayashankar et al. [2025].\n*   These baselines are also listed in Appendix A.1 (lines 617, 626).\n\n---\n\n### **3. On Applicability (Distillation vs. Training from Scratch)**\n\nWe thank you for bringing the recent SOTA work [1] (Lu & Song, ICLR 2025) to our attention. In fact, our framework is general and can be applied to **training one-step models from scratch**, and we provide strong empirical evidence for this.\n\n*   In **Table 1**, the section titled **\"Training from scratch: One-step models\"** (lines 219-224) is dedicated to this setting.\n*   The results show that VarFlow achieves **state-of-the-art performance** when trained from scratch, obtaining the best FID on ImageNet 64x64 (3.19), ImageNet 256x256 (6.44), and MS COCO (9.26).\n\nThis demonstrates that the core principle of VarFlow—directly matching noisy distributions via the energy distance—is a powerful and general learning objective for generative models, not just a distillation technique.\n\nWe will revise the abstract, introduction, and conclusion to more clearly emphasize the dual applicability of VarFlow for both distillation and training from scratch, highlighting this as a key strength of our proposed framework.\n\n---\n\nWe hope these clarifications and proposed revisions have addressed your concerns. We would be grateful if you would consider raising your score in light of these clarifications."}]

Author rebuttal:

Meta-review: The paper proposes VarFlow, a training objective for either one-step (or few-step) generative models training or distillation on top of pretrained diffusion models. VarFlow minimizes the energy distance between noisy data and noisy model distributions across multiple noise levels instead of using prior score distillation techniques.  The paper is appreciated in the following perspectives:

- It addresses a high-impact problem (fast sampling) with a simple, theoretically grounded objective.

- It demonstrates state-of-the-art or competitive results without auxiliary score networks and works in the case of training from scratch (with proper warm-up schedule).

- The paper contains comprehensive, careful experimentation and ablations.

The reviewers have raised their concerns regarding the missing computation report, lack of literature survey, etc., which have been addressed in the rebuttal. **The reviewers also notice the claim regarding training from scratch needs clarification and further exploration: how much the model relies on the warm-up. Although no further experiments are requested, the authors need to carefully revise the paper to clarify the training details in order to address these perspectives and avoid any misunderstanding**. Other than that,  the reviewers agree that the paper makes an important contribution in the scope of diffusion model distillation, and with revision the paper meets the acceptance threshold. Thus recommend an acceptance of this paper.
Decision: Accept (poster)