DistillKac: Few-Step Image Generation via Damped Wave Equations
Authors: Weiqiao Han
Chenlin Meng
Christopher D Manning
Stefano Ermon
Venue: ICLR 2026 Poster (2026)
Keywords: generative models
Kac flow
damped wave equation
telegrapher equation
finite-speed probability flow
classifier-free guidance
endpoint distillation
few-step sampling
Primary area: generative models

Abstract: We present DistillKac, a fast image generator that uses the damped wave equation and its stochastic Kac representation to move probability mass at finite speed. In contrast to diffusion models whose reverse time velocities can become stiff and implicitly allow unbounded propagation speed, Kac dynamics enforce finite speed transport and yield globally bounded kinetic energy. Building on this structure, we introduce classifier free guidance in velocity space that preserves square integrability under mild conditions. We then propose endpoint only distillation that trains a student to match a frozen teacher over long intervals. We prove a stability result that promotes supervision at the endpoints to closeness along the entire path. Experiments demonstrate DistillKac delivers high quality samples with very few function evaluations while retaining the numerical stability benefits of finite speed probability flows.

Published as a conference paper at ICLR 2026
DISTILLKAC: FEW-STEP IMAGE GENERATION VIA DAMPED WAVE EQUATIONS
Weiqiao Han MIT
Chenlin Meng Stanford
Christopher D. Manning Stanford
Stefano Ermon Stanford
ABSTRACT
We present DistillKac, fast image generator that uses the damped wave equa- tion and its stochastic Kac representation to move probability mass at finite speed. In contrast to diffusion models whose reverse time velocities can become stiff and implicitly allow unbounded propagation speed, Kac dynamics enforce finite speed transport and yield globally bounded kinetic energy. Building on this structure, we introduce classifier-free guidance in velocity space that preserves square integra- bility under mild conditions. We then propose endpoint only distillation that trains student to match frozen teacher over long intervals. We prove stability re- sult that promotes supervision at the endpoints to closeness along the entire path. Experiments demonstrate DistillKac delivers high quality samples with very few function evaluations while retaining the numerical stability benefits of finite speed probability flows.
INTRODUCTION
Diffusion models have catalyzed an enormous research wave. Since the seminal Denoising Dif- fusion Probabilistic Models (DDPM) paper, there are now tens of thousands of diffusion-related publications, and the DDPM work alone has accrued well over twenty-five thousand citations (Ho) al. 2015). Conceptually, diffusion models couple a stochastic for- ward noising process with reverse-time ordinary differential equation (ODE)/stochastic differen- tial equation (SDE) whose density evolution is governed by the Fokker-Planck equation, linear second-order partial differential equation (PDE) (Song et al., 2021b). Learning consists of esti- mating time-indexed scores/velocities and integrating them for sampling (Ho et al.) 2020 [ong & Ermon 2020 Song al.) 2021a) Recently, growing body of work explores probability flow back-
al. 2020 Sioh--iickstein
bones: deterministic ODE flows trained via flow matching and its optimal transport variants (Lipman)
ct al. 2023 Tong et al. 2024), rectified flows (Liu et al. 2023). and unified stochastic interpolants
that bridge flows and diffusion (Albergo et al. 2023).
Yet the PDE lens generative modeling need not be limited Pokker-Planels Other classical PDEs offer different structural advantages but remain comparatively underexplored. For example, cettrrsstaticsiisspiee fields (Xu et al., 2022 2023).
Poisson flow models recast generation through
Most relevant to our work, Duong et al. al, (2026) replaces the Fokker-Planck equation with the telegra- pher equation (1-dimensional damped wave equation) and its stochastic Kac representation, proving finite-speed probability flows that are Lipschitz in Wasserstein distance with globally bounded ve- locity norms. Crucially, telegrapher/Kac dynamics enforce finite propagation speed: starting from localized mass, the distribution remains inside a causal cone. In contrast, diffusion spreads mass instantaneously with effectively infinite propagation speed, which also manifests stiff, rapidly growing reverse-time velocities near terminal time (Yang et al.) (023). Indeed, diffusion arises as a limit of the damped wave equation as the damping and propagation speed tend to infinity. This structural gap, finite speed versus infinite-speed propagation, will be central to modeling and algorithms.
In this paper, pivot to the damped wave equation and its stochastic Kac representation. which induces fellcevvlocity probability flows. Compared to diffusion ODEs/SDES, whose velocity norms blow up terminal time, Kac dynamics yield globally bounded kinetic energy and Lipschitz regularity in Wasserstein space. Intuitively, the finite-speed cap e acts like a built-in stability con- straint: trajectories cannot accelerate without bound, characteristic fronts move faster than e, and

---

Published as a conference paper at ICLR 2026
Parabolic PDE
Elliptic PDE
Hyperbolic PDE
Fokker-Planck equation
Poisson equation
Damped wave equation
Diffusion models
Poisson flow models
Kac flow models
Sohl-Dickstein et al. 2015]
Xu et al. 2022 2023
Duong et al. 2026 and this work
Ho et al., 2020) and tens of thousands of other diffusion
papers
Table I: Three PDE lenses for generative modeling: representative model families and citations. Our work belongs the hyperbolic family, shown in the third column.
late-time integration is less stiff. Building on this structure, we introduce guided Kac flows for condi- tional generation and distilled Kac flows for few-step sampling: (i) we adapt classifier-free guidance (Ho Salimans 2022) directly in velocity space while preserving bounded-energy guarantecs, and (ii we develop endpoint-only distillation scheme with provable endpoint-to to-trajectory stability bound. Empirically, these properties support stable few-step samplers with strong sample quality. Theoretically, they clarify why endpoint matching suffices under finite-speed Kac dynamics.
2
THEORETICAL FOUNDATIONS
2.1 PARTIAL DIFFERENTIAL EQUATIONS
The time evolution of the probability density of the diffusion process is governed by the Fokker-
Planck equation
0,pll,x) V (p)))( + (D,
(I)
where t E R is time, E Rdd is state, put, x) is the drift vector, and Dtt, oo is the diffusion tensor. The heat equation is a special case of the Bokker-Plance' equation.
While the Fokker-Planck equation is linear, second-order parabolic PDE, there are other important classes of linear second-order PDEs, such as elliptic and hyperbolic equations. In particular, the wave equation and its damped variant canonical examples of hyperbolic PDEs. The damped wave equation takes the general form
d x) (0:u(t, = At(,tx))
where e 0 is a constant and e is the speed of wave front. The telegraph(er) equation 1-dimensional damped wave equation.
2.2 KAC PROCESS AND RANDOM FLIGITT
Just as the diffusion process associated with the Fokker-Planck equation, the damped wave equa- tion also corresponds to stochastic process.
Define stochastic process (X()) in Rd, 2 1, as follows:
1. Initial condition. The particle starts at the origin X(0) = Rdd
2. Velocity and speed. The particle moves with constant speed c At any time, the velocity is (tt) = cU))), where U(t) is random vector on the unit sphere s = {u IRd |||| = 1}.
3. Direction changes. The direction process {U{+) changes at the jump times of a ho- mogeneous Poisson process (t) of rate X > 0. That is, if the jump times are
0 sg < S1 82
S,, then:
On each interval [s, (s)-1.8;), the velocity is constant: V(t) = cU/--
The random directions {U3} i.i.d. with uniform distribution on the unit sphere
8d-1
2

---

Published as a conference paper at ICLR 2026
The position of the particle at time is obtained by integrating the velocity
N(() (s; 53-1)Uβ-
x()) / V(s)ds
(3)
==1
where so 0 and (())))
When d 1, 8 = (+1), and the particle flips directions between left and right. The d = 1 is usually called telegraph process or Kac process (Goldstein, 1951 Kac 1974). When d 2 2. the process is called a random flight (Orsingher & De Gregorio 2007).
Theorem 1. (Orsingher & De Gregorio 2007) For d 1, and the distribution of the state (t) can be obtained analytically, while for any other positive integer d there is no explicit formula.
Theorem 2. (Orsingher & De Gregorio
2007
For = 1 and 2, the state distribution solves the
damped wave equation.
2.3
IMAGE GENERATION VIA COMPONENT 1-D TELEGRAPHER EQUATION
Duong et al. 2026) study Kac process and the telegrapher equation (the 1-D damped wave equation), and derive a closed-form expression for the conditional velocity u(t, x m/co) for any time i, and noisy æ E R given data zo- Leveraging this expression, they propose simple procedure to learn the data distribution from noise:
1. Given data To, sample a time and then draw the state x from the analytic state distribution (Theorem l.
2. Evaluate the closed form conditional velocity v(t, x xo)) and fit neural network og (t(t,a)) to it by regression (e.g. minimizing (l]])=((,,)) - vll, (l)
At inference, draw 2T ~ p(T, a) (noise) and integrate the learned velocity field backward time from t = T to 0 obtain a data sample To-
For multi multi-dimeensonal data, c.g., images, they model each coordinate with an independent 1-D Kac process, i.c., a component-win product construction.
Because each coordinate follows the 1-D Kac dynamics, the component-wise image model inherits finite-spedd speed propagation and dimension-awe
energy bounds. In 1-D, the Kac solution started at zo supported on (To -0--c. ct, To cf), where e is the wave front speed Equation ( and the conditional velocity satisfies =[],) < c, with v(t, + tc. Applying this per coordinate, the component-wise construction for images propagates inside the fxc causal cone (x : ||æ zolloo S
dt) Moreover, for the resulting d-dimensional (product) Kac flow (14)c20 there exist dimension- aware global bounds
W2 p4) S cvdjt s/,
[ellez eVd fora.e E [0, 1].
and the support grows most linearly in time. We will leverage these properties to prove bounded energy under classifier-fie ffree guidance and
stability for distillation.
2.4 NOTATIONS
Let / E IR denote time, x E RRd the state, and E N a class label. Let P2 (IRd) denote the set of Borel probability measures on Rd with finite second moment, endowed with the quadratic Wasserstein distance
w2(,R) inf / xdd TEIl(,N))
|| vIT
for U.D E P2(R)), where I(p,v) v) is the set of couplings of u and v. For a measurable map F: - RR, the pushforward measure of μ is F# μ F-1
We use Kac process when we talk about individual random paths (simulation/noising)) and Kac Kac flow when we talk about how distributions move in time and the ODE integrate inference.

---

Published as a conference paper at ICLR 2026
Algorithm Endpoint Distillation (N steps)
Inputs: N, At, Üy (normal/CFG velocity), 1,, B, mav_iter &- 0g (frozen), student Ue f- Üg
2: Init: teacher
3: for i = 1 to maz_iter do
4: sample {(t(h) xz ( ())
5: x:0) - TEACHER ug, g(6), (b)), ()) ., At)
6:
x((b) - x2(5) uo (1(())) ,x(6),
(b) (
7:
[(0) + EE ||ā(0) x+ 12:
0 t 0 - VL
8: end for
CLASSIFIER FREE GUIDANCE VELOCITY SPACE
While Duong et al. 2026) focus unconditional image generation, address both unconditional and conditional generation, and further introduce classifier free guidance in velocity space.
Let (t z) and uo (t, be the learned unconditional and conditional velocities, respectively. We define the velocity guidance with strength wt) 2 0 by
o(t, y) vy (t, w()) x; v) (t, 2)),
(6)
and integrate the reverse ODE (0() ((, v) from / = to 0 to generate data samples.
Let (t, x: Assumption 3. -Square-integrabbe guidance gap) For E [0, 1), Exzn (l, v) <
== no(t, x: ot) denote the conditional-unconiitinn
velocity gap.
for the random condition y, equivalently, EylEa-p, (t,, <0.
Theorem 4. (Energy bound under guidance) Suppose the unconditional Kac flow satisfies [[v r2))) S cvd for a.e. t, and Assumption 3 holds. If (w(t)) is finite for a.e. t, then o(t, vu) E (14) a.e. t.
The proof is in the Appendix. The assumption that (t, (t.t,,))l S <edd for a.e. t follows from corresponding (1)) bound on the velocity field in Duong al. (2026) under its stated hypothesis, which is generally satisfied. In practice. we may take w(t) to be constant so the requirement that [t)))] is finite for a.e. t automatically satisfied. The conclusion ū(t, E L'((4) for a.c. t then implies that the guided velocity has finite kinetic energy, unlike diffusion, where the kinetic energy can diverge near data time.
DISTILLATION
We distill a student from a frozen teacher by endpoint matching. The teacher is either a learned flow or a classifier-free guidance model. Unlike progressive distillation (Salimans & Ho [2022). which performs two-step distillation per iteration, we match over an arbitrary number of substeps
with N > 2. The distillation algorithm is shown in Algorithm [ At each training iteration and for each sample b, the TEACHER routine integrates the frozen teacher velocity backward from g(b) to {(b) At using N uniform substeps (e.g., explicit Euler, midpoin//RK2.
(aam--Bashforth22) producing the reference endpoint ( The endpoints {(b) and (b) At are chosen to coincide with the student's 's segment boundaries: for an M-step student with a uniform time grid = 4 2 ...
1M+1 = 0, (b) = tx. then (b) At 1++1. Over the same interval [E(6] - At, {b)) the student takes explicit Euler step (backward in time), and is trained with MSE loss to match x(()) Both the teacher and the student are initialized from the same pre-trained velocity model Ug. and the teacher remains frozen throughout training.
Let u and v be the velocity fields for the teacher and the student Kac flows, respectively. The corresponding ODEs defined on [0, 1] are
c u(t, so
(7)
J/2 u(t, v/2
(8)
4

---

Published as a conference paper at ICLR 2026
Let H4 and V4 be the respective distributions of (p and 1/. Let P--tt and --- denote the flows associated with Oi and 1h,, respectively. That is, for S $ S & 1, the flow map Pa-+ : R - R is the unique map such that for every R the trajectory + (2)+((z) is absolutely continuous on [s, 1] and solves the ODE Equation for a.e. T E [s, 4/. with the initial condition yos = 2::
D (-) = L. u(r, 2 (.)r
(9)
The flow p-- is also well defined when 0 S - 8 Σ 1. In particular, sampling from noise (t = 1) to data (t 0) corresponds to the single map 42--0-
We now lay out mild conditions on the teacher and student dynamics that ensure the flows are well defined. The subsequent assumptions and lemmas formalize these regularity and bounded- ness requirements and show that small discrepancies do not explode as we evolve in time. With this groundwork in place, the main stability result will convert endpoint agreement into closeness of entire trajectories, and the concluding corollary will turn that principle into concrete accuracy guarantees for practical, few-step students.
Assumption 5. (Spatial Lipschitz drifts) There exists L(t) such that, for a.e. t E [0, 1J. both u(t, and ((t,,:))) are L(t)-Lipschitz in x on the support of pr U uss and Jo L(t)dt < c0.
Lemma 6. (Lipschitz constants of flow maps) Under Assumption 3 for any 0 8 S 1 S 11.1. Lip(#.-x)) S exp(Sx L(r)dr), and similarly for V-----
The proofs of this lemma and the next are provided in the appendix. Here Lip( (+. denotes the Lipschitz constant of the flow Dz-st-
Lemma 7. (Pushforward contraction and coupling) For any ) map F Rd + Rd and v.v (R2), (2 Fuy)= S Li2(F))2(u,, v). Moreover, for any probability mea- sure V E )2R(R)), and measurable maps F,G Rd Rd. with J [1](())) < co and f [IG(=)))dd(z)) < oo, one has (F G I (Ex~~ (F()) G(X)
-
The following result captures the core stability principle behind endpoint distillation: if student matches the teacher at the end of each interval, then the two paths remain close throughout that interval.
Theorem 8. (Endpoint--ee teapppinttto--raaeee trajeccyy stability) Lete: := W2(Hs,D)) be the discrepancy at some time s E [0. Under Assumption 5 for every E [s, 1].
W2(ur, S exp (f, Lrrdar) s+ [ exp (S. ((r)drr) [lu(t,;v)) (1)=
(10)
Proof. By well-posedness, Mr = D and v, = 1 VorrHu,, Apply the triangle inequality and Lemma 7
W2(H,, v4) < W2 2 2 + W2
4 - #1)7 S Lip ( W (2 v.) + (Ex~e, || [ (X) V -- (X)
(11)
(12)
By Lemma Lipl I S L(r)dr), giving the first term.
For the second term, define ( == T (X) - V (X) for X ~ us. Then u(t, $ (X):)) v(t, v (X): We have
di
d A
d A < 11 d Arl Arl (by
inequality, see Equation A-8
(13)
di
di
||u(1,, + (X):y) v01, V. (X):vv)
(14)
|l( 4 - - ult,, V. (
SL(() | (by Assumption 5
+ |lllt, V. Vx-re(X); vlt, a (X):8)]
(by triangle inequality)
(15)
5(t 2 v..)e)
5

---

Published as a conference paper at ICLR 2026
By Grönwall's inequality with | 0.
|XIIX S S: Teep (J. [()oarr 8(1, (X)))de
(16)
Take (v,) norms of both sides,
[X)((X) S 1." eex (J. L(r)ar) ö(t, v (x-r(
(17)
(P(v))
$ [. exp (J." L(r)ar) 18(t, (x),()))]] (v.
(18)
'." exp Jo" E(r)ar) [l((t, : (.)
(19)
In (8) we used Minkowski's integral inequality. In (19), we used pushforward identity V Vo-c4% Ft.
Setting s = 0, i.e., supervising only the data endpoint at t O, yields, for all r [0, 1],
W2((,; s exp (["1) 1. Wa(Ho, + L'" exp (f.z)
()) [lu(t, - v(t, (0) (2(,)) (2
(20)
which is the endpoint-only distillation bound propagating the I = 0 mismatch to the whole trajectory.
Corollary 9. (Few-step students: order-p one-step methods) Let the student be one-step method (advancing to the next state using only the current state) of local order p 2 1 with nodes 1 = f 2 ... +M++1 = 0 and steps h2 tt t&+1- Assume that along student trajectories, the teacher drift v has bounded time derivatives to order p and is L())-Lipschitz in x. Then there exist constants Tk (depending on local p-th time derivatives along leacher paths and spatial Lipschitz, moduli
T((1)) such that
[lu(s,;;)) - v(8, -w))l S Txhhtt RN1
(21)
Consequently,
We(mr, (7) S exp (T.1) e+ exp (%, (
M [0.1]
Tan p+1
(22)
For explicit Euler (p (p=1) and uniform steps by = 1/M, the second term scales like O(M-1) up to smoothness constants.
For explicit Euler (p = 1), we take the student velocity v(s, u((x+11, for all Is [1++1, (e). Then, by the Fundamental Theorem of Calculus,
[lu(s :) v(s. (0..::8)ll..0 -)))))))))))))) = [lu(ss (g) -
(23)
< |00u(r,;:)ll
(24)

---

Published as a conference paper at ICLR 2026
Figure I: Uncurated CIFAR-10 (left) and CelebA-64 (right) samples generated by Guided Kac Flow (midpoint integrator, 100 steps, guidance w for CIFAR-10 and 0 for CelebA-64),
Integrating both sides over the interval Ix.
S I"
1|u ulI
ean Jenn
(25)
ft (S"t,, do) (10ru(r; (o)l (by Tonelli's theorem) (26)
de
(tx - : (
(27)
1x+1
sup (0.u(r; rrl
(1x r)dr
(28)
1
sup [10,u(r, : (10l))))))))))
h
(29)
2
reh
we recover Equation 21
For first-order student, doubling the number of steps typically cuts the error by about half. For a higher-order student, the improvement is even faster. In practice, we balance efficiency and ac- curacy: higher-order students often require more evaluations per step, so we choose the order and step count to hit a target quality within given compute budget. The constants depend on how smooth the teacher is and on the strength of guidance, but the qualitative rates are robust thanks to the finite-speed and bounded-energy structure of the Kac backbone.
EXPERIMENTS
5.1 SETUP
We train on CIFAR-10 (32x32), CelebA-64 (64x64), and LSUN Bedroom-256 (256x256) The velocity backbone UNet, and the hyperparameters and training details are provided in Appendix The procedure for selecting Kac flow hyperparameters is described in Appendix [B.2 For classifier-free guidance, we use a constant guidance strength w on CIFAR-10 and unconditional generation on CelebA-64 and LSUN Bedroom-22 (i.e., w 0).
B.I
5.2
MAIN RESULTS
Our main CIFAR-10 results are summarized in Table 2 We train a conditional generative model and apply velocity guidance at evaluation, referred to Guided Kac Flow. Figure 2 plots FID

---

Published as a conference paper at ICLR 2026
CIFAR-10
C6
Method
FID
NFE
Conditional
FID
NFE !
Kac Flow (Duong al. 2026)
6.42
100
Guided Kac Flow (ours, 100 steps, midpoint)
3.54
200
V
3.36
200
Guided Kac Flow (ours, 100 steps, AB-2)
3.58
100
4
3.50
100
DistillKac (ours)
3.72
3.42
20
4.14
4
V
4.36
4
4.68
2
V
5.66
2
5.66
7.45
DDPM Ho 2020
3.17
1000
DDIM
Song al. 2021a
4.04
1000
3.51
1000
4.16
100
6.53
100
4.67
50
9.17
50
6.84
20
13.73
20
13.36
10
17.33
10
Progressive distillation (Salimans & Ho 2022
2.57
8
3.00
4.51
9.12
EDM (Karras ct al.
2022)
1.79
35
V
1.97
iCT (Song & Dhariwal 2024)
2.46
2.83
Soft Diffusion
Daras et al
2023)
1.85
300
PDM-DDPM+ (Wang et al.
2023
1.77
50
Soft Truncation-G++ (Kim et al.
2023
1.34
131
Table 2: Main results on CIEAR-10 and CelebA-64. Lower is better for FID. We report means 50k samples. Ours denotes the Kac backbone with guidance and endpoint distillation. NFE counts velocity evaluations; midpoint integrator uses 2 function evaluations per step.
CIFAR FID w for Guided Kac Flow (100 steps)
CIFAR FID Integration Steps
400
(lower is better)
4.4
AB-2
Midpoint
FID (lower is better)
Guided Kac Flow, w=1.2, mid.pint Guided Kac Flow, w-1.2, ab2
4.2
Euler
DistillKac
4.0
200
3.8
3.6
1.0
1.1
1.2
1.3
1.4
1.5
20
40
60
80
100
(classifier-free guidance strength)
Steps
Figure 2: Left: CIFAR-10 FID vs. guidance strength w for the 100-step Guided Kac Flow. Second- order integrators (midpoint and AB-2) outperform Euler. Right: CIFAR-10 FID vs. integration steps. DistillKac substantially reduces FID at 20, 4, 2, and steps relative to Guided Kac Flow.
as a function of guidance strength w. We evaluate three time integrators: (1) Explicit Fuler (first- order); (2) Midpoin//RK2 (second-orders (3) in Figure[2, second-orde -odder methods, midpoint and AB-2, yield lower FID. Because AB-2 requires one function evaluation per step whereas midpoint requires two, AB-2 offers better trade-off.
(AB-2; second-order). As shown
accuuracy
Starting from 100-step Guided Kac Flow model teacher, distill 20-step student and then iteratively distill to 4, 2, and steps, each stage using the previously trained student as the teacher. We refer these distilled students as DistillKac. 2 reports their FIDs. As shown in Figure
8

---

Published as a conference paper at ICLR 2026
CIFAR-10: Time vs Steps
Celeba-64: Time vs Steps
1250
800
Time (secondds)
Time (seconds)
600
750
400
500
200
250
5
10
15
20
10
15
20
Steps (NFE)
Steps (NFE)
Figure 3: Time (seconds) to sample 50k images with one H100 GPU at 2, 4, and 20 steps (NFE).
distillation significantly improves FID at 20, 4, 2, and steps relative to the original Guided Kac Flow at the same step counts. DistillKac reduces the sampler from 100 to step with FID rising only from 3.58 to 5.66 (+2.08). We attribute this robustness to the endpoint-to toatn--ttttrraetoryyy stability of Kac flows (Theorem 8)
Analogously, we train an unconditional CelebA-64 Kac flow model. To distinguish it from Distil- IKac students, still call it Guided Kac Flow teacher (w 0). Starting from the teacher with 100 steps, we successively distill it to 20, 4, 2, and steps. As the step count decreases, FID increases from 3.42 (20 NFE) 7.45 (1 NFE). For comparison, the original Guided Kac Flow teacher yields 11.23 at 20 NFE and 443.01 at NFE (See Appendix (rpendii(B.4).... B.4). This indicates that distillation preserves image quality far better than the teacher in the few-step regime.
St) can perform better than single-stage distillation (S} + Se) (See Appendix B.5). Multi stati- schedules (i) allow more hyperparameter tuning and (ii) reduce per-stage runtime, enabling more effective hyperparameter search.
Multi-stage distillation (S) S2
6 RELATED WORK
iteratively halves the sampling steps by training student to reproduce the effect of two teacher steps one step. Consistency Models (Song et al.] 2023) learn a single network with multi-time consistency objective so that sample can be pro- duced in or evaluations via the probability flow ODE. Our endpoint-only distillation trains each student segment to match the teacher integrated over N 2 2 substeps on that segment and pro- vides an endpoint-to-trajecctory stability guarantee under Kac dynamics. Conceptually, our scheme is closer to Progressive Distillation and, like Meng et al. (023). we allow the teacher to employ classifier- free guidance.
Progressive Distillation (Salimans & Ho 2022)
Beyond pixel-space accelerated samplers, distillation of diffusion models in the latent space of a pre- trained autoencoder widely used for large images. Meng et al. (2023) distills classifier- free guided diffusion models into few-step latent samplers. Latent Consistency Models (Luo et al., 2023) extend consistency models to the latent space of pre-trained latent diffusion models, such as Stable Diffu- sion. Latent Adversarial Diffusion Distillation (Sauer et al. (024) distills large latent diffusion transformer by training student in latent space using the teacher both as a generator of synthetic guided samples and as multi-scale, noise-aware discriminator backbone, enabling stable adversar- ial one-step or few-step sampling without decoding to pixels. Although we focus on pixel-space framework representation-agneen
Kac backbones for image generation, our andpoin-ddistilaate and could be applied equally well to latent Kac flows on top of pre-trained VAE. In addition, our cechittetuur-agnnsttee and can be applied not only to UNets but also to transformers and other architectures.
framework
Our initial teacher, Guided Kac Flow, a conditional Kac flow velocity model trained under hyper- bolic, finite-speed Kac dynamics, i.e., a second-order system with an explicit velocity variable. By contrast, flow matching (Lipman et al. 2023) and rectified flow (Liu et al. 2023) learn first-order ODE velocity fields along chosen data noise interpolants, with independent velocity state no wave-like propagation, and thus no built-in finite-speed constant. Moreover, classical diffusion has infinite propagation speed, and in learned diffusion models the score/velocity field becomes ill-

---

Published as a conference paper at ICLR 2026
Figure 4: Uncurated LSUN Bedroom-256 samples generated by Guided Kac Flow (midpoint inte- grator, 100 steps, guidance w = 0).
Figure 5: Uncurated LSUN Bedroom-22 samples generated by DistillKac (Euler integrator, 20 NFE).
conditioned as -> 0 (at the end of the denoising process), with Lipschitz blow-ups observed and analyzed (Yang et al. 2023 Duong et al. (2026).
7 DISCUSSION AND FUTURE WORK
While our CIFAR-10 and CelebA-64 FIDs are below the state-of-the-art diffusion models, we do make SOTA claims. Instead, results show that Guided Kac Flow and DistillKac produce competitive, high-fidelity images at low NFE. We expect further improvements from stronger back- bones and additional tuning of the Kac flow hyperparameters. These orthogonal to our core contribution. Our goal is to highlight Kac flow based generative modeling as a credible alternative diffusion and to encourage follow-up work in this direction.
Unlike diffusion models, whose full probability path satisfies the Fokker-Plancl equation, Kac flows enjoy ID Feynman-Kae correspondence with the telegrapher equation, yielding bounded-speed probability flow. As shown in Theorem 2 for higher dimensions (d > 2), the solution to the damped wave equation need not coincide with probability density for the Kac process (and may fail to be valid density). Consequently, the Kac flow paper (Duong et al.) (0266) adopts component- wise multi-D process built from independent ID Kac marginals, for which the continuity equation and Lipschitz and finite-speed properties hold. This raises natural open question:
struct genuinely d-dimensional stochastic process (with possibly dependent coordinates) whose probability path satisfies hyperbolic PDE while preserving mass, perhaps under specific boundary conditions or coupling structures? Establishing such a model, along with stability guarantees and asymptotic connections to diffusion, would broaden the toolbox of finite-speed generative flows.
REFERENCES
Michael Albergo, Nicholas Boffi, and Eric Vanden-Fijnden. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint 2rX3v:2300300877 2023.
Giannis Daras, Mauricio Delbracio, Hossein Talebi. Alex Dimakis, and Peyman Milanfar. Soft diffusion: Score matching with general corruptions. Transactions on Machine Learning Research, 2023.
10

---

Published as a conference paper at ICLR 2026
Richard Duong, Jannis Chemseddine, Peter K Friz, and Gabriele Steidl. Telegrapher's generative model via kac flows. arXiv preprint arXhv:2006.064117, 2026.
Sidney Goldstein. On diffusion by discontinuous movements, and the telegraph equation. The Quarterly Journal of Mechanics and Applied Mathematicx, 4(2): 1951.
Jonathan Ho and Tim Salimans.
Classifier-free diffusion guidance.
arXiv preprini
wrXiv:2207.12590, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 -6111, 2020.
Mark Kac, A stochastic model related the telegrapher's equation. The Rocky Mountain Journal of Mathematics, 4(3):497-509, 1974.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577 2022.
Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang. and Il-Chul Moon. Refining gener- ative process with discriminator guidance in score-based diffusion models. In 40th International Conference Machine Learning, JCML. 2023, pp. 16567-16598. MI Research Press, 2023.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow match- ing for generative modeling. In The Eleventh International Conference Learning Representa- tions, 2023.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. International Conference on Learning Representations, 2023.
Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.0437 2023.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14297-14306, 2023.
OpenAl. guided-diffusion (github repository). https://github. cww coppo/aaaacomm/maaaa openai/ guided-diffusil 2021. MIT License. Commit: 22e0df8, accessed: 2025-07-20.
Enzo Orsingher and Alessandro De Gregorio. Random flights in higher spaces. Journal of Theoret- ical Probability, 20(4):769-800) 806, 2007.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022.
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rom- bach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIG- GRAPH Asia 2024 Conference Papers, pp. 1-11, 2024.
Jascha Sohk-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265. pmlr, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 202
Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024.
Yang Song and Stefano Enmon. Improved techniques for training score-basee based generative models. Advances in neural information processing systems, 33:12438-1244 12448, 2020.

---

Published as a conference paper at ICLR 2026
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021b.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, pp. 3221 -32252. PMLR, 2023.
Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024.
Zhendong Wang, Yifan Jiang, Huangjie Zheng. Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou, et al. Patch diffusion: Faster and more data-efficient training of diffusion models. Advances in neural information processing systems, 36:72137-72154, 2023.
Yilun Xu, Ziming Liu, Max Tegmark, and Tommi Jaakkola. Poisson flow generative models. Ad- vances in Neural Information Processing Systems, 35:16782-16795, 2022.
Yilun Xu, Ziming Liu, Yonglong Tian, Shangyuan Tong, Max Tegmark, and Tommi Jaakkola. Pfgm++: Unlocking the potential of physics-inspired generative models. International Con- ference Machine Learning, pp. 38566-3888 38591. PMLR, 2023.
Zhantao Yang, Ruili Feng, Han Zhang, Yujum Shen, Kai Zhu, Lianghua Huang, Yifei Zhang, Yu Liu, Deli Zhao, Jingren Zhou, et al. Lipschitz singularities diffusion models. In The Twelfth Inter- national Conference Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2026
LLM USAGE
LLM use disclosure. We used large language models (i) improve the grammar and wording of this manuscript and (ii) assist with experimentation by drafting portions of the code and suggesting debugging steps. All scientific ideas, claims, experiment designs, and final implementations the authors. We reviewed and verified all LLM-assisted text and code before use, and the authors remain fully responsible for the content of this paper.
by
A EXTRA PROOFS
A.1 PROOF OF THEOREM 4
By triangle inequality, for a.c. t,
(0(4:5)))l>))m) S |lva(t, (LE +
(A-1)
s sevd+ (olt) [M(M))
(A-2)
cvd+ + (w( Br-ma (t,
(A-3)
(A-4)
From Equation 1 Equation A-2 we used the assumption that (ll(t,))l S cvd for
a.e. t. From Equation A-3 to Equation (A44), we used Assumption 3 and the assumption that [ur(t] is finite for a.e. Therefore, (14 (1e) for a.e.
A.2 PROOF OF LEMMA 6
Let H, Z E Rd and t + (z)+ (2) be teacher characteristics, i.., the solution trajectories (integral curves) of the ODE defined by Equation (7) Let
(1) $ (x))(x)) $)))(2).
(A-5)
Then
d A(t) u(t, T. (,)(());)) ult, $ f.-r(2):))
(A-6)
dt
So
Att)
|a(t)|
||(!I]| dt A(t))
(A-7)
dt
(t) A(t)
(by Cauchy- inequality)
(A-8)
|
dt
[t,, Φb ())=(();)) ult, T (=)+v())
(A-9)
< L(t)] 1b (x) (x)( (by Assumption
(A-10)
A(1)
By Grönwall's inequality,
1 (x:) - +.)+(())] s exp (l. ((r))r)) || ||00 2|1.
(A-11)
hence
Lip( S exp ( terjar).
(A-12)
13

---

Published as a conference paper at ICLR 2026
A.3 PROOF OF LEMMA 7
Let π be an optimal coupling between μf and y for W2. Then
W2 FF F&v) - / |1F((a) - F(())) (by definition of W2)
(A-13)
- Lip(t)) / || 2112 (by definition of Lip(F))
(A-14)
Lip(F) (by definition of W2).
(A-15)
Let X and set Y F(X), G(X). Let 7 be the joint distribution of (Y, Z). i... = (F x uv Then y is coupling of F# v and Gæu, ie., E (T G
By the definition of pushforward,
{ RdXRd ly =11 2) / [I (())) (dx))) Bx - -(x)) (A-16)
JRd
By the definition of W2.
W 1F G+v) = (ell(Fvr,Gau) f xdxxd 11 - = 1l2 2)
inf
(A-17)
[ JRRXRR ((X) - G(X)II
11 - =112
(A-18)
(A-19)
14

---

Published as a conference paper at ICLR 2026
B ADDITIONAL EXPERIMENTAL DETAILS
B.I EXPERIMENT CONFIGURATIONS
We use the UNet architecture from OpenAl's guided-diffusion github repository (OpenAl 2021
to train models on CIFAR-10 and CelebA-64. Following Duong et al. (2026), we define mean- reverting Kac process by
M, == f(t)Xo + Kete, E [0, 1],
(B-1)
where Xo Hi denotes the data, K2 is Kac process started from O, and f.g [0,1] + IR smooth functions satisfying
f(0)- = 1, f(1) =0, g(0) 0, =1)
(B-2)
In particular, MM Xg and M1 K1. Thus f and g are hyperparameters of the mean-reverting Kac process. Let a == 2. where & is the damping coefficient in the damped wave equation (Equa- tion (2) and let e denote the wave speed. The pair (a, c) are additional hyperparameters of the mean-reverting Kac process. Table summarizes the UNet, training, and Kac process hyperpa- rameters. We adopt early stopping when the validation FID curve flattens.
Section
Parameter
CIFAR-10
CelebA-64
LSUN bedrooms-256
Input resolution
32x32
64x64
256x256
Base channels
256
192
256
Residual blocks stage
2
Dropout
0.0
0.0
0.1
UNet
Head channels / attn
64
64
64
Channel multipliers
[1, 2, 3]
[1. 2, 4]
[1. 1, 2, 4]
Attention resolutions
[16, 8, 4]
[32, 16]
[32, 16, 81
Scale-shift norm
yes
yes
yes yes
Resblock up/down
yes
yes
Use AMP
no
no
yes
Optimizer
AdamW 10-4
AdamW
Adam W
Learning rate
x
2 x 10 44
Batch size
128
128
64
Weight decay
0.01
0.01
0.01
LR schedule
Flat + Cosine
Flat + Cosine
Flat + Cosine
Warmup steps
2%
2%
2%
Training
Flat steps
58%
58%
58%
Cosine steps
40%
40%
40%
Gradient clip
1
EMA decay
0.9999
0.9999
0.9999
Training steps
300k-400k
300k-400k
300k-800k
Early stopping
yes
yes
yes
f(() (())
t
Kac
(a,c)
(25,2)
(3000, 20)
(3000, 20)
Table B-1: UNet, training, and Kac process hyperparameters for CIFAR-IO and CelebA-64,
We visually inspect 100 generated samples every Sk training steps, and compute FID using 2,000 samples every 10k steps. For CIFAR-10, we trained on single NVIDIA A100 or H100 GPU. The model typically converged after approximately 100k steps (es 15 hours), at which point the training curve plateaued and we can apply early stopping. For CelebA-64, we trained H100 GPU. The training curve plateaued after around 150k steps (=e 30 hours) and full training to 400k steps required approximately 80 hours. For LSUN Bedroom, we trained on 4xH100 GPUs. The training curve plateaued after about 300k steps (+= 3 days), but continued to improve very slowly thereafter. Extending training 400k-800k steps required approximately 4-8 days in total.
single NVIDIA
15

---

Published as a conference paper at ICLR 2026
B.2 KAC FLOW HYPERPARAMETERS
As detailed in Appendix B.1 Kac flow hyperparameters comprise a, and the functions f and g. We first use a lightweight UNet to search over different values a, c, estimating FID with 2k samples efficiently identify promising settings. We then adopt deeper UNet described in B. 18.Taa and report final FID using 50k samples. Following Duong et al. (2026), we set f(t) = t, and consider g(t) = or t
s()) =
g(t) =
(a, c)
FID
(a,c)
FID
(25, 1)
20.16
(900, 10)
16,42
(25, 2)
19.83
(1500, 10)
16.60
(25,3)
19.38
(1500, 12)
15.83
(25,4)
20.25
(3000, 10)
18.11
(100,5)
36.23
(3000, 20)
15.30
(900, 10)
37.31
(6000, 20)
16.96
Table B-2: FID@2k samples with lightweight UNet on CelebA-64
Table B-2 reports FID computed from 2k samples using lightweight UNet CelebA-64. The set of hyperparameters (a, = (3000, 20). f(t) = 1, gt) = 12 yields the lowest FID and can be adopted for subsequent experiments. The hyperparameters of the lightweight UNet listed in Table B-3
Input resolution
64x64
Base channels
128
Residual blocks / stage
2
Dropout
0.1
Head channels / attn
64
Channel multipliers
[1, 2, 2] [16]
Attention resolutions
Scale shift norm
no
Resblock up/down
yes
Table B-3: Lightweight UNet Hyperparameters
B.3
FID FOR GUIDED KAC FLOw ON CIFAR-10
Table B-4 and Table B-5 tabulate the FID values used to generate Figure AB-2 uses one function evaluation per step whereas midpoint uses two evaluations per step.
1.0
1.1
1.2
1.3
1.4
1.5
midpoint
4.20
3.73
3.54
3.56
3.72
4.10
AB-2
4.32
3.80
3.58
3.68
3.85
4.21
Euler
4.52
4.18
4.00
4.00
4.21
4.47
Table B-4: FID for Guided Kac Flow (100 steps) on CIFAR-10
steps
100
80
64
50
25
20
10
5
4
2
midpoint
3.54
3.55
3.59
3.60
3.87
4.39
13.03
49.14
70.36
202.37
396.46
AB-2
3.58
3.62
3.67
3.77
8.25
13.99
51.36
125.63
165.
346.16
396.38
Table B-5: FID vs. integration steps for Guided Kac Flow (w = 1.2) on CIFAR-10.
16

---

Published as a conference paper at ICLR 2026
B.4 FID FOR GUIDED KAC FLOW ON CELEBA-64
Table B-6 reports FID for the Guided Kac Flow teacher (w 0) across various NFE using the midpoint and AB-2 integrators. AB-2 uses one function evaluation per step, whereas midpoint uses two evaluations per step. Consequently, midpoint cannot be run at NFE = 1. Figure B-1 shows FII vs. NFE for both the Guided Kac Flow teacher (w = 0) and the Kistill student. Dis substantially reduces FII) at 20, 4, 2, and steps relative to the Guided Kac Flow teacher (w 0).
NFE
100
20
4
2
I
midpoint
3.61
11.31
71.28
443.01
AB-2
3.50
11.23
56.73
190.03
443.01
Table B-6: FID for the Guided Kac Flow teacher (w 0) on CelebA-64.
CelebA-64 FID VS NFE
FID (lower better)
400
Guided Kac Flow. w-0, midpoint
Guided Kac Flow, w-0, ab2
300
DistillKac
200
0
20
40
60
80
100
NFE
Figure B-1: CelebA-64 FID vs. NFE. DistillKac substantially reduces FID at 20, 4, 2, and I steps relative to the Guided Kac Flow teacher (w 0).
B.5 MULTI-STAGE DISTILLATION SINGLE- STAGE DISTILLATION
Table B-7 shows that multi-stage distillation (S1 S2 .....- St) performs better than a single-
stage distillation (S1 + S2), where S; is the step count of the model. In Table B-7 4 + 2 +
means start from 4-step DistillKac and successively distill it to 2 and 1 steps, while 4 + means we directly distill the same 4-step DistillKac to step. Similarly, 100 - 20 - 4 + 2 + means start from Guided Kac Flow teacher with 100 steps and progressively distill it to 20, 4, 2, and steps, while 100 - means we directly distill the same Guided Kac Flow teacher to 1 step. At the distillation stage S + S1+1. the teacher substep is the quotient S (assuming S SS,, i... for each sample, the student advances one step while the teacher must simulate S S/41 steps. The runtime at this stage is proportional S Therefore, multi-stage distillation has a shorter per-stage runtime and makes hyperparameter search more tractable.
W
1.0
1.1
1.2
1.3
1.4
1.5
6.95
5.78
5.66
6.01
6.57
7.22
4 1
6.71
5.74
5.72
6.13
6.77
7.40
100
20 + 4 - 2 6.95
5.78
5.66
6.01
6.57
7.22
100
+
13.06
10.83
9.71
9.34
9.40
9.67
Table B-7: FID for multi-stage distillation and single-stage distillation.
B.6 EMPIRICAL EVIDENCE FOR ENDPOINT-TO TOOPIIINTTTO-TRAECCTOOR STABILITY
We compute the squared discrepancy between the Guided Kac Flow teacher (w 0) and DistillKac students with M E {20, 4, 2, 1} steps (See Figure B-2p. For each DistillKac student with
M steps, start from the same initial noise and both the teacher and the student. We then
17

---

Published as a conference paper at ICLR 2026
evaluate the teacher the M student time steps {ty ] k= and compute the mean, samples, of | 2v11² across these time steps, where Iy and ãg denote the teacher and student states at time step tg. As shown in Figure B-2, the mean squared teacher-student discrepancy grows t approaches O, and the 20-step DistillKac student is consistently closer to the teacher than the 4-, 2-, and 1 1-step DistillKac students. This is consistent with the andpoint-o--rraeecne stability bound in Theorem 8 once the student matches the teacher at segment endpoints, the trajectory mismatch remains controlled throughout the interval.
To further quantify this effect, we plot, over many random initial noises, the mean trajectory dis- crepancy versus the endpoint discrepancy (See Figure B-3). For each sample, the mean trajectory discrepancy is computed as the average of || &*ll over all teacher time steps {tx [= and the endpoint discrepancy is || - .x| || the final time step lx. The scattered points cluster tightly around a straight line, indicating an approximately linear relationship between trajectory and end- point errors. Empirically, endpoint mismatch almost completely explains the observed trajectory discrepancy, supporting our endpoint-only distillation scheme.
Teacher-Student Trajectory Mismatch (CelebA)
DKac20
DKac4
Squared Difference
DKac1
60
40
-30
Figure B-2: Mean squared trajectory mismatch between the Guided Kac Flow teacher (w 0) and DistillKac students with 20, 4, and steps. Here, DKac20, DKac4, DKac2, and DKacl denote DistillKac students with 20, 4, 2, and steps, respectively.
Trajectory vs Endpoint Mismatch (100 Steps)
Trajectory vs Endpoint Mismatch (20 Steps)
Mean Irajectory Difference
Mean Trajectory Difference
25
20
15
10
40
50
20
30
40
50
Endpoint Difference
Endpoint Difference
Figure B-3: Mean trajectory discrepancy versus endpoint diserepancy over 100 samples. Each blue dot corresponds to one sample, and we fit straight line to the scattered points. Left: Comparison between the Guided Kac Flow teacher (w 0) and the DistillKac student with 20 steps. The mean trajectory discrepancy is computed over 100 teacher steps. The points have Pearson correlation 0.9992 and linear regression R 0.9983. Right: Comparison between DistillKac students with 20 and 4 steps. The mean trajectory discrepancy is computed over 20 teacher steps. The points have Pearson correlation 0.9976 and linear regression I22 0.9952.
18

---

Published as a conference paper at ICLR 2026
B.7 MORE GENERATED IMAGES
Figure B-4: Uncurated CIFAR-10 10 samples generated by DistillKac (Euler integrator, 20 NFE).
Figure B-5: Uncurated CelebA-64 generations with 1, 2, and steps (NFE).
19

---

Published as a conference paper at ICLR 2026
25
2
Figure B-6: Uncurated CelebA-64 samples generated by DistillKac (AB-2 integrator, 20 NFE).
20

---

Published as a conference paper at ICLR 2026
T
2
Figure B-7: Uncurated CelebA-64 samples generated by DistillKac (Euler integrator, 4 NFE).
21


Reviews:
[{"review_id": "onX17pOfng", "reviewer": "Reviewer_wRPC", "summary": "This paper introduces DistillKac, a new framework for fast image generation based on damped wave equations and their stochastic Kac representation. Unlike diffusion models that rely on parabolic Fokker–Planck dynamics, DistillKac models generative flows as finite-speed hyperbolic PDEs, ensuring bounded kinetic energy and improved numerical stability. The authors extend this formulation with (1) velocity-space classifier-free guidance, which allows controlled generation without destabilizing the dynamics, and (2) endpoint distillation, a novel few-step training strategy where a student model learns from the teacher’s terminal distribution rather than stepwise trajectories.", "questions": "1. Can the authors provide more empirical evidence for the endpoint-to-trajectory stability theorem, e.g., quantitative correlation between endpoint and full-trajectory discrepancies?\n2. How does finite propagation speed affect perceptual smoothness or texture diversity compared to diffusion’s infinite-speed propagation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The theoretical formulation is elegant and original, bridging kinetic transport theory (via the Kac process) with modern generative modeling. Modeling probability flow through damped wave equations offers a compelling alternative to diffusion-based parabolic PDEs.\n2. The proposed endpoint distillation is conceptually simple yet powerful, backed by a provable endpoint-to-trajectory stability guarantee. This provides a rare theoretical justification for few-step model distillation.\n3. The velocity-space classifier-free guidance is well-motivated and resolves instability issues common in diffusion guidance, maintaining bounded energy across time.", "weaknesses": "1. While mathematically grounded, the physical intuition behind using wave dynamics for generative flow could be developed further — especially how finite-speed propagation impacts sample diversity or convergence in practice.\n2. The evaluation scope is modest, focusing mainly on CIFAR-10 and CelebA-64. Demonstrations on higher-resolution or more complex datasets (e.g., LSUN, ImageNet) would strengthen the empirical case.\n3. The architecture choice (UNet) and the fixed set of integration schemes limit understanding of how well DistillKac generalizes to larger backbones (e.g., DiT) or different ODE solvers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ldowsb76yV", "reviewer": "Reviewer_H4Mg", "summary": "The paper presents a fast image generator, DistillKac,  that uses the damped wave equation and its stochastic Kac representation to move probability mass at finite speed. Based on that, it introduces classifier-free guidance in velocity space that preserves square integrability and proposes endpoint-only distillation that trains a student to match a frozen teacher over long intervals.", "questions": "see weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Explore a new equation form (damped wave equation) for the diffusion process.\n2. Provide a solid and detailed proof of the formulation, including the error bounds.", "weaknesses": "1. If I’m getting this right, the comparison in Figure 2 is made against normal sampling, specifically the non-distilled version of the model at the same number of steps. But doesn’t that seem a bit unfair? Plus, similar distillation methods can also achieve just a slight increase in FID. How does that show that your distillation method is better? It might be helpful to include comparisons with other distillation methods.\n\n2. In the related work section, you only mention a few papers that focus on CIFAR-10 and CelebA-64, but there’s actually a lot of literature on distillation in latent space that has algorithms similar to yours. For instance, the skipping steps in LCM[1] have the same meaning as N in your Algorithm 1. It would be great to see some mention and analysis of those similarities.\n\n3. While the proof goes into detail about the error and stability bounds, the experiments don’t seem to show better error and stability compared to other distillation methods. This leaves us unsure about the advantages of your proposed distillation method in practice.\n\n[1]Luo, Simian, et al. \"Latent consistency models: Synthesizing high-resolution images with few-step inference.\" arXiv preprint arXiv:2310.04378 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pLGHD9mOCL", "reviewer": "Reviewer_FCnC", "summary": "DistillKac presents an image generative model grounded in the damped wave equation, a hyperbolic PDE represented stochastically through the Kac process. Unlike diffusion-based ODEs or SDEs, whose velocity norms can diverge near the terminal time, Kac dynamics maintain globally bounded kinetic energy and Lipschitz continuity in Wasserstein space. The main contributions include adapting classifier-free guidance directly in velocity space while preserving bounded-energy guarantees, and introducing an endpoint-only distillation scheme with a theoretical stability bound linking endpoint consistency to full-trajectory alignment. Empirically, these features enable stable few-step sampling with competitive image quality, and theoretically, the work clarifies why endpoint matching is sufficient under finite-speed Kac dynamics.", "questions": "Overall, the work would benefit from comprehensive runtime and stability analyses to substantiate that the proposed Kac flow offers practical, measurable improvements beyond theoretical guarantees.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is clearly written and presents solid theoretical foundations with rigorous stability proofs. \n- Adopting a hyperbolic PDE instead of the conventional diffusion formulation offers a novel structural perspective in generative modeling, characterized by finite propagation speed and bounded velocity. \n- The introduction of Kac flow–based generative modeling provides an innovative alternative to diffusion approaches, and the inclusion of stability guarantees and asymptotic links to diffusion broadens the methodological toolkit for finite-speed generative flows.", "weaknesses": "- While this paper provides solid theoretical formulations and formal stability proofs, it lacks empirical analysis demonstrating how Kac flow concretely improves stability in practice. Theoretical claims of bounded energy and finite propagation speed are not supported by detailed quantitative studies.\n- Although Table 1 attempts to ensure fairness by matching the number of function evaluations (NFEs) across methods, the comparison would be more convincing if it also included training time, FLOPs, or wall-clock cost, since these reflect actual computational efficiency.\n- The paper does not show whether the claimed stability advantages translate into observable robustness improvements, such as reduced divergence, fewer NaNs, or more consistent quality under large classifier-free guidance scales or few-step sampling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CPAUGKw2dk", "reviewer": "Reviewer_eWqj", "summary": "The authors propose DistillKac a strategy for distillation for generative models whose densities evolve with the damped wave equation. This parameterization should remove the singularities at the end points which is common in diffusion/flow models. This achieved by fixing the propagation speed to some bounded value. The authors then propose a distillation strategy for this existing family of models.", "questions": "1. Why do you refer to the random paths and evolution of the densities as the *Kac process* and *Kac flow* (resp.)? On line 117 you say when $d \\geq 2$ such processes are called *random flights*. Aren't you working with $d \\geq 2$ objects? The terminology is a bit confusing\n2. How does this work relate to [1]?\n3. Do Theorem 8 and Corollary 9 apply to any flow maps or only those admitted by the *Kac process*?\n4. Likewise, how does Theorem 8 compare to [2, Theorem 4.1] outside of W1 vs W2?\n5. How do you justify the coordinate-wise independence?\n6. Please address any questions raised in the weaknesses sections.\n\n\n[1] Skreta et al., 2025, Feynman-Kac Correctors in Diffusion:\nAnnealing, Guidance, and Product of Experts, ICML, https://arxiv.org/pdf/2503.02819\n\n[2] Dou et al., 2024, Provable Statistical Rates for Consistency Diffusion Models, https://arxiv.org/pdf/2406.16213", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "I liked the detailed maths and found the Kac representation to be interesting.\n\n* Theorem 8 and Corollary 9 seem interesting tools for analysis in the literature on (consistency) distillation.\n* Theorem 4 seems novel within the context of CFG\n* Addressing the problems of singularities in diffusion/flow models is important", "weaknesses": "## Primary concerns\n### Theory\n* How does this work clearly differentiate from [1]? It is unclear where [1] ends and where the contributions of this work begin. My rough understanding is that this work is [1] plus Theorems 4, 8 and 9 with distillation?\n* I'm surprised that learning 1-D independent coordinate processes yields anything useful. It seems strange that this works as there is often interactions between the different coordinates. This seems to be an **incredibly strong** assumption to make.\n* How do you initially train the teacher? Flow matching and diffusion models are nice because of simulation-free training, it is not clear to me that the Kac representation yields this for the teacher.\n* Also since in Section 2.2 the process $X(t)$ is defined as a jump process it doesn't seem right to use standard numerical schemes for ODEs (like Euler or Heun) to integrate teacher model as suggested in line 203.\n* Also since the Kac process is stochastic why are the flow maps governed by an ODE, shouldn't the flow maps be random variables *a la* [3]?\n\n### Empirical\n* Both CIFAR-10 and CelebA-64 are quite small and generally considered toy datasets, in particular since one of the contributions is improved CFG I feel like more compelling experiments for guidance should be included than just CIFAR-10 (32 x 32).\n* The empirical results don't look great for Kac Flows. The traditional diffusion models seem to still outperform them despite the singularity issues, so what is the advantage? This seems especially true when compared to consistency models.\n* Overall, the empirical results are wholly **uncompelling** in convincing the reader of the advantages of Kac flows over standard flow/diffusion models.\n\n## Minor comments / suggestions\n* Table 1 should get reworked it is unclear what each row corresponds to. Some labels would be nice, e.g., PDE, SDE, papers.\n* Since there is a mixture of prior results in the theoretical sections, maybe highlight your contributions with a colored box?\n* FID is a notoriously fickle and unwieldy metric to work with. Recent papers looking at text-guided generation [2] have included CLIP-score  [4] and Image Reward [5].\n\n## References\n[1] Richard Duong, Jannis Chemseddine, Peter K Friz, and Gabriele Steidl. Telegrapher’s generative\nmodel via kac flows. arXiv preprint arXiv:2506.20641v3, 2025.\n\n[2] Skreta et al., 2025, Feynman-Kac Correctors in Diffusion:\nAnnealing, Guidance, and Product of Experts, ICML, https://arxiv.org/pdf/2503.02819\n\n[3] Kunita, Hiroshi. Stochastic flows and jump-diffusions. Springer Singapore, 2019.\n\n[4] Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" International conference on machine learning. PmLR, 2021.\n\n[5] Xu, Jiazheng, et al. \"Imagereward: Learning and evaluating human preferences for text-to-image generation.\" Advances in Neural Information Processing Systems 36 (2023): 15903-15935.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)