From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training
Authors: Julius Berner
Lorenz Richter
Marcin Sendera
Jarrid Rector-Brooks
Nikolay Malkin
Venue: Accepted by TMLR ()

Abstract: We study the problem of training neural stochastic differential equations, or diffusion models, to sample from a Boltzmann distribution without access to target samples. Existing methods for training such models enforce time-reversal of the generative and noising processes, using either differentiable simulation or off-policy reinforcement learning (RL). We prove equivalences between families of objectives in the limit of infinitesimal discretization steps, linking entropic RL methods (GFlowNets) with continuous-time objects (partial differential equations and path space measures). We further show that an appropriate choice of coarse time discretization during training allows greatly improved sample efficiency and the use of time-local objectives, achieving competitive performance on standard sampling benchmarks with reduced computational cost.

Published in Transactions on Machine Learning Research (01/2026)
From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training
Julius Berner
jbernerenvidla.com
California Institale of Technology
NVIDIA
Lorenz Richter"
richter@zib.de
Zuse Institute Berlin
dida Datenschmiede GmbH
Marcin Sendera"
marcin. sendirrsnd edu..
Jagiellowiaa Universily
Mila, Unáversité de Montréal
Jarrid Rector-Brooks
jartid-rctoo--rrokkhhhl quebec
Mila, Université de Montréal
Dreamfold
California Institate of Technology
Nikolay Malkin
nmalkin@ed,
University of Edinburgh
CIEAR Fellow, Learning in Mochines and Brains
Reviewed on OpenReview: https: // openreview. net/ forum? id= Jd=DLD3aJUwD0
Abstract
We study the problem of training neural stochastic differential equations, or diffusion models, to sample from Boltzmann distribution without access to target samples. Existing methods for training such models enforce time-reversal of the generative and noising processes, using either differentiable simulation off-policy reinforcement learning (RL). We prove cquivalences between families of objectives in the limit of infinitesimal discrctization steps, linking entropic RL methods (GFlowNets) with continuous-Lime objects (partial differential equations and path space measures). We further show that an appropriate choice coarse time discretization during training allows greatly improved sample efficiency and the use of time-local objectives, achieving competitive performance on standard sampling benchmarks with reduced computational cost.
1 Introduction
We consider the problem of sampling from distribution on Rd with density Ptarget; which is described by unnormalized energy model Ptarget(x)) (x) exp(-E(z))/Z with Z =f² exp(-E(x)) dr. We have access to E but not to the normalizing constant Z or to samples from Pearget This problem is ubiquitous in Bayesian statistics and machine learning and has been an object of study for decades, with Monte Carlo methods (Duane et al., 1987; Roberts & Tweedie, 1996; Hoffman et. al., 2014; Leimkuhler et al., 2014; Lemos et al., 2023) recently being complemented by deep generative models (Albergo et al., 2019; Noé et al,, 2019; Gabrié et al., 2021; Midgley et al., 2023; Akhoun-SSaogh al., 2024).

---

Published in Transactions on Machine Learning Research (01/2026)
Building upon the success of diffusion models in data- driven generative modeling (Sohl-Dickstein et al., 2015; Ho et al., 2020; Dhariwal & Nichol, 2021; Rombach et al., 2021, inter alia); recent work (e.g., Zhang & Chen, 2022; Berner et al., 2022; Vargas et al., 2023; Richter Berner, 2024; Vargas et al., 2024; Sendera et al., 2024) has proposed solutions to this problem that model generation as the reverse of a diffusion (noising) process in discrete or continuous time (Fig. 1). Thus Ptarget is modeled by gradually transporting samples, by a sequence of stochas- tic transitions, from a simple prior distribution Ppeior (e.g., a Gaussian) to the target distribution. When a dataset of samples from Ptarget is given, diffusion models are trained using a score matching objective equivalent to variational bound on data log- likelihood (Song et al., 2021a). The problem is more challenging when we have samples but can only query the energy function, as training methods necessarily involve simulation of the generative process. (We survey additional related work in SA.)
Contimuugs lime processes
Generative process
Diffusion process
Neural SDE
Reverse SDE
Paths Fx
Patls Y Qy
Ruler-Marnyyma discretization
Contavoos-tmme limit Eulet-Marnyans (Prog. 3.1)
Continaoustime limit (Prop. 3.13
discretization
Policy 7
dojeetaves
Policy rev. MDP
Trajectorios 8 8~ 3x
Trajectories y 4g
Discrste-line processes
The problem of making contimuous- forward and reverse processes determine the same path space measure is approximated by matching distributions over discrete-time trajectories.
Figure 1:
In continuous time, we assume the generative process takes the form of a stochastic differential equation (SDE) (with initial condition Pprior and diffusson coefficient o):
dX, = (Xe, dt + (t) Xo Pprior.
(1)
When the drift j is given by a parametric model, such as neural network, (1) is called a neural SDE (Tzen & Raginsky, 2019; Kidger et al., 2021a; Song et al., 2021b). The goal is to fit the parameters as to make the distribution of X, induced by the initial conditions and the SDE (1) close to Ptarget
In discrete time, we assume the generative process is described by a Markov chain with transition kernels 78(Nn11 | X,),, n = N, - 1, and initial distribution X, ~ Pprior. The goal is to learn the transition probabilities से. as to make the distribution of 8N close to Ptarget- This is the setting of stochastic normalizing flows (Hagemann et al., 2023), which are, in turn, a special case of (continuuus) generative flow networks (GFlowNets; Bengio et al., 2021; Lahlou et al., 2023).
Training objectives for both the continuous-time and discrete-time processes are typically based on mini- mization of bound on the divergence between the distributions over trajectories induced by the generative process and by the target distribution together with the noising process. These objectives may rely on differentiable simulation of the generative process (Li et al., 2020; Kidger et al., 2021b; Zhang & Chen, 2022) or on off-policy reinforcement learning (RI.), which optimizes objectives depending on trajectories obtained through exploration (Nüsken & Richter, 2021; Malkin et al., 2023). Objectives may further be classified as global (involving the entire trajectory) or local (involving a single transition). Common objectives and the relationships among them are summarized in Fig. 2.
Any SDE determines discrete-time policy when using a time discretization, such as the Euler-Maruyaan integration scheme; conversely, in the limit of infiiitesimal time steps, the discrete-time policy obtained in this way approaches the continuous-time process (Kloeden & Platen, 1992). The question we study in this paper is how the training objectives for contimoous-imme and discreto-time processes are related in the limit of infinitesimal time steps. We formally connect RL methods to stochastic control and dynamic measure transport with the following theoretical contributions:
(1) We show that global objectives discrete time converge to objectives that minimize divergences between path space measures induced by the forward and reverse processes in continuous time (Prop. 3.3).
(2) We show that local constraints enforced by GelowNet training objectives asymptotically approach partial differential equations that govern the time evolution of the marginal densities of the SDE under the generative and noising processes (Prop. 3.4).
These results motivate the hypothesis that an appropriate choice of time discretization during training can allow for greatly improved sample efficiency over the standard practice of training diffusion samplers in
2

---

Published in Transactions on Machine Learning Research (01/2026)
Globel consistencyy
Local consiatency
Global KL divergence
Path measure proportionality
Reversal formula
Density evolution
Eignuiient] w
Path KL (15)
2nd momentt
/ VarGrad
PP,
Nelson's is ideatity (12)
to V-p
Fokker- PDEs (10)
log 4t
Richter at al,
[log
log 2)
o²v logp
Fod bord
Agp -- (-(T))) A Ap - (-(5T)) 5.
At + 0 limis (Prop. 3.2)
Ad limis (Prop, 3.31)
Vantshing Olv'dr)
Vanishing to D122)
Riiinaan sum
Riiisssan sum
aue. Faeei dn (Prop. 3:4)
R.i.. it, (Prog. 3:41
Eggadiout]] W
Trajectory KL (3)
TB / VarGzad (7)
GFN theoey
Detailed balance (8)
B3 log di
Ridhter al.
Bgg
log di eq
lop 2)
Bengio
log (2.) 7 aflaa [x,]] - log [Potz (stnes) To
ats (oa smsill
Lablon
Ou-polle policy objectives
Og policy objecticns
Training objectives for neural SDEs (top row) and their approximations by objectives for discrete-time policies (bottom row). On-policy objectives minimize a divergence by differentiating through SDE integration, while off-policy objectives enforce local or global consisteney constraints. Our results explain the behavior of discrete-time objectives as the time discretization becomes finer.
Figure
fine, uniform time discretization. Training with shorter trajectories obtained by coarse time discretizations would further allow the use of time-local objectives without the computationally expensive bootstrapping techniques that are necessary when training with long trajectories. Coufirming this hypothesis, we make the following empirical contribution:
(3) In experiments on standard sampling benchmarks, we show that training with nonuniform time dis- cretizations much coarser than those used for inference achieves similar performance to state-of-the-arl methods, at a fraction of the computational cost (Fig. 6).
2 Dynamic measure transport in discrete and continuous time
Recall that our goal is to sample from a target distribution Ptarget = exp(-E(x)) given by a continuous energy function E gd + IR To achieve this goal, we present approaches using discrete-time policies in the framework of Markov decision processes (MDPs) in $2.1 and continuous-time processes in the context of neural SDEs in $2.2. In particular, we will draw similarities between the two approaches and show how time discretizations of neural SDEs give rise to specific policies in MDPs in $2.3. This allows us to rigorously analyze the asymptotic behavior of correspouding distributions and divergences in $3. Note that our general assumptions can be found in $B.1.
Our exposition synthesizes the definitions for MDP policics (Bengio et al., 2023; Lahlon et al., 2023), results on neural SDEs for sampling (Richter & Berner, 2024; Vargas et al., 2024), and PDE perspectives (Máté & Fleuret, 2023; Sun et al., 2024). The results in $3 extend classical results on SDE approximations (see, e.g., Kloeden & Platen (1992)) to objectives for reffusiio-basedd samplers.
2.1 Discrete-time setting: Stochastic control policies
((xtt))) ((x,11))
((r,t))
A discrete-time Markovian process X with density PP(X)-
a distribution over RRd-valudd variables 8o...., ....,,xx &x can be identified with a policy 7 in the deterministic Markov decision process (MDP) (S,A, R) depicted in Fig. 3, given by
Porior
-o
x x-1
-((x))
7(a 0) = P( a) Pprior (a),
Xa
X
8y
(2) Figure The MDP and policy representing the pro- cess IP, a distribution over = (Xo...,,Xy, We sometimes write 7n 2)) for नें (- (z, t,)) for convenience. We relegate formal definitions $B.2; in short, the states are pairs of space and time coordinates (x,, (.)) (together with abstract initial and terminal
na (z. (.) = P(8
8, 2).
3

---

Published in Transactions on Machine Learning Research (01/2026)
states), actions represent steps from Xx to Xn+1 (taking action leads to state (a,t bn+1)), and the reward for terminating from a state (x,4x) is set to -E(x). The learning problem is to find 7 whose induced distribution over Xy is the Boltzmann distribution of the reward.
This learning problem differs from that of standard reinforcement learning, where one wishes to marimize the expected reward with respect to a policy in the MDP. However, it is close to the setting of maximum-entre entropy reinforcement learning (MaxEnt RL; Ziebart, 2010), in which the objective includes the policy entropy as a regularization term. With such regularization, the optimal policy samples the Boltzmann distribution of the negative reward, which is the target distribution in our setting. This observation is the basis for the connection between control (policy optimization) and inference (sampling) (Levine, 2018). (See Eysenbach & Levine (2022) for the connection to KI-constrained adversarial policy optimization, Zhang & Chen (2022); Domingo-Entn Emrich et al. (2025) for the related derivation of sampling via stochastic control with a diffusion policy, and Tiapkin et al. (2023); Deleu et al. (2024) for the connections between MaxEnt RL and off-policy objectives for training samplers of discrete sequentially constructed objects. In short, the time-local detailed balance objective (8) minimizes the error in a soft Bellman equation, while trajectory-level objectives (7) minimize a path consistency objective (Nachum et al., 2017).)
Distributions over trajectories. The possible trajectories in the MDP starting al. and ending in J. have the form (Irtu to) - ..... ( (x) L,, which we abbreviate Iia + Itz the policy 7े for N + steps starting at yields distribution over trajectories Ita - It1 + Ftys L.C.,
FiN- Following
N-1
N-1
p( = P(x)
P(X (8,) = Pprior (xo)
( Xn). ==
II
I
(3)
n=
The same construction is possible in reverse time: density Ptarget over XN and policy & (analogously to (2) defining transition probabilities from X441 1 to Xx) on the reverse MDP yields Markovian distribution over trajectories Q, given analogously to (3) in reverse time. Given (forward) policy, the reverse policy generating the same distribution over trajectories can be recovered using the marginal state visitation distributions via the detailed balance formula (8).
Radon-Nikodym derivative and divergences. The distributions P., Q determined by a pair of policies 7, & and densities Ppriors Pearget allow us develop divergences (Josses) for learning the parameters of suitable parametric families of policies. Our goal is to make the forward and reverse processes approximately equal by minimizing divergence between the distributionn over their trajectories. The densily ratio of these distributions, also known as Sauoon-Nikoyyn derivative, is given by
dr dQ (X)
P(X) = P( PP( 8n) (8)
i (Rat1 8nn) I ({ 8n+1) Ptarget ( (xx) IT n (8n xn-1)
Pprior (8) - T Rn
(4)
(S
Using (4), we can write the Kultback-Leide (KL) divergence
dp log (8)
DKI. ( ②) = Ex
dQ
(5)
N-1 नें 2n(8nt1 8n)
=E =E x
log Pprior ( (2) + E(xN) + log
+ log Z.
==0 & F &x+1)
Since log 7 is constant, this expression can be minimized via gradient descent on the parameters of the policies, for instance by zeroth-order gradient estimation (REINFORCE; Williams (1992)). If the policies allow for a differentiable reparametrization as a function of noise (e.g., if they are conditionally Gaussian) we can use a deep reparametrization trick, amounting to writing the KL as a function of the noises introduced at each step. In particular, by fitting the parameters of से and for so that the two processes are approximate time-reversals of one another, we also get an approximate solution to the sampling problem, f.e., XN is approximately distributed as the target distribution Ptarget. This can be motivated by the data processing inequolity, which yields that
DRL. (xx)) Ptarget (xx)) DRL Q)
(6)
4

---

Published in Transactions on Machine Learning Research (01/2026)
We can also consider other divergences between two measures P and Q. For instance, the trajectory balance (TB, also known as second-momend, Malkin et al. (2022); Nüsken & Richter (2021)) and related log-variance
(LV, also known as VarGrad, Richter et al. (2020) divergences are given by
(P. = E BR ( dQ
dP (X)
)
o) 27
dr (8). dQ
and D P, Q) = Var Xnri
log
(7)
where the density ratio inside the square is given by (4) and W is a reference measure. We are free in the choice of reference measure, which allows for exploration in the optimization task (in RI, this is called off-policy training). We note that the second-moment divergence in (7) requires either the choice of a constant about which the moment is computed, which can be taken to be the normalizing constant log Z of Ptarget: if it is known, or a learned approximation. The LV divergence coincides with TB when e is taken to be batch-level estimate of log Z (see, e.g-, Malkin et al. (2023, $2.3)) While estimators of the two divergences in (7) have different variance (which is related to basetines in RL), the expectations of their gradients with respect to the policy of P coincide when W = P and are then, in turn, equal to the gradient of the KL divergence (5) (Richter et al., 2020; Malkin et al., 2023). In $2.2, we will see that one can define analogous concepts in continuous time.
Local divergences. Instead of looking at entire trajectories, we can as well define divergences locally, i.e., on small parts of the trajectories. To this end, one can define the so-called detailed balance (DB) divergence as
Pa (Rn) (xn+1 Xn) ( &or (Xn Xn+2) )
DB. (P. Q.P) =Exa log
(8)
for the time step n, where Pm is learned estimate of the density of Xt for 0 < n < N, while Po Pprior and PN Ptarget are fixed. Minimizing the DB divergence enforces that the transition kernels से and & of P and Q. respectively, are stochastic transport maps between distributions with densitics Pn and pn-1. for each n. If the policies and density estimates jointly minimize (8) to 0 for some full-support reference distribution W and all n, it can be shown that they also minimize the trajectory-level divergences (7); see Bengio et al. (2021) for the discrete case, Lahlou et al. (2023) for the contus case, Malkin et al. (2023, SC.1) for the connection to nested variational inference (Buchner, 2021), and Deleu & Bengio (2023) for the connection to detailed balance for Markov chains. The divergence used for training may be a (possibly weighted sum of the DB divergences (8) for n O. ......N- N 1. 'Subtrajectory' interpolations between the global TB objective (7) and the local DB objective (8) exist: see $B.4 and Nüsken & Richter (2023).
Uniqueness of solutions. Learning both the generative policy न and the time-reversed policy f in the general setting as above leads to non-unique solutions. We can achieve uniqueness of the objectives by prescribing &o (as in diffusion models), adding additional regularizers (as in Schrödinger (bride)))idggs), or prescribing the densities (f((8))) N-1 and imposing constraints on the policies (as in annealing schemes); see Blessing et al. (2024, Tables 6 & 7) and Sun et al. (2024, Table 1).
2.2 Continuous-time setting: Neural SDEs
We consider neural stochastic differential equations (neural SDEs) with isotropic additive noise, i.e., families of stochastic processes X = (X.)) given as solutions of SDEs of the form
dx, = 7X,,, () di + dW),
Xo ~ Ppriors
where 7h: Rd x (0, + Rd is the drift (or control function), parametrized by a neural network?: a: [o, 1] - Ry0 is the diffusion rate, which in this paper is assumed to be fixed (more generally, it could be a d x d matrix that depends also on X2); and W, is a standard d-dimensional Brownian motion. Using a time discretization, the drift H, together with the noise given by the diffusion rate and the Brownian motion, can be connected to policy से of MDP, which can be sampled to approximately simulate the process X (see $2.3).
OOur result Prop. 3.4 suggests weighting of NA in the notation of $2.3, but our experiments showed no significant. difference between such weighting and uniform one. For notational convenience, we da not make the dependence of X the neural network parameters explicit.
5

---

Published in Transactions on Machine Learning Research (01/2026)
Distributions over trajectories. Similar to the previous section, we can define measure the trajectories of the process X. Since the trajectories f. + Xx are almost surely continuous, the distribution (also known as law or push-forward) of the process X defines path space measure IP, which is a measure on the space C(O0, 1], R2) of contimuous functions, representing the distribution of trajectorics of X. We will show in $2.3 that such a path measure can be interpreted as the limit of distributions over discrete-time trajectorics as in (3) when the step-sizes 1x+1 - in tend zero.
We can also define the time marginals p: R [0. R, where for each time t E [o, 1], p(::t) gives the density of Xt. In measure- thherretc notation, the time marginals are the densities of the pushforwards of the path measure P by the evaluation maps X + X₁ sending a continuous function (trajectory) to its value at time t. Thus, we will also denote the distribution of the time marginals by Pr. The evolution of p is governed by the Makker-Planch equation (FPE), which is the partial differential equation (PDE)
0² 0.p - (p7)) Ap, p): Pprior:
(10)
where Ap denotes the Laplacian of p. The Bookeer-Plancee equation generalizes the continuity equation for ordinary differential equations, which corresponds the case o O. It expresses the conservation of probability mass when particles distributed with density p(:,t) stochastically transported by the drift 7 and diffused with scale a. While such a PDE perspective is only possible in continnous time, in $3 we derive that certain MDPs satisfy FPEs in the limit of finer time discretizations.
Reverse process. As for reverse-time MDPs, we can also define reverse-time SDEs
dX, = (X,, dt + o(t) dW., X1 ~ Ptarget:
(11)
where We is a reverse-time' Brownian motion and Tú is suitable drift, potentially also parametrized by a neural network. This SDE gives rise to another path space measure Q). While in discrete time ($2.1) local reversibility is given by detailed balance (8), in continuous time one can characterize when the path space measure Q of the reverse-ting SDE in (11) coincides with the path space measure of the forward SDE in (9) by a local condition known as Nelson's 's identity (Nelson (1967), also attributed to Anderson (1982)), which states that Q = P if and only if
I = o² V'V logp and
Q1 = P1,
(12)
where p denotes the densities of P's time marginals. It can be shown that substituting this expression into the FPE for the backward process recovers the FPE (10) for the forward process, and similarly that the KL divergence, given by (15) below, between the forward and backward processes is zero.
Radon-Nikodym derivative and divergences. Since we typically cannot compute the time marginals, we cannot directly use Nelson's identity to solve the sampling problem. However, similar to $2.1, we can establish learning problems to infer the parameters of the neural networks 7Ý. TT, so that the induced terminal distribution of the forward SDE (9) is close to the target, P1 & Ptarget, in some suitable measure of divergence.
The tool to define such learning problems is Girsanov's theorem (see, e.g., Särkkä & Solin (2019, Theorem 7.4) and Sottinen & Särkkä (2008, Theorem A.2)), which states the following. Tet P()) and IP(2) be the path space measures defined by SDEs of the form (9) with drifts (7(1), (12). Then, for pl?)-s every X E C((0, 1), Rd), the Radon-Nikodym derivative is given by
d1 log
/''
1 (Xe, 12 17 (X,, 12 dt + S' (X,,) ( 2t{2)2
d2 (X)
dx,- (13)
att)²
An intuitive explanation of (13) using a discrete-time approximation can be found in Särkkä Solin (2019, $7.4) or in the proof of Lemma B.7. The same result holds for reverso-time processes as (11) with dX, replaced by integration against the reverse-time process dx,,. Using a reversible Brownian motion as a
We refer to Kunita (2019); Vargas al. (2024) for details on reverse-time SDFs and backward Itô integration.
6

---

Published in Transactions on Machine Learning Research (01/2026)
reference path measure (see Léonard (2014; 2013)), we thus express the Radon-Nikodym derivative
between the path measures IP and Q of the forward and reverse-time SDFs in (9) and (11) Itô integrals:
dP (X) = log Pprior (X)) log dQ
(X,, 1)1 (X,,
+ 0
dt
Ptarget (X1)
(t)
(14)
(X,,t) o())
dx, L'" (X,,t) ot)²
dx
see (Vargas et al., 2024, Prop. E.1). A related result was derived by Richter & Berner (2024, Prop. 2.3, Remark A.1) using the conversion formula So f(X,,)) t) dX, fo f(X,,t) -xx, So (t) f(),, dt. By integrating (14) over X ~ P, one derive that the KI. divergence is given by an expression analogous to (5):
(I, Q) = Ex-p log Pog (Xo) ( L. (X,,) T(. -V di
al
+log Z. (15)
20(1)2
Informally; the derivation uses that in expectation over X P, the integral with respect to dX, in (14)
is the sum of an integral with respect to 7 (X²) dt and stochastic integral with zero expectation.
The KL divergence can also be interpreted as the cost of contimuous-time stochastic optimal control problem (Dai Pra, 1991; Berner et al., 2022). Some objectives, such as those in Zhang & Chen (2022), optimize the parameters of the drift defining P by minimizing variants of the KL divergence (15) approximately: by passing to a time discretization of the SDE ($2.3) and expressing the objective as a function of the Gaussian noiscs introduced at each step of the SDE integration, amounting to a deep reparamctrization trick. For suitable integration schemes (Vargas el al., 2023; 2024), the discretized Radon-Nikodym derivative can be written density ratio, that this approach corresponds to optimizing discrete-time KL as in (5).
Analogously to the sierete-time setting (7): we can also consider the second-moment or log-variance divergences DWW (P, Q) = Ex-w (1og 18 (x)) 27 and (TP, Q) = Varx>w [log dP (x)] where W is a nq
reference path space measure. These divergences were explored by Nüsken & Richter (2021).
Local time reversal: PDE viewpoint. The continuous-tine perspective also offers to employ the PDE framework for learning the dynamical mcasure transport. Recall that the density p of the process X defined in (9) fulfills the Fokker-Planck equation (10). One can thus aim to learn 1/
to make it satisly the FPE, with the boundary values p(:, 0) Pprior and p(-, 1)) = Ptarget, where p is either prescribed or also learned (as done in Máté & Fleuret (2023)). In Sun et al. (2024, $3.3 and $A.3) it is shown that when using suitable losses on this problem one recovers a loss equivalent to DrB. When choosing the diffusson loss from Nüskkn & Richter (2023), one recovers continuous-lime variant of DSubTB (see BB.4) and thus DDB- In $3, we show that it also works the other way around: we can start with the discrete-time detailed balance divergence and derive PDE constraints in the limit.
2.3 From SDEs to discrete-time Kuler-Maruyane policies
Simulation of the process X can be achieved by discretizing time and applying a numerical integration scheme, such as the Euler- Marly-Maruyama scheme (Maruyama, 1955). Specifically, one fixes sequence of time points
to < t < tN- and defines the discreto-time process X = (Xa) by
Xo ~ Ppriors X = Xx, + 78 ((8nttn) + (tn)) / Atn En. En &n~N(O,L)
(16)
where Al., bx+1 In- This defines the policy 7(a| (x, 4x)) Naa (1, AM, on an MDP in (2). It is clear by comparing (2) and (16) that this distribution exactly coincides with the distribution PP in (3) over sequences (X,, XI...., XN) of the aalle--Mrryam-ddiseetteee will discuss below, with decreasing mesh size, the marginals P(X,,) of the n-th step of the discretized process converge to the marginals t,)) t,)) of the continuous-imme process at time in- Basod on the Central Limit Theorem, such convergence can also be shown for non- non-Gaussian policies that satisfy suitable consistency conditions (Kloeden & Platen, 1992 $6.2).
process X. As we
7

---

Published in Transactions on Machine Learning Research (01/2026)
Finally, the same discretization is possible for reverse time: reverso-time process of the form (11) with drift function 77 together with target density Ptarget determine policy fo on the reverse MDP, corresponding to reverse Suler-Maruyama integration:
NN Ptargets X, 8nt1 To (X (e4tl,,+4444 1)) ((lx+t) Aln En ~~~N10,
(17)
However, note that the Buler-Maruyanr discretizations of a process and of its reverse-time process defined by (12) do not, in general, coincide. That is, policy on the reverse MDP can be constructed either by discretizing an SDE to yield policy on the forward MDP, then reversing it, by discretizing the reverse SDE to directly obtain policy on the reverse MDP, possibly with different results. In particular, the Gaussianity of transitions is not preserved under time reversal: the reverse of a discrete-time process with Gaussian increments does not, in general, have Gaussian increments. However, Nclson's idontity (12) shows that the two are equivalent the continuous-Lime limit.
The discretization allows us compare the Radon-Nikodym derivatives: those of the discretizations in (4) and of the continuous-time time processes in (14). In particular, in Lemma B.7 we will show that these expressions are equal in the limit.
3
Asymptotic convergence
3.1 Distributions over trajectories
A standard result shows that the discretized process X converges to the continuous counterpart X as the time discretization becomes finer, i.e., as the maximal step size max N-1 At, goes to zero (Maruyama, 1955). The precise stalement of convergence requires the processes to be embedded in a common probability space. Let be the mapping from the observation space of X (discrete-time trajectories) to that of X (continuuus-the paths) that takes sequence X,....,Xe Xn, to the function f E C(O0, 1], R²) defined by f(t,) = X, and linearly interpolating between the t, (note that implicitly depends on the discretization). We then have convergence of ((X) to X:
Proposition 3.1 (Convergence of Fuler-Maruyama scheme). As max N-1 Al, + 0, ((8) converges weakly and strongly to X with order y 1 and the path measures IP converge weakly to IP.
We refer the reader to $B.3 for definitions of strong and weak convergence. The result can be found in Kloeden & Platen (1992, $10.2, $10.3) and we refer to Baldi (2017, Corollary 11.1) and Kloeden & Neuenkirch (2007, $2.1) for the convergence of path measures. Generally, the Euler-Maruyana scheme has order of strong couvergence 7 1/2. However, since we consider additive noise, í.e., a not depending on the spatial variable z, the Milstein scheme reduces to the Euler- Maler--aaaaaaa scheme and we inherit order 7 = as stated in Prop. 3.1 (Kloeden & Platen, 1992).
3.2 Radon-Nikodym derivative and divergences
Beyond the convergence of path measures, this section shows more relevant for practical applications that commonly used local and global objectives converge their continuous-lime counterparts as the time discretization refined. To this end, we leverage Lemma B.7, which analyzes the convergence of time discretizations of Radon-Nikodym derivatives dP appearing in (14) to their discrete-time analogs 4P We aip
note that Vargas al. (2024, Proposition E.1) shows that, for constant a, an Buler-Maruyama discretization density ratio as in (4). This also implics that the ratio in the detailed balance
of dP aq can be written divergence in (8) arises from a single-step Euler-Maruyama approximation of the Radon-Nikodym derivative aq
dP on the subinterval [tn, tn1]. We present proofs of all results this Section in SB.6.
8

---

Published in Transactions on Machine Learning Research (01/2026)
Global objectives: Second-moment divergences approach the continuous-time equivalents. The following key result. uses convergence of the Radon-Nikodym derivatives (Lemma B.7):
Proposition 3.2 (Convergence of functionals). If Q2 W are path measures of three forward-time SDEs, the drifts of the SDEs and their derivatives bounded, and f C (R, R) has at most polynomially growing dericalives, then
E xx { (105
dPP a dd)
dIp mANA den -0, Exwer f (105 dQ (x)
(x))
We now show that the recond-momen losses in (7) converge their continuous- counterparts. Proposition 3.3 (Asymptotic consistency of TB and VarGrad). Under the assumptions of Prop. 3.2, the diveryences D PP. Q and D (IP, Q) converge D (I, Q) and D (I, Q), respectively.
The convergence holds for the TB divergence in (7) with respect to any c, i.e,, E [log dP dQ
- showing
that Prop. 3.3 continues hold if one uses a learned estimate of the log-partition function log 2 as the scalar the TB divergence, as typically done in practice.
Consider pair of forward and reverse SDEs with drifts 1l and μ1, respectively, defining processes IP and Q, and suppose that P: Rd [0, + I is density estimate with ()-,0) = Pprior and p(:, Ptarget-
Local objectives: Detailed balance approaches the Fokker-Planck PDE.
For < < l' < 1, consider any time discretization in which & and L' are adjacent time steps (4, = / and fn+1 = t'). The discretization defines a pair of policies 7, & corresponding to aaler-Maruyama discretizations of the two SDEs. Let define the detailed balance discrepancy:
(x(x) Th (x' xx)
AL-U(1, = log
(18)
Pn++ (x') n+1
1x x²)²
where we set (x) = ((x, tn). Recalling the definition (8), we have that
D DB. Q.D) Bz
A -14 (Zno 2n+z)?]
(19)
The following proposition will show that the two SDEs are time reversals of one another if and only if certain asymptotics of the DB discrepancy vanish. It is proved using technical lemma (Lemma B.8), which shows that the asymptotics of the discrepancy in h are precisely the errors in the satisfaction of Nelson's 's identity and the Fokker-Plande equation.
Proposition 3.4 (Asymptotic equality of DB and FPE). Under the smoothness conditions in Lemma B.8, nh, F. joinlly salisfy Nelson's identily (fr 1t V²V log at (x,,1) if and only if
Ar (Fe, 2eHn) = for almost every 2,
lim h
Vh
where F4th Ft (1,, + o(t) Vhe. If addition
lim E. Ex-A( h-
(et, =0,
then the Fokker-Plant Plank equation is satisfied at (z,,t). If both conditions hold at all (2r,t) E Rd (0, 1), then ,,,, tú define pair of lime-reversed processes with marginal density p.
In particular, this result shows that if we impose a parametrization of 7h and to as two vector fields that differ by p²V log P, where p is fixed or learned marginal density estimate, then asymptotic satisfaction of DB implies that the continuous-time forward and backward processes coincide.
9

---

Published in Transactions on Machine Learning Research (01/2026)
Processes defined by discrete-time reversal. The generative and diffusion processes play symmetric role in Prop. 3.4. However, some past work starting from Zhang & Chen (2022), whose experiment settings we adopt in $4 has defined & as the reversal of the Euler- Maruyama discretization of a forward SDE, rather than as the Euler-Maruyann increments. To ensure the applicability of the results to the experiment setting, we need slight generalization:
discretization of backward SDE, in a case where the former has Gaussian
Proposition 3.5 (DB and FPE for Brownian bridges). The results of Prop. 3.4 hold if (()) is constant and T is the discrete-time reversal of the Euler- Maler-Maruyaaa discretization of the process dX, = att) dW,, Pprior (x) - (x; 0, Oo)))
The results in this section guarantee that global and local objectives with different discretizations are approximating unique contimuuus-iime objects when max At,. + 0. This justifies training and inference of samplers with different discretizations, allowing us to greatly reduce the computational cost of training (see $4). These observations are particularly relevant for diffusion-based samplers which rely on discretization of (partial) trajectories during training. In contrast, for generative modeling, one can use denoising score-malching objectives which can be minimized without any discretization in continuous time.
4 Experiments
We evaluate the effect of time discretization on the training of diffusion samplers. We follow the training setting from Sendera et al. (2024) throughout, extending their published code with an implementation of variable time discretization (see SC.1 for details). The following objectives from $2 are considered:
Path integral sampler (PIS) (Zhang& & Chen, 2022): The trajectory-level KT. divergence (5), which approximates the path space mcasure KL (15) is minimized via the deep reparamctrization trick (i.e., through differentiable simulation of the generalive SDE, hence necessarily on-policy).
Trajectory balance (TB) and VarGrad: The trajectory-level divergences of the second-moment type (7), optimized either on-policy or using the off-policy local search technique introduced in Sendera et al. (2024). As TB and VarGrad are found to be nearly equivalent in unconditional sampling settings, we consider VarGrad only for conditional sampling (see Fig. 9).
Detailed balance (DB): The time-local detailed balance divergence (8), and its variant FL-DB, which places an inductive bias on the log-density estimates - first used by Wu et al. (2020); Máté & Fleuret (2023) and evaluated in the off-policy RL setting by Zhang et al. (2024); Sendera et al. (2024) that assumes access to the target energy at intermediate time points (see $B.5).
Each objective is additionally studiod with and without the Langevin parametrization (LP), a technique in- troduced by Zhang & Chen (2022) that parametrizes the generative SDE's drift function via the gradient of the target energy. The assumptions made by each objective are summarized in Table 1.
Table Properties of training objectives. Variants with LP also use the intermediate energy gradient.
Property J Objective
PIS TB TB/VarGrad DB FL-DB
Time-local
x
Off-policy
x
/
Use intermediate energy
x
x
/
Use energy gradient
x
X
X
The noising process is always fixed to the reverse a Brownian motion, following Zhang& & Chen (2022) and subsequent work. The following densities are largeted:
Standard targets 25GMM (2-dimensional mixture of Gaussians), Funnel (10-dimensional funnel-shaped distribution), Manywell (32-dimensional synthetic energy), and LGCP 16600ddimension log- -ag-Gassian Cox process) as defined in the benchmarking library of Sendera el. al. (2024).
VAE: the conditional task of sampling the 20-dimensional latent z a variational autoencoder trained on MNIST given an input image z, with target density p(z x) x p(æ 2))(z).
Bayesian logistic regression problems for the German Credit and Breast Cancer datasets (25- and 31-dimensional, respectively), from the benchmark by Blessing et al. (2024).
We use a well-ostablished primary motric: the ELBO of the target distribution computed using the learned sampler and the true log-pas oog-partitinn function, estimated using N-step Auler-Maruyama integration. In our notation, the ELBO is log 7 E E -[(xx) ) + log (x XN) log P(X) (see (32) for details).
10

---

Published in Transactions on Machine Learning Research (01/2026)
Uniform
Random
Equidistant
While recent work diffusion samplers has used discretization with uniform-length time intervals for both training and sampling, we vary the time discrctization. Unless stated otherwise, we evaluate ELBO using Neval= = 100 uniform discretization steps. However, during training, we vary the number of time steps Muain and their placement, considering three schemes (see Fig. 4) for partitioning the interval [0, 1]:
060
0.15
0.25
0.50
0.55
1.00
0.25
0.50
0.55
Figure 4: Sampled 10-step discretizations of the unit inter-
val using the three schemes considered.
Uniform: Time steps uniformly spaced: t
-_ for i= = O...,, Nuain.
25GMM (d=
Neran
LLangvvin parametrization
Langovin parmmstrization
Random: We sample ii.d. numbers
20..... 2Nyai -1
u((1,c); where is
ILBO gae (oo dug Aopggaaa
DB
Rt-D
Th
Unitem
constant (we take c = 10), and define
At
4 = -=E4 At..
Σ 2j=0
2j
j=0
The interval lengths thus sum 1, and no have ratio of lengths greater than c. (We also tested sctting the (0 < i Mtrain) to be ii.d. samples from U([0, 1J) sorted in increasing order, but this scheme was mumerically unstable.)
Training discretization steps
Training discretization steps
Ponnel (d'm 10)
-Laggvin parametrization
Langovin parametrization
ILBO gap (100 step integration)
10P
Equidistant: We sample 1 U((e,, 2/Nuran - e)), where for
10-4, then set 4 = 11 (i 1)/Nrrain
for 1, 1,.... . Neaain - 1. Thus At, - for all < < Nuain - 1, i.e., all intervals are of equal length except possibly the first and last.
Nemsi
50
100
Triinnn discretization steps
Training diseretization steps
Results: Training-time discretization. In Fig. 5, we show the ELBO gaps on three of the datasets for different training-time discretizations as function of Nuain We observe that, for all ob- jectives, training with Random discretization con- sistently outperforms Uniform discretization with small number of steps, with the two converging as Ntrain increases to approach Neval = 100. The Equidistant discretization performs similarly to Random in most cases (see Fig. 10).
Manywell (d -
LLngevin parametrization
Langevin parametrization
ELBO deg (100 sup cormrnan
10 20 100
10 20
300
Training discretization
Training discretization
Notably, the time-local objectives (DB, FL-DB) per- form similarly to the trajectory-level objectives (TB,
Figure 5: Difference between true log and ELBO as a function of Nucain always evaluating with 100-step uni- form integration. Additional targets in Fig, 8 and Fig. 9, Equidistant results in Fig. 10.
PIS) when trained with few steps. However, as Nurain increases, the time-local models' performance typi-
cally plateaus (on some largets, they diverge with 100 steps). These results suggest that time-local objectives trained with nonuniform discretization and few steps can be a viable alternative to trajectory-level objectives in high-dimensional problems where the memory requirements training on long trajectories are prohibitive.
Results: Time efficiency. The training time per iteration expected to scale approximately linearly with the trajectory length Ntrai- Fig. 6 (left) confirms this scaling and illustrates the relative cost of different objectives: FL-DB and methods using the Langevin parametrization are the most expensive, they require stepwise evaluations of the target energy and its gradient, respectively. Fig. 6 (right) shows the ELBO gap plotted against training time, demonstrating that methods with nomuniform discretization achieve a superior trade-off between training time and sampling performance.
11

---

Published in Transactions on Machine Learning Research (01/2026)
Manywell (2-
Manywell (d- 52)
Langevin parametrization
Langevin panmmrriizaton
Langevin parametrization
Langevin parametrization
100
ELBO gap (100--tep sttp integration)
Ft-400
TB
DB
111
DB PS
Unitern dac. Randon disc.
Training (6)
10
101
201
10F
Training discretization steps
Training discretization steps
Runtime (6)
Runtime (s)
Figure 6: Left: Time to train for 25k iterations on Manywell as function of Nuzan, mean and std over 3 runs (note the log-log scale). Right: Runtime and ELBO gap, showing that Random discretization gives superior balance of speed
and performance. The marker area is proportional to Naai Results for 25GMM and Funnel densities in Fig, 11.
Manywell (d= 32)
Results: Inference-time discretization. To study the effect of campling-time discretization, we train models with Nerain 10 steps (using TB with Langevin parametrization) and different placement of time steps, then evaluate with dif- ferent Neval E (1, 1003. From Fig. 7, we observe that randomized discretization (Random or Equidistant) during training leads to smooth ELBO curves as a function of Neval: whereas training with Uniform discretization gives unstable be- havior with periodic fcatures at multiples of Ntrain; which may be due both the restricted set of inputs & to the model I (x; () during training and to the harmonic timestep embedding in the model architecture. This result is further evidence that nonuniform discretization during training yields more robust samplers that are less sensitive to the choice of Neval-
10a
Jenden dic.
ELBO gop
10¹
10
20
30
50
50
70
100
Integration steps
Figure 7: ELBO gaps for models trained with various discretization schemes and Ntrain 10, then evaluated with various numbers of integra- tion steps Neval. Results on Manywell energy; others shown in Fig. 12.
Additional results. Figures complementing those in the main text appear in $D.2 and D.3, while $D.1 contains more metrics and comparisons in tabular form. In particular, we combine the above objectives with the off-policy local search of Sendera et al. (2024) to achieve near-state-of-the-ar results with much coarser (nonuniform) time discretizations during training, whereas local search does not help the performance of methods using coarse Uniform schemes (Table 2).
5 Conclusion
We have shown the convergence of off-policy RL objectives used for the training of diffusion samplers to their contimuous-time counterparts. Those are Nelson's 's identity and the Fokker-Planck equation for stepwise objectives and path space measure divergences for trajectory-level objectives. Our experimental results give a first understanding of good practices for training diffusion samplers coarse time discretizations. We expect that the increased training efficiency and the ability to use local objectives without expensive energy evaluations are especially beneficial in very high-dimensional problems where trajectory length is a bottlencck, noting that trajectory balance was recently used in fine-tuning of diffusion foundation models for text and images (Venkatraman et al., 2024) and sampling in latent spaces of other large-scale models (Venkatraman et al., 2025). The success of training with coarse, non-uniform steps suggests a path toward such high-dimensional sottings.
Future theoretical work could generalize our results to diffusions on general Riemannian manifolds and to non-Marken Mon-Markovinn continuous- time processes, such as those studied in Daems et al. (2024); Nobis et al. (2024), as well as to discrete-space sampling problems for which diffusion samplers have recently been developed (Zhu et al., 2025; Guo al., 2025)- exploiting the unifying connections among diffusion generative processes general state spaces (Pauline et al., 2025).
12

---

Published in Transactions on Machine Learning Research (01/2026)
Acknowledgments
We thank Alex Tong, Alexandre Adam, and Pablo Lemos for helpful discussions at carly stages of this project.
J.B. acknowledges support from the Wally Baer and Jeri Weiss Postdoctoral Fellowship. The research of L.R. was partially funded by Deutsche Forschugggemeinschaaft (DFG) through the grant CRC 1114 "Scaling Cascades in Complex Systems" (project A05, project number 235221301). The rescarch M.S. was funded by National Science Centre, Poland, 2022/45 2012445NNSS6/0333
The research enabled in part by computational resources provided by the Digital Research Alliance of / ca), Mila (https:/ //mila. quebec), and NVIDIA.
Canada (https://allinn)
Code
Code is available at / http:///twrggggggddfffttisooooo/ sttto/////teeeettgrr....
com
References
Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Jocy Bose, Sarthak Mittal, Pablo Lemos, Cheng-Haa Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, and Alexander Tong. Iterated denoising energy matching for sampling from Boltzmann densities. International Conference on Machine Learning (ICML), 2024.
Michael Albergo, Gurtej Kanwar, and Phiala E Shanahan. Flow-based generative models for Markov chain Monte Carlo in lattice field theory. Physical Review D, 100(3):034515, 2019.
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 (3):318-326) 326, 1982.
Michael Arbel, Alex Matthews, and Arnaud Doucct. Anncaled flow transport Monte Carlo. International Conference on Machine Learning (ICML), 2021.
Paolo Baldi. Stochastic calculus. Springer, 2017.
Emmanuel Bengio, Moksh Jain, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow network based generative models for non-iterative diverse candidate generation. Neural Information Processing Systems (NeurIPS), 2021.
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward Hu, Mo Tiwari, and Emmanuel Bengio. GelowNet foundations. Journal of Machine Learning Research, 24(210):1 55, 2023.
Julius Berner, Lorenz Richter, and Karen Ullrich. An optimal control perspective on diffusion- generative modeling. arXiv preprint orXiv:2211.0136, 2022.
Denis Blessing, Xiaogang Jia, Johannes Esslinger, Francisco Vargas, and Gerhard Neumann. Beyond ELBOs: A large-scale evaluation of variational methods for sampling. International Conference on Machine Learning (ICML), 2024.
Johannes Buchner. Nested sampling methods. arXiv preprint arXiw:2101.09675. 2021.
Nicolas Chopin. A sequential particle filter method for static models. Biometrika, 89(3):539-55) 2002.
Rembert Daems, Manfred Opper, Guillaume Crevecoeur, and Tolga Birdal. Variational inference for SDEs driven by fractional noise. International Conference on Learning Representations (ICL.R, 2024.
Chenguang Dai, Jeremy Heng, Pierre E Jacob, and Nick Whiteley. An invitation to sequential Monte Carlo samplers. Journal of the American Statistical Association, 117(559))1587-16000 2022.
Paolo Dai Pra. A stochastic control approach reciprocal diffusion processes. Applied mathematics and Oplimization, 23:1) 329, 1991.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. Sequential Monte Carlo samplers. Journal of the Royal Statistical Sociely Series B: Statistionl Methodology, 68(3):411-48) 2006.
Tristan Deleu and Yoshua Bengio. Generative flow networks: Markov chain perspective. arXiv preprint arXi::230.001222, 2023.
Tristan Deleu, Padideh Nouri, Nikolay Malkin, Doina Precup, and Yoshua Bengio. Discrete probabilistic inference as control multi-path environments. Uncertainby Artificial Intelligence (UAT), 2024.
Alexander Denker, Francisco Vargas, Shreyas Padhy, Kieran Didi, Simon Mathis, Riccardo Barbano, Vincent Dutordoir, Emile Mathicu, Urszula Julia Komorowska, and Pictro Lio. DEFT: Efficient fino-tuning of diffusion models by learning the generalised I-transform. Neural Information Processing Systems (NeurIPS), 2024.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. Newral Information Processing Systems (NewrIPS), 2021.
Carles Domingo-Earich, Michal Droadzal, Brian Karrer, and Ricky TQ Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. International Conference on Learning Representations (ICLR), 2025.
Arnaud Doucet, Adam M Johansen, et al. A tutorial particle filtering and smoothing: Fifteen years later. Handbook of nonlinear fillring, 12(656-704):3, 2009.
Arnaud Doucet, Will Sussman Grathwohl, Alexander GDG Matthews, and Heiko Strathmann. Score-based diffuusion meets annealed importance sampling. Neural Information Processing Systems (NeurIPS), 2022.
Simon Duane, A.D. Kennedy, Brian Pendleton, and Duncan Roweth. Hybrid Monte Carlo. Physics Letters B, (9)(2): 222, 1987.
R. Durrett. Brownian Motion and Martingales in Analysis. Wadsworth & Brooks Cole Mathematics Series. Wadsworth Advanced Books & Software, 1984.
Benjamin Eysenbach and Sergey Levine. Maximum entropy RL (provably) solves some robust RL problems. International Conference on Learning Representations (ICLR), 2022.
A Friedman. Partial differential equations of parabolic type. Prentice-Hale Hall, 1964.
Avner Friedman. Stochastic differential equations and applications. In Stochastic differential equations, pp. 75-148. Springer, 1975.
Marylou Gabrié, Grant M Rotskoff, and Eric Eijuden Efficient Bayesian sampling using normalizing flows to assist Markov chain Monte Carlo methods. arXiv preprint Xki::2107.0800, 2021.
Wei Guo, Jaemoo Choi, Yuchen Zhu, Molei Tao, and Yongxin Chen. Proximal diffusion neural sampler. arXiv preprint Xr 03824, 2025.
Paul Lyonel Hagemann, Johannes Hertrich, and Gabriele Steidl. Generalized normalizing flows Markov chains. Cambridge University Press, 2023.
Jonathan Ho, Ajay Jain, and Picter Abbecl. Denoising diffusion probabilistic models. Neural Information Processing Systems (NeurIPS), 2020.
Matthew D Hoffman, Andrew Gelman, et al. The No-U-Turn sampler: adaptively sctting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research (JMLR), 15(1):1593- -161133331311,,,, 2014.
Robert E Kass, Bradley P Carlin, Andrew Gelman, and Radford M Neal. Markov chain Monte Carlo in practice: roundtable discussion. The American Slalislician, 52(())93-110) 100, 1998.
Patrick Kidger, James Foster, Xuechen Li, and Terry Lyons. Neural SDEs as infiitteddimensional GANs. International Conference on Machine Learning (ICMT., 2021a.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Patrick Kidger, James Foster, Xuechen Chen Li, and Terry Lyons. Efficient and accurate gradients for neural SDEs. Newral Information. Processing Systems (NewrIPS), 2021b.
Peter E Kloeden and Andreas Neuenkirch. The pathwise convergence of approximation schemes for stochastic differential equations. LMS journal of Computation and Mathemalics, 10:235 00:235-253, 2007.
Peter Kloeden and Eckhard Platen. Stochastic differentiol equations. Springer, 1992.
Hiroshi Kunita. Stochastic flows and jump-diffusions. Springer, 2019.
Salem Lahlou, Tristan Deleu, Pablo Lemos, Dinghuai Zhang, Alexandra Volokhova, Alex Garcan, Téna Néhale Exzine, Yoshua Bengio, and Nikolay Malkin. A theory of continuous generative flow networks.
International Conference on Machine Learning (ICML), 2023.
Jean-François Le Gall. Brownian motion, martingales, and stochastic calculus. Springer, 2016.
Benedict J. Leimkuhler, Charles Malthews, and Michael V. Trelyakov. On the long-time integration of stochastic gradient systems. Proceedings of the Royal Sociely A: Mathematical, Physical and Engineering Sciences, 470, 2014.
Pablo Lemos, Nikolay Malkin, Will Handley, Yoshua Bengio, Yashar Hezaveh, and Laurence Perreault- Levasseur. Improving gradient-guided nested sampling for posterior inference. International Conference on Machine Learning (ICML), 2023.
Christian Léonard. A survey of the Schrödinger problem and some of its connections with optimal transport. arXiv preprint arXiv:1308 0215, 2013.
Christian Léonard. Some properties of path measures. Séminaire de Probabilités XLVI, pp. 207- 270, 2014.
Sergey Levine. Reinforcement learning and control as probabilistic inference: Tutorial and review. arxiv preprinl Xrxi::180) 00900, 2018.
Xuechen Li, Ting-Kau Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. Artificial Intelligence and Statāstics (AISTATS), 2020.
Kanika Madan, Jarrid Rector-Brooks, Maksym Korablyov, Emmanuel Bengio, Moksh Jain, Andrei Nica, Tom Bosc, Yoshua Bengio, and Nikolay Malkin. Learning GFlowNets from partial episodes for improved convergence and stability. International Conference on Machine Learning (ICML), 2023.
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajoctory balance: Improved credit assigument in gflownets. Newral Information Processing Systems (NeurIPS), 2022.
Nikolay Malkin, Salem Lahlou, Tristan Deleu, Xu Ji, Edward Hu, Katic Everctt, Dinghuai Zhang, and Yoshua Bengio. GFlowNets and variational inference. International Conference on Learning Representations (ICLR), 2023.
Gisiro Maruyama. Continuous Markov processes and stochastic equalions. Rendiconts del Circolo Malematico di Palermo, 148-90, 1955.
Bálint. Máté and François Fleuret. Learning interpolations between Boltzmann densities. Transactions on Machine Learning Research (TMLR), 2023.
Alex Matthews, Michael Arbel, Danilo Jimenez Rezende, and Arnaud Doucet. Continual repeated annealed flow transport monte carlo. International Conference on Machine Learning (ICML), 2022.
Laurence Mlling Midgley, Vincent Stimper, Gregor NC Simm, Bernhard Schölkopf, and José Miguel Hernández- Lobato. Flow annealed importance sampling bootstrap. International Conference Learning Representa- lions (TCLR), 2023.
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, and Dale Schuurmans. Bridging the gap between value and policy based reinforcement. learning. Neural Information Processing Systems (NTPS), 2017.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Radford M Neal. Annealed importance sampling. Statistics and computing, 11(2):125-139, 2001.
E Nelson. Dynamical theories of Brownian motion. Press, Princeton, NJ, 1967.
Gabriel Nobis, Maximilian Springenberg, Marco Aversa, Michael Detzel, Rembert Dacms, Roderick Murray- Smith, Shimichi Nakajima, Sebastian Lapuschkin, Stefano Ermon, Tolga Birdal, Manfred Opper, Christoph Knochenhauer, Luis Oala, and Wojciech Samek. Generative fractional diffusion models. Neural Information Processing Systems (NeurIPS), 2024.
Frank Noé, Simon Olsson, Jonas Köhler, and Hao Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning. Science, (eoo6647))caawwl11 2019.
Nikolas Nüsken and Lorenz Richter. Solving high-dimensional Hamilton Jacobi Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space. Partial differential equalions and applicalions, 2(4):48, 2021.
Nikolas Nüsken and Lorenz Richter. Interpolating between BSDEs and PINNs: Deep learning for elliptic and parabolic boundary value problems. Journal of Machine Learning, 2023.
Ling Pan, Nikolay Malkin, Dinghuai Zhang, and Yoshua Bengio. Better training of GFlowNets with local credit and incomplete trajectorics. International Conference on Machine Learning (ICML), 2023.
Kushagra Pandey, Maja Rudolph, and Stephan Mandt. Efficient integrators for diffusson generative models. International Conference on Learning Representations (ICLR), 2024.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshmi- narayanan. Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res., 22((7):1 64, 2021.
Vincent Pauline, Tobias Höppe, Kirill Neklyudov, Alexander Tong, Stefan Bauer, and Andrea Dittadi. Foundations of diffusion models general state spaces: self-contained introduction. ar preprint arXiv:2512.05092, 2025.
Angus Phillips, Hai-Dang Dau, Michael John Hutchinson, Valentin De Bortoli, George Deligiannidis, and Arnaud Doucet. Particle denoising diffusion sampler. International Conference on Machine Learning (ICML), 2024.
Dominic Phillips and Flaviu Cipcigan. MetaGFN: Exploring distant modes with adapted metadynamics for continuous GFlowNets. arXiv preprint orXiv:2408.1500 15905, 2024.
Lorenz Richter and Julius Berner. Improved sampling via learned diffuuiins. International Conference on Learning Representations (ICLR), 2024.
Lorenz Richter, Ayman Boustati, Nikolas Nüsken, Francisco J. R. Ruiz, and Ömer Deniz Akyildiz. VarGrad: A low-variance gradient estimator for variational inference. Neural Information Processing Systems (NeurIPS), 2020.
Garoth O Roberts and Richard L Tweodic. Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli, pp. 341-363, 1996.
Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Lars Ruthotto, Stanley J Osher, Wuchen Li, Levon Nurbekyan, and Samy Wu Fung. A machine learning framework for solving high-dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences, 117((1):9183-9111 2020.
Tim Salimans, Diederik Kingma, and Max Welling. Markov chain Monte Carlo and variational inference: Bridging the gap. Tn International conference machine learning, pp. 1218 1226. PMLR, 2015.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Simo Särkkä and Arno Solin. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector- Brooks, Alexandre Adam, Yoshua Bengio, and Nikolay Malkin. Improved off-policy training of diffusion samplers. Newral Information Processing Systems (NeurIPS), 2024.
Neta Shaul, Juan Perez, Ricky T. Q. Chen, Ali Thabet, Albert Pumarola, and Yaron Lipman. Bespoke solvers for generative flow models. International Conference on Learning Representations (ICLR), 2024.
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning (ICML), 2015.
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Newral Information Processing Systems (NeurIPS), 2021a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishck Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations (ICLR), 2021b.
Tommi Sottinen and Simo Särkkä. Application of Girsanov theorem to particle filtering of discretely observed continuous-time non-lincar systems. Bayesian Analysis, 3(3):555 584, 2008.
Jingtong Sun, Julius Berner, Lorenz Richter, Marius Zeinhofer, Johannes Müller, Kamyar Azizzadeneshelii and Anima Anandkumar. Dynamical measure transport and neural PDE solvers for sampling. arXiv preprinl crXiv:2407.07877 2024.
Daniil Tiapkin, Nikita Morozov, Alexey Naumov, and Dmitry Vetrov. Generative flow networks as entropy- regularized RL. Artificial Intelligence and Statistics (AISTATS), 2023.
Belinda Tzen and Maxim Raginsky. Neural stochastic differential equations: Deep latent Gaussian models in the diffusion limit, orXiv preprint arXiv:1905. 09883, 2019.
Francisco Vargas, Will Grathwohl, and Arnaud Doucet. Denoising diffusion samplers. International Conference on Learning Representations (ICLR), 2023.
Francisco Vargas, Shreyas Padhy, Denis Blessing, and Nikolas Nüsken. Transport meets variational inference: Controlled Monte Carlo diffusions. International Conference on Learning Representations (ICLR), 2024.
Siddarth Venkatraman, Moksh Jain, Luca Scimeca, Minsu Kim, Marcin Sendera, Mohsin Hasan, Luke Rowe, Sarthak Mittal, Pablo Lemos, Emmanuel Bengio, et al. Amortizing intractable inference in diffusion models for vision, language, and control. Neural Information Processing Systems (NeurIPS), 2024.
Siddarth Venkatraman, Mohsin Hasan, Minsu Kim, Luca Scimeca, Marcin Sendera, Yoshua Bengio, Glen Berseth, and Nikolay Malkin. Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative models. International Conference Machine Learning (ICML), 2025.
Alexandra Volokhova, Michal Koziarski, Alex Hornández-Gareía, Cheng- Hao Liu, Santiago Miret, Pablo Lemos, Luca Thiede, Zichao Yan, Alán Aspuru-Guzik, and Yoshua Bengio. Towards equilibrium molecular conformation generation with gflownets. Digital Discovery,
2024.
Martin J Wainwright, Michael Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1-2):1-305, 2008.
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229-25: 256, 1992.
Hao Wu, Jonas Köhler, and Frank Noé. Stochastic normalizing flows. Neural Information Processing Systems (NeurIPS), 2020.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Dinghuai Zhang, Ricky T. Q. Chen, Nikolay Malkin, and Yoshua Bengio. Unifying generative models with GFlowNets and beyond. Xrxiv preprint arXiv:2209. 00n:209900206, 2023.
Dinghuai Zhang, Ricky Tian Qi Chen, Cheng- Hao Liu, Aaron Courville, and Yoshua Bengio. Diffusion generative flow samplers: Improving learning signals through partial trajectory optimization. International Conference Learning Representations (ICLR), 2024.
Qinsheng Zhang and Yongxin Chen. Path integral sampler: a stochastic control approach for sampling. International Conference on Learning Representations (ICLR), 2022.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. Interna- lional Conference on Learning Representalions (ICLR), 2023.
Yuchen Zhu, Wei Guo, Jaemoo Choi, Guan-Horng Liu, Yongxin Chen, and Molei Tao. MDNS: Masked diffusion neural sampler via stochastic optimal control. Neural Information Processing Systems (NeurIPS), 2025.
Brian D Ziebart. Modeling Purposeful Adaptive Behavior with the Principle of Maximum Cousal Entropy. Carnegic Mellon University, 2010.
18

---

Published in Transactions on Machine Learning Research (01/2026)
A Additional related work
Classical sampling methods. The gold standard for sampling is often considered Annealed Importance Sampling (AIS) (Neal, 2001) and its Sequential Monte Carlo (SMC) extensions (Chopin, 2002; Del Moral et al., 2006). The former can be viewed as special case of our discrele-time setting, where, however, the transition kernels are fixed and not learned, thus requiring careful tuning. For the kernels, often a form of Markov Chain Monte Carlo (MCMC), such Langevin dynamics and extensions (e.g., ULA, MALA, and HMC) are considered. While they enjoy asymplotic convergence guarantees, they can suffer from slow mixing times, in particular for multimodal targets (Doucet et al., 2009; Kass et al., 1998; Dai et al., 2022). Alternatives are provided by variational methods that reformulate the sampling problem as an optimization problem, where a parametric family of tractable distributions is fitted to the target. This includes mean-field approximations (Wainwright et al., 2008) as well as normalizing flows (Papamakarios el. al., 2021). We note that MCMC can also be interpreted as a variational approximation in an extended state space (Salimans et al., 2015).
Normalizing flows. There exist various versions of combining (contimuous-time or discrete-time) normaliz- ing flows with classical sampling methods, such as MCMC, ATS, and SMC (Wu et al., 2020; Arbel et al., 2021; Matchews et al., 2022). Most of these methods rely on the reverse KL divergence that suffers from mode collapse. To combat this issue, the underlying contimuity equation (and Hamilton-Jacobi-Belln equations in case of optimal transport) have been leveraged for the learning problem (Ruthotto et al., 2020; Máté & Fleuret, 2023; Sun et al., 2024). However, in all the above cases, one needs to either restrict model expressivity or rely on costly computations of divergences (in continuous time) or Jacobian determinants (in discrete time). Our Prop. 3.4 shows that, in the stochastic case, the discrepancy in the corresponding Planne--llank equation - an expression involving divergences and Laplacians - can be approximated by detailed balance divergences, which require no differentiation.
Diffusion-based samplers. Motivated by (annealed) Langevin dynamics and diffusion models, there is growing interest in the development of SDEs controlled by neural networks, also known as neural SDEs, for sampling. This covers methods based on Schrödinger (Half-)bridges (Zhang & Chen, 2022), diffusion models (Vargas et al., 2023; Berner et al., 2022), and annealed flows (Vargas et al., 2024). These methods can be interpreted as special cases of stochastic bridges, aiming at finding a time-reversal between two SDEs starting the prior and target distributions (Vargas et al., 2024; Richter & Berner, 2024). In particular, this allows to consider general divergences between the associated measures the SDE trajectories, such as the log-variance divergence (Richter et al., 2020; Nüsken & Richter, 2021). We note that there has also been some work on combining classical sampling methods with diffusion models (Phillips et al., 2024; Doucet. et al., 2022).
GFlowNets. GFlowNets are originally defined in discrete space (Bengio et al., 2023), but were generalizod to general measure spaces in (Lahlou et al., 2023), who proved the correctuess of objectives in continuous time and experimented with using them to train diffusion models as samplers. However, the connection between GFlowNets and diffusion models had already been made informally by Malkin et al. (2023) for samplers of Boltzmann distributions and by Zhang et al. (2023) for maximum- likelihood training, and the latter showed connection between detailed balance and sliced score matching, which has a similar flavor to our Prop. 3.4. GFlowNets are, in principle, more general than diffusion models with Gaussian noising, as the state space may change between time steps and the transition density does not need to be Gaussian, which has been taken advantage of in some applications (Volokhova et al., 2024; Phillips & Cipeigan, 2024).
Accelerated integrators for diffusion models. We remark that there has been great interest in developing accelerated sampling methods for diffusion models and the related continuous normalizing flows (e.g., Shaul et al., 2024; Pandey et al., 2024). In particular, one can consider higher-order integrators for the associated probability flow ODE (Song et al., 2021b) or integrate parts of the SDE analytically (Zhang & Chen, 2023). However, we note that this research is concerned with accelerating inference, not training, of diffiusion models and thus orthogonal to our research. While Denker et al. (2024) and Venkatraman et al. (2024) explored subsampling strategies for the gradient estimators to reduce memory costs, they did not analyze the effect of randomized time steps. For generative modeling, one has access to samples from the target distribution, allowing the use of simulation-free denoising score matching for training. For sampling problems
19

---

Published in Transactions on Machine Learning Research (01/2026)
without access samples, diffusion-basee methods, such as those outlined in the previous paragraphs, need to rely on costly simulation-based objectives. However, our findings show that we can significantly accelerate these simulations during training with negligible drop in inference-tine performance.
B
Theory details
B.1 Assumptions
Throughout the paper, assume that all SDEs admit densities of their time marginals (w.r.t. the Lebesgue measure) that sufficiently smooth such that have strong solutions to the corresponding Fokker-Planck equations (see, e.g,, Friedman (1964; 1975); Durrett (1984) for corresponding conditions). In particular, we assume that' Ppriors Ptarger E C (12,, R20o) are bounded. Furthermore, we assume that E C ([0, 1] R R²) for all drifts H4, i.e., they are infinitely differentiable, and satisfy uniform (in time) Lipschitz condition, i.c., there exists a constant C such that for all 2.y y E IRd and E [0, 1] holds that
[l( uly, t)ll - - -
(00)
Moreover, we assumee that the diffusion rate satisfies that a C ([0, 1]. R30). These conditions guarantee the existence of unique strong solutions to the considered SDEs (see, e.g., Le Gall (2016)). They are also sufficient. for all considered path measures to be equivalent and for Girsanov's theorem and Nelson's 's relation to hold. Moreover, they allow the definition of the forward and backward Itô integrals via limits of time discretizations that are independent of the specific sequence of refinements (Vargas et al., 2024).
While we use these assumptions to simplify the presentation, we note that they can be significantly relaxed. However, the above conditions hold for neural networks with MLP architectures whose activations are Lipschitz and infinitely differentiable, such as the GELU-activated models we consider in this paper.
B.2 Formal definition of the MDP
We elaborate the definition of the MDP in $2.1.
The state space is
N S = (0) U U (I,tt): E (( U(L)),,
(21)
n=0
S,
where and I are abstract initial and terminal states.
The action space is A R
The transition function T: x A + S describing the deterministic effect of actions is given by
T(o.a) = to), tx()) = (a, tn+1) n n< N T(L,a)=
(22)
The reward is nonzero only for transitions from states in SN and is given by R(x, (N) E
It is arguably more natural from control theory perspective to treat the addition of (e.g., Gaussian) noise as stochasticity of the environment, making the policy deterministic. However, we choose to formulate integration as a constrained stochastic policy in a deterministic environment to allow flexibility in the form of the conditional distribution. We also note that the policy at 1 irrelevant since 1 an absorbing state.
B.3 Numerical analysis
Definition B.1 (Strong convergence). numerical scheme X = (Nx) is called strongly convergent of order y if
max E - xx,.]] -I,1I S 0=
max At,, ==
(23)
*Note that we also consider samplers using Dirac delta prior, which can be treated by relaxing our conditions (Dai Pra, 1991). Under the policy given by (16), we can equivalently consider a (discrete-time) setting on the time interval [t), 1] using a Gaussian prior with learned mean and variance o² (fo))Adv.
20

---

Published in Transactions on Machine Learning Research (01/2026)
where 0 < C' oo is independent of N E N and the time discretization 0 < < <.. <N = 1.
Definition B.2 (Weak convergence). A mumerical scheme X = (Xx)) is called weakly convergent of order
if
max O=
[LS((x)) -I |||| -
N max Alm)) n=n
(24)
for all functions f in a suitable test class, where consider f E C IR) with at most polynomially growing derivatives. The constant 0 < C' < oo is independent of N E N and the time discrctization 0 0= to < 4 < ... < IN = 1, but may depend on the class of test functions considered.
Note that if f is globally Lipschitz, then strong convergence implies weak convergence. The converse does not hold.
Let us also consider continuous version s(X) of the numerical scheme X = (Xx,) defined by x(8), &, and linearly interpolating between the (ns where we note that implicitly depends on the discretization. We can then define the pushforward t.IP of the distribution I of X on the space of continuous functions C([O, 1], Rd.. We say that e.IP converges weakly to the path measure P of X if for any bounded, continuous functional f: C((0, 1], ) + R it holds that
E X [S(X] - Ex-P [S(X]]
(25)
as max, At, + O.
B.4 Subtrajectory balance
Generalizing trajectory balance (7) and detailed balance (8), we can define divergences for subtrajectories of any length k by multiplying the log-ratios appearing in (8) for several consecutive values of n, which through telescoping cancellation yields subtrajcctory balance divergence, defined for any < k N by
D Tub (P Q.P) = E _ log Pn
II 7 (8n+et1 Xn-e)
27
(26)
Pitth + I & #(xnts Xnts+1)
The subtrajectory balance (SubTB) divergence generalizes detailed balance and trajectory balance, one has
D D Dub (P, P D (IP, Q.P) and S (.P. - D Q))
The SubTB divergence was introduced for GFlowNets by Malkin et al. (2022) and studied as a learning scheme, in which the divergences with different values of & are appropriately weighted, by Madan et al. (2023). SubTB was lested in the diffusion sampling case by Zhang et al. (2024), although Sendera et al. (2024) found that it is, in general, not more effective than TB while being substantially more computationally expensive.
B.5 Inductive bias on density estimates
We describe the inductive bias on density estimates used in the FL-DB learning objective. While normally one parametrizes the log-density as neural network taking æ and t as input:
log (2, t) (N,
the inductive bias proposed by Wu et al. (2020); Máté & Fleuret (2023) and studied earlier for GFlowNet diffusion samplers by Zhang et al. (2024); Sendera et al. (2024) writes
log p(z, t) = -tt (x)) + (1 - log (x) + NNo
where (rer(;;t) is the marginal density at time t of the uncontrolled process, i.e., the SDE (1) that sets 7 = 0 and has initial condition Pprior- Thus a correction is learned to an estimated log-density that interpolates between the prior at t = 0 and the target at 1.
The acronym 'FL'' stands for 'forward-looking' referring to the technique studied for GFlowNets by Pan et al. (2023) and understood form of reward-shaping scheme in Deleu et al. (2024).
21

---

Published in Transactions on Machine Learning Research (01/2026)
B.6 Proofs of results from the main text
Proposition B.3 (Convergence of functionals). P, Q, W are path measures of three forward-time SDEs, the drifts of the SDEs and their derivatives are bounded, and f E C (R, R) has at most polynomially growing derivatives, then
E Ex
[s
(1005
(x)) dQ
dip
)]
muxa Atn A0- Exaw [s (1o8
dIp IQ
(x)
Proof of Prop. 3.2. As shown in the proof of Lemma B.7, log dP (X) can be viewed as component of the Euler-Maruyama integration of Itō process (with space-dependent diffusion) evaluated at time 1. Given assumptions on the coefficient functions (see also $B.1), the result follows by weak convergence; see, e.g.: Kloeden & Platen (1992).
aQ
Proposition B.4 (Asymptotic consistency of TB and VarGrad). Under the assumptions of Prop. 3.2, the divergences D [P, Q) and DEV (P, QQ) converge to D (TP QQ) and D (IP, Q2), respectively.
Proof of Prop. 3.3. Immediate from Prop. 3.2, taking f(x) x² and f(z)
Proposition B.5 (Asymptotic equality of DB and FPF)). Under the smoothness conditions in Lemma B.8, Tt, D. jointly satisfy Nelson's identity (p π o² l) at (x,,t) if and only if
lim R-0
A (tt)) for almost every 2,
Vh
where Fith = + (Ix, + (tt)hhz. If in addition
A =0,
lim Ex-A(0.1a) h-0
then the Fokker-Plant equation is satisfied at (Ir,t). If both conditions hold at all (Ir,t) E Rd (0, 1), then Tt T define pair of time-reversed processes with marginal density P.
Proof of Prop. 3.4. We write pe(x), (x(x), or for p(x, t), (x, (t)) for convenience. By Lemma B.8, the first condition implics that for almost all 2;
(z loggog(())))))) (7) (x))) = 0,
(27)
which implies Nelson's 's identity at (x,, t). while the second condition implies that
O, log) (2.), (aogū(()))) + (V, (x)))) + of Alog
[l(()) ST c( (') 0. (28)
of
Substituting the expression (27) into (28) and simplifying, we get
o (z)) - (V, () (7) (z.), Vlog + of (Alog + | (z.)))))))
2
which gives exactly the logarithmic form of the Fokker-Planck equation.
Proposition B.6 (DB and FPE for Brownian bridges). The results of Prop. 3.4 hold if (t)) is constant and f is the discrete-time time reversal of the Euler- Maruyaaa discretization of the process dX, = a(t) aW, Pprior (x) Nx;; Ta).
22

---

Published in Transactions on Machine Learning Research (01/2026)
Proof of Prop. 3.5. Using the changes of variables + followed + t- do, it suffices to show this for 0, 1, making the process standard Brownian motion (the change of bounds for t is insubstantial as the conditions are local in time).
Let f the backward policy as originally defined.
The reverse drift is 'I (2. t) = 2. so we have
T (Iz (eth) = N ( ( t metth
Let ' the discrete-time reversal of the forward-discretizee Brownian motion. By elementary properties of Gaussians, we have
t =11 F' Teth) = N Tei t+ Fith t ")
Let ( Fe4n) and 2 (Ze, Tzth) be the discrepancies (18)) defined using fo and fr, respectively. We will show that replacing A by A' does not affect the asymptotics in Prop. 3.4.
We have
Ac-SCH(DESRCCHN) - (10251((112thh) = log F' 1zth) - log & (xt xz-h)
2
(
-1
d dlg - h
+
It
t 2oth
2
zinh
-1
(1
1+ )
h
h
dlogg
+ Tzth +
1 h Be
2
Sctting 2ith 2z 7e(xz)) + Vhz, the above becomes
-1 d + 0(h3) + 1 (M]11)++
2 t
For fixed z, the Vi-order asymptotics of this expression vanish. In expectation N( Ta), the h-order asymptotics vanish because E [||]|]] = d.
B.7 Technical lemmas
Lemma B.7 (Convergence of Radon-Nikodym derivatives). (a) Let (P(1) and TP(2) be the path space mea- (1).(2)
sures defined by SDEs of the form (9) with invtial conditions Pprior and drifts (1.022)) Let ((1).(2) be the Fuler- Fuler-Moruyamae Maruyama-diseree discretized measures with respect to time discretization (tn) For almostt apa) every X C((0, Rd), dp(a) (Xto...ty) detes (X) as max, At, 0, where Xto.utyy is the restriction of X to the times lo
ap())
(b) The same is true for a path space measure P defined by a forward SDE with initial conditions and a measure Q) defined by a reverse SDE with terminal conditions: if IP and Q2 are the discrete-time processes given by Buler-Marugaam and reverse Suler-MMaryaan integration, respectively, then for Q-almost every E C(O0, Rd),, as max, Al, -
dP
(X + aq (X).
aQ
23

---

Published in Transactions on Machine Learning Research (01/2026)
Proof. We first show (a). We have
dP(1)
P Prrior (Xo)
II N-1
n= π n (Xln41 lnt1 (Xin)
log dP(2) (X (x) = log
P Porior (Xo)
-- n=ル
स n (Xtntt1 (xm))
Pprior (Xp)
N-1
N (X it + 7. (1)) (Xt, tn) otnn) atm))
log (2)
log
Pprior (Xo)
(X it, + + (Xt, tn At o (tnn) At,)
==0
(1) Pprior (Xo)
N
Tt (2) (Xt, bn ( = (7) (712)(X (X 11) Aln
log (2)
Pprior (Xo)
20((n.)
(2) (X,,, (n) - (Xt,)
(Xe,,
(Xiut1 - Xc.) )
(29)
ott,)
This is precisely the (Riemann) sum for the integral defining the continuous-time Radon-Nikodyy derivative (13); by continuity and assumptions in $B.1, the sum approaches the integral max, At, 0.
We now show (b) assuming (a). Let P be the path mcasure defined by Gaussian (0, initial conditions and drift 0 and Po its discretization. Similarly, let Q2" be defined by Gaussian terminal conditions and zero reverse drift and let Q" be its reverse-time discretization. By absolute continuity, we have
dP dQ (X) = (P/dP(() aPo dQQ dQQ()(((X) d (X),
dPP ddp
(x) aP (X dIo (X (X
(dQ/dQo(XX (x) dQo
By (a), d / (X) + dP PP/dPP (X), and similarly for Q. It remains to show that dP0 /d (Xt....n + log dIP" (x) = Mog.N(Xo:000) (Xg; I). Indeod, we have
log / (X = log N(No;
N
N(Xe,, XI, 1 (tn-1))t N(X(n-1i (tn)
log
N,X1:0,T)
0,
X - X 11
=log N(X1;0,1)
2Atn-1
(tx))
12
+ + a(hn)
(1n-1))
a.a. log V(No:0,1 + d log N(X1;0,T)
o(1)
o(0) + //'
do(1)² (t -2
= =oddloogort o())
N(Xo;
log N(X;;0,T)
(30)
which coincides with the continuous-time Radon-Nikodym derivative.
Lemma B.8 Contimuous-time asymptotics of the DB discrepancy). Let us define the abbreviations Pc( (x)), on to refer to p(x,t) t), (x, t), o(t.. Suppose that The and T are continuously differentiable in x and once in and that log Ps is continuously differentiable once in t and twice in x.
(a) For given 2, the asymptotics of the DB discrepancy at (ze, t) are of order vh and are given by
lim 1 0
A0 7.(x.) 012) (2, V (xx) (T7x(xx) (x))))))-
Vh
24

---

Published in Transactions on Machine Learning Research (01/2026)
(b) The expectation of the DB discrepancy over the forward policy (i.e., over 2 2~ NO,, I)) is asymptotically of order h, with leading term
1 (0s
lim BeL (orss (xx) h
h
= 8 log (e))) + (Tre(1)), V ( + (V. Tr((oe))
of
The [((x)) - tI (((( 1
A (1 -
+ 2
of
Similarby, the expectation over the backward policy is
lim E (((nal(() H [A
O. log (Te() V Vlogi( (V. Te((s)) Be(x)) [e( 1) of
of A Aog
Proof. We will simultaneously show (a) and the first part of (b). The second part of (b) is symmetric, by reversing time.
Identifying teth with 24 + (1.(h + orvhz, we will analyze the leading asymptotics of the DB discrepancy, i.e, Ae-e+ (Te, Tt+n) = vi(z....) + h(...) ) + O(h3/2). The coefficient of vh will the scalar product of = with term that is independent of a and equals the expression on the right side in (a), and thus vanishes in expectation over The coefficient of h, in expectation over 2, will equal the expression the right side in (b).
We can show using Taylor expansions that
log (14h(1 (1))))
V2 V.
h 0. (og + (7,(())) Vlogā(())) + 1 olte, 4 (olog((a))))) + 0(1/33)
(31)
Now we are going analyze the second part of (18), which involves the policies. We have
t (x' Ie+h)
log म. (Ie+h xz)
1i Fith + ( of L4
| x4- (T:))
2no 2 + d
2
0,h
2TO²
[ Ftth + S T (3)-
|1, v /22l13
d log 0tth
2
0 2
20
-1
[|,, Feth + (1
||
dhō, + O(h2).
2
02 24 h
2
Next we will write
Ft - Feth + (1en(FE41) 2i Etth + Te(xxth (7te()))) - (e44())))))
=-o,Vhz (Z), (144(Z044))6
25

---

Published in Transactions on Machine Learning Research (01/2026)
and substitute this into the first term above, yielding
| Ft 21th + (2
dhōd, 0z) + O(h2)
2
ozhn h VorTiz (Te((1)) (F)+)(1)))
2
-1
||=||2 dhōg + O(h2)
Ltrh
2
(1((
( h (orz, [l(()))
It (c Vh.
207th
o7th
0111=112
| (logor)) + 0(12)
20 i+hh
2
(z, oi (TTe(3)) [(((x)))) + Belz, o. NiV (1(())))) (
Vh
L4th
[1(( (T((11)1 (1ggo)))
+ |
o2
+ h
+0(h8/2)
203
= (1og (1h+
= Vh (z, -o, ( (2x1) - (z()))
+ h (T(( -
(z, (x))22))) (11=11- - d) (logo,)) 0(1112))
20
Combining with the terms in (31), we get that the coefficient of vh is exactly as desired. For the coefficient. of h, and the terms of the form (z....) and || d vanish in expectation over z. For the terms that are quadratic in 2, Hutchinson's formula implics that
E2~A(O,, 2/=, (T.())) (V, (x.)))), Eg [iz, V log (og A log (xr.)
Putting these identities together, obtain that
lim ( (04) k-+
= -. log + (1:), V Vlogf( + go? logp.(z;) (7 (x:))1
+ (V, Tr(()))),
20
which is equivalent to the expression in (b).
C Experiment details
C.1 Training settings
All models trained for 25,000 steps using settings identical to those suggested by Sendera et al. (2024) (https://ga) thub. cww//oomm//// GFNOrg/ (eetOrrggfnddffii gff-ddifusion). For DB, we use the same learning rates as for SubTB (10-3 for the drift and 10-2 for the flow function), and for PIS, 10-3 or 10-4 depending on its stability in the specific case.
Training times are mcasured by wall time of exccution on a large shared cluster, primarily on RTX8000 GPUs. Although all runs were assigned by the same job scheduler, some variability in results is inevitable due inconsistent hardware.
26

---

Published in Transactions on Machine Learning Research (01/2026)
D Additional results
D.1 Additional metrics and objectives
In Table 2, we show extended results on the four unconditional sampling benchmarks from Sendera et al. (2024), reporting the ELBO log 2 and imporlance-weighted ELBO log ZRW Specifically, the two are computed as
K
2 X
K
Q (0))
log
2 =
E(XN)
+ log
log Z
log
K
P(X
K
( (0)
i=1
(32)
1
-E(X + log ( x
1 log Z log
Q(
log RRW = log
exp
K
P(X
K
P P(X
where { (.)) ((K)
~ P and we note that Ellog 2] log Z DKL (P. Q) log Z and E| Z. We
take K = 2000 samples and report the difference between the ground truth log Z and the ELBO when log Z is known.
These results are consistent with the conclusions the main text. Notably, when combined with local search, coarse nonuniform discrctizations continue to show results comparable those of 100-step training discretization in most cases. Table shows results on two additional target energies and on the conditional VAF task.
27

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: ELBOs and IS-ELBOs on 25GMM, Funnel, and Manywell well (absolute error from the true value).
25GMM (d= 2)
Tritning dicretization
s0et random
step equidisant
10-step uniform
100-ttep molorm
Exsluttio slage-
10
100
100
100
Algorithmm Metric
A
Alig gln
A
Alig 200
Alee.
Ale goa
Ale
Alce 2008
Alg 2700
Aing, 2008
Alee
Allg 2000
PTS
1.
2:30
1.5
2.30.0a5
1.0200
1:31100
2.10.034
0.90.434
L22.00
1.44a0
2:10,300
0.90.031
8.77,0.9
1.021806
1.13.000
1.021841
TB +
1.71a006
0.020037
0.471s06
0.002:004
L.7:00
0.362007
0.42.031
0.00001
1.67:00
0.00u.02
10:35-275
1.87
0.15.am
0.000411:01
VarGra
2.32x0as
1.04aac4
1:52xom
1.8lion
2,03-aaa
1.19.a
L.Baan:
2.12.000
1/02.as
9.15.a37
0.924..
1.12-am
1.0248m
VerGod +18 18
1/88gnrr
0.tigrcs
0:37aam
0.07gme
1.07gm
0.07gmer
0.33gear
0.02gne:
1.0gpm
0.00gna7
8:25gm
0.15
000
PIS + L.P
2.80anar
1
1.8810m
0.10
2.77:m3
1.0
1 LMMans
0.0520m
277sam
1.0g0ass
1.70:ae
0. 0.4329 un
TB LP
1.57amo
0.
0.30samm
0.072am
1.50
0.01amms
0.90
0.03
2.70
0.11cmas
0/4Se
0.10cmm
0.0
TD LP
1.781.00
0.02000s
0.111006
0.02:004
182.am
0.08.08
0.4..05
0.072015
1/681800
0.05.002
8.77115
1.3013.6
0.15.am
0.118m
VarGna LP
1.50atm
0.03aass
0.30amm
0.01ame2
1.46saa
0.07amss
0.82
0.04an4
1.58aam
0.02a4ss
5.52-130
0.58
0.15-mm
0.008
VerGnd 18
1.88mmm
0.00samm
0.07a0m
0.34g
0.07.50s
0.0420a7
7.07a1
Funnel (d=10)
Triining diserctiaation +
10 step pandons
stst equidistont
10-t uulbora
100 500p uniSom
Esalation steps-
10
100
10
100
100
100
Algottthm Metrie
&llg
Z
ailog 2
Allg
Alog
Alog
Alog2
aolog 2llogg
Alog
à 2
A Al
log 2
L.llson
0.72aum
1.1gamn
0.30amm
0.72auu
0.02
111au
0.38ax10
8.03e423
1.6
0.52maam
0.08emra
73
0.flauss
0.76axon
0.481204
1.00se
0.47mam
0.74aam
107aam
0.42am
10.5043:2
2.20a..5
0.54aaua
0.26anes
T9+4 + 18
1.10gnmm
0.00g0ss
1.13smm
D-llasmm
(oMegnes
1.1ga
208
1.00
0 0.3t ea
VarCzed
1001oa
0.000as
0.20100
0.42100e
1iliim
0,361211
0.761am
0.48uats
1007.a0
0.60100
2,4111.0
0.501em
0.17.003
VaGGdd LS
1.8
0.65auss
LiBeemn
1.58eam
0.32
1286010
0.45emss
0.30ea10
780
3.93esu
E.llas
0.02
PIS IP
E.llson
0.71amm
0.28amm
LiBlzam
0.50sam
0.00au
110sm
0.57aro
8.85mma
0.50amia
0.13a
1.010m
0.002012
0.7210m
0,201012
044.mm
10
03819m
3007mmm
0.36:3357
0.48:0.006
0
LP
1.30sem
0.06anas
0.501om
0.30aa0
12763m
0.22aus
126.
0.48aam
10.16.
14.28-1451
0.82anvas
0.25
VarGrdd
1/88amm
0.72aam
037ase
038ents
107:mm
0-48-ammm
VarCrod
0.4610a4
0,501006
0333a
14615
0,41100
100913
0.35..5
132.405
0.461001
10.68.1241
0.77
Manywell (d 32)
Training discretization
10-step random
10-step uniform
100-step uniform
Exaluation steps
10
100
10
100
100
Algorithm Metric
A log Z
A log Hw
A log Z
A log 2!W
A log Z
A log 2
A log Z
A log 2 WW
A log Z
A log 2/w
PIS (Ir - 10-3)
2.7020.30 14
30
4.7410.15
2.7720.05
14.0820.1
33.84 69.72 2.9720a7
3.8710.03
2.60 as
PIS (Ir = 10-1)
14.34=02
3.23
6.37=0.0
2.80=0.20
14.1620.7
2.86.077
76.30=1.0
35.66=1.45
4.17=0004
2. 2.62=2.06
TB
24.96
2.92=1.10
3.49 4a
2.70=0.11
11.81+0.7
2.35+200
62.95=10.12
30.07=8.79
1.05
2.75
TB LS
15.24
1.5de0.77
7.2450.4
0.55
14.862.00
0.45
51.0844.27
16.82=a.08
4.5246.91
#.37
VarGrad
14
2.79
5.64:0 ae
2.7720.05
14.8010.14
2.862101
71.71
35.53311.51
4.04
2.78
VarGrad + 18
16.02
2.8410.15
7.0310.00
2.0020.48
16.0810.75
3.2611.10
69.14_12.30
28.45
6.53.1..5
2.43
PIS + LP (Ir 10-2)
13.97 18
2,1520025
4.34
1. 1.6920.41
d diver e f
3. 3.60
1.37=0.22
PIS + LP 10-)
31. 3..0000000
1.16=a.45
17.55+99.2
1.39
3187=a.21
5.26#3.00
35.9620.34
8. 8.12+1.0
14.71+00.0
0.50ee.76
TB LP
14.87
3.02+1.25
4.72so.2r
2.66:00.0
14.62:0.21
3.27
19:06:1.49
4.20
3.66.0026
2.42:0.32
TB LS LP
13.58
0.6010.25
2.40g0.30
0.00
13.6710.44
0.81
24.32:1.02
2,1010.45
1.81
0.03..0.7
VarGrad + L.P
14.79...59
3.11.
4.68
2.71
14.63.0.20
3. 3.15.0.02
20.7213:32
3.8911.72
3.41
2.09
VarGrad + 1.8 + LP
16.2420.70
1.31
5.12 se
0.3220.31
14.22...22
0.35:20.0s
22.80
1.71
1.7720.06
0.05::0.06
28

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: ELBOs with different numbers of training and integration steps Credit, Cancer, the conditional VAE, and LGCP. Training on LGCP was often unstable, consistent with findings of prior work, so fewer methods are reported.
Credit (d= 25)
Training discretization +
10-step random
10-step equidistant
10-step uniform
100-step uniform
Algorithm J. Evaluation steps -
10
100
10
100
10
100
100
PIS TB VarGrad
-671.68
17
-667.05
1037.03
-1171.35:14.90
-1130.57
-606.61 :..0
-1301 ,50=..6
-911
-1318.
14=22.18
-898
98
-1281
+9.74
-1179.87=30.r
6631.08-=2...
-1279.95=14.0
-817.
25
40=10.49
638.67414.12
-1261.02
-1172.46a82.0
-631.84+8.20
PIS LP
-1175.16=14.14
01110001
-1183.60=11.0
-669.30=16.34
-1174.25<17.000
-1114.56=44.66
-608.29
TB LP
-1342.96=2.77
-913.63+18.57
13600.88:32.44
-956,97
-1300.17#8.20
=1265,11=26.7
-600.19=2.79
VarGrad + LLP
-1303.67
-876.12
-1323.16as..
-933
-1281.1520.40
-1186.96#.60.e
-651.98eo.u
Cancer (d= =d=31 31)
Training discretization
10-step random
10-step equidistant
10-step uniform
100-step uniform
Algorithm Evaluation steps +
10
100
10
100
10
100
100
PIS TB VarGrad
-6. 16.6011.000 80
9.51
-7.7320.3
9.1521145
-8.9414877
-4933.64
17.64_12.51
-48.57
-28 1271
-59.7714..2
18.81118881
-35.4221876
5.32
-28.97
-5.8410.5
-31.8312.8
-11.7625.00
-30.0913.76
3966.701357.24
9.41=1.77
PTS LP + LP
-12.27:2.99
7.30=1.02
-16.8723.36
6.35
-11.51
-3649.25-=22.70
19.47:1.87
29.39.43.0
-4.33
.1.522228.7
16.600
-24.3311.4s
2778.75-334.22
11.5610.59
VarGrad +
-30.55.10144
-1.6911.94
-28.1614..0
-6.0514.59
-26.3611.30
-978.6021.0.2
13.4112.19
VAE (d = 20)
Training discretization
10-step random
10-step equidistant
10-step uniform
100-step uniform
Algorithm Evaluation steps
10
100
100
10
100
100
PIS
-1
-104.52
-117.6821.99
-104.29.0.as
-117.74.m.2
-154.8820.31
-102.71
TB VarCrad
-161.97.1.8
-149.86
4.0
-162,7214.85
7149.7610:75
-160.491u.6
-161.9015.0
-142.88.0.14
-122.04:1.06
-109.45
1.00
-170.5114.75
-150.7147.45
-120.981um6
-133.3014.9
-104.1620.07
PIS LP
115.90aa.64
-100.20=..a
-115.8140.a1
-100.13
-115.83+n.sa
-120.61s144
-99.341a.40
TB + LP
1s
-114.80=1.o7
-140.72a1.10
-114.81+1.99
-137.54+261
-136.61+2.66
-109.25=1.08
VarGrad + LP
-118.52
-102.24=0.77
-138.51+0.70
-113.49
-117.35+a.90
122..22a.70
-9..01+0.a7
LGCP 1600)
Training discretization +
10-step random
10-step uniform
100-step uniform
Algorithm Evaluation steps
10
100
10
100
100
PIS TB
-1471.16za
-1467.8522.59
-1471.49
-1729.56:103.00
14455.1420.78
-1618.86#3 o1
-1617.35=1.34
-1617.33=6.54
-1666.87.13.78
-1619.89.6.
TB + LS
-1878.87
-1880.52+13.07
-1877.13+18..
-1703.60=36.3e
-1891.62+4.77
PIS + LP
472.2441.66
343.18 182.33
-211.79+208.40
473.74=1.14
TB + L.P
332. 082.1644.44
461.53+1.16
337.3 10.13
11931.42:2636.33
468.68+-4.13
TB + T. + LP
341.53++0.36
472.43:20.42
341.63=a.16
-77-64477.72
451.89+33.28
29

---

Published in Transactions on Machine Learning Research (01/2026)
D.2 Additional figures
Cancer
Credit
Langevin parametrization
+ Langevin parametrization
LLnggvin parmmstizatiin
Langevin parametrrizatin
FL-DB
Unitin
(100 stop integrazion!
000-
ELBO (100-ttpp
460-
200-
20
Training stops
Training
stops
Training discretization steps
Training
stops
Figure 8: Results extending main text Fig. 5: Credit and Cancer densities.
VAE
Langevin parametrization
+ Langevin parametrization
ELBO (100-step integration)
DB
TB
4x 10
FL-DB
Uniform disc.
VarGrad
Random disc.
3xx 102
PIS
102
20
50
100
20
50
100
Training discretization steps
Training discretization steps
Figure 9: Results exlending main text Fig. on the conditional VAE larget density.
25GMM)
VAE
LLngevin parametrization
Langevin parametrization
parametrization
Langevin parmmetiizatinn
gap (100-stpp integratio))
ELBO Integrazion)
FL
Randen die Eiuiuiiam
Vietina
3xc00
11.05
DS
Randen dac.
FIS
Equidstar Joc.
100
10
50
100
100
Training discretization steps
Training discretization steps
Training discretization steps
Training discretization steps
Figure 10: Comparison of Random and Equidistant distretizations on the 25GMM (unconditional) and VAE (conditional) largets.
30

---

Published in Transactions on Machine Learning Research (01/2026)
25GMM (d- 2)
250MM (d=
Langevin parumetrization
Langevin parametrization
Langevin parametriizatio
Langevvi parametrization
Pis
gop (100. step integration)
11P
FL-DS
DB
TB
DB
Unifom dàc.
Training ())
Raxlen
10V
10
10a
Training discretization steps
Training discretization steps
Runtime (5)
Runtime (4)
Funnel (d 10)
Funnel (d 10)
pacmmetrization
Langevin porametrization
Langevin parametrization
Langovin parametrization
gap (100-step Integration)
10
FL.
PIS
FLDS
TB
nB
TR
Unitoom
10
Training
100
107
Training discretization steps
Training discretization stops
Runtime (s)
Runtime (6)
Figure 11: Results extending main text Fig. Efficiency of nonuniform coarse discretizations Funnel and 25GMM densities.
25GMM (d 2)
Funnel (d 10)
Unifeem
Unifonn disc.
10
Equidiitant disc.
10a
Equidstart disc.
Random
Randona
ELBO gap
ELBO gap
10P
10
20
40
50
70
90
100
10
30
40
70
90
100
Integration steps
Integration steps
Figure Results exlending main text Fig. Evaluation models trained with Niaai sleps using varying numbers of integration steps.
31

---

Published in Transactions on Machine Learning Research (01/2026)
D.3 VAE reconstructions
MNIST data
VAE reconstruction (pretrained encoder)
7
2
1
6
4
1 4
7
0
4
4
9
6
9
0
6
9
0
]
5
0
5 9
6
9
0
|
5
10-step Uniform training Reconstruction from 100-step integration
10-step Uniform training Reconstruction from 10-step integration
7 5
9 8
7
2
4
4 S
0
1
S
S 9 0
0
0
9
0
10-step Random training Reconstruction from 100-step integration
10-step Random training
Reconstruction from 10-step integration
7 9
7
4
4 5
59
0
5
0
8 9
0
6
0
10-step Equidistant training
10-step Equidistant training
Reconstruction from 100-step integration
Reconstruction from 10-step integration
7
7 2 4
9
q
7 7
7
O
4
4
9
0
1
5
S 9
9
0
1
8
Figure 13: Mode of decoder p(. | =) evaluated on encoded latents the VAF experiment: input data æ reconstruction using sampled from the pretrained VAE encoder (top row) and reconstructions using sampled from diffusion encoders (next three rows).
32


Reviews:
[{"review_id": "1eAxp3RD8U", "reviewer": "Reviewer_srtk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n* Conceptual unification. The asymptotic view clarifies how seemingly different discrete objectives relate to diffusion-like path measures; this helps practitioners reason about when an objective is appropriate and suggests principled hybrids (e.g., PIS).  \n* Actionable guidance. The finding that training can safely use coarse/non-uniform discretizations while evaluation uses fine grids is practical and easy to adopt; the paper substantiates this point with careful ablations across datasets and objectives. \n* Breadth of objectives and targets. The work does not champion a single loss; instead it positions TB/DB/VarGrad/FL-DB/PIS within one family and evaluates them side-by-side on synthetic and conditional targets, including higher-D LGCP. \n\n\n**Weaknesses** \n\n* *Theoretical assumptions.*\nThe asymptotic equivalences rest on a vanishing step-size limit. In practice, training uses 2–20 steps while evaluation may use 100+ steps. It would help to quantify finite-stepsize error (e.g., via non-asymptotic bounds or empirical convergence curves versus step size) to indicate where the theory is a good proxy for practice. (The figures provide qualitative trends, but a bound or controlled ablation would strengthen the claim.) \n* *Clarity on “non-uniform” schemes.*\nThe paper mentions “random” and “equidistant” discretizations; it would help to explicitly define the sampling procedures and any bias-variance trade-offs they introduce (e.g., are random grids re-drawn per minibatch? fixed over training? how are early vs late intervals weighted?). The captions point to these setups, but a short algorithm box would remove ambiguity.  \n* *Metrics and downstream impact.*\nMost results use ELBO/BO gaps and reconstructions. Are there downstream sampling-quality metrics (e.g., for image VAEs) or lower-variance estimators of log-likelihood that better capture the benefit of the path-space view? The tables for VAE/Cancer are helpful, but an end-task comparison (or a runtime-vs-likelihood Pareto plot) would make the efficiency story better. \n* *Generality beyond studied targets.*\nThe targets are well-chosen but still modest (e.g., 25GMM/Funnel/VAEs). A short discussion about extending to high-dimensional image or diffusion-policy tasks (and any stability issues anticipated there) would broaden the paper’s impact narrative.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**\n\nThe paper develops a unifying view of several discrete-time policy learning objectives (Trajectory Balance, Detailed/Forward-Looking Detailed Balance, and a variance-reduced objective) by connecting them to continuous-time diffusion samplers via asymptotic analysis. In the small-stepsize limit, the authors show these discrete objectives converge to path-space divergences associated with continuous-time samplers, and they propose a general “policy-induced sampler” (PIS) objective that subsumes common objectives and suggests new ones. They then exploit this lens to argue—and demonstrate empirically—that training with coarser and non-uniform time discretizations can be substantially more efficient without sacrificing evaluation quality, especially when evaluation is performed with many integration steps. \n\n**Contributions** \n\n1. Asymptotic equivalences between discrete-time objectives (TB/DB/FL-DB/VarGrad) and continuous-time path-divergences, yielding a principled bridge between policy-learning and diffusion sampling.  \n2. PIS objective family that recovers TB, DB/FL-DB and VarGrad as special cases and provides a single training handle across tasks.  \n3. Efficiency via coarse & non-uniform training discretization. The paper compares uniform vs. equidistant/random (non-uniform) grids and finds that training on a coarse non-uniform grid and evaluating with a fine grid often yields the same (or better) ELBO/BO metrics at lower training time.  \n4. Empirical validation across synthetic targets (25GMM, Funnel), conditional VAE targets, and higher-dimensional settings (e.g., LGCP). Tables/figures report ELBO/BO gaps and wall-clock trends under different discretizations and algorithmic choices (including a “Langevin parametrization (LP)” variant).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weakness part.", "broader_impact_concerns": "No concerns as far as I know.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bjxkQZMjUx", "reviewer": "Reviewer_RDmC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is interesting to machine learning community that can leverage similar continuous time techniques to speed up discretized training. I think the paper is quite solid in the theoretical and algorithmic fronts but maybe the empirical ablations can be made more solid and convincing as a way to improve the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a theoretical approach to using continuous time sampling techniques for discretized training in practice. The paper derives the theoretical approach, guarantee and algorithm for adapting discretized training to continuous sampling, hence accelerating the sampling procedure significantly. The paper shows improvements over baseline methods for speed and quality)", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "===== *applications to rl cases* =====\n\nIf I understand the methods correctly, the fast sampling comes from the fact that when sampling from the model itself we can leverage continuous time techniques and hence the speed up comes from sampling from a parameterized model itself. If we want to apply the method to the general RL case, where we alternate between sampling from models and the environment, the application becomes more challenging. Is it fair to say so? I think it'd be nice to discuss the implication of such cases because as the title suggests \"policies\", I'd assume it implies applications to RL specific use cases, but I think the applications in the paper are pure generative modeling.\n\n===== *figure 5 right plots* =====\n\nThe figure 5 right plot shows that random discretization provides a good balance between speed and quality, as shown on the parato fronts. However, the comparison between different training methods is a bit unclear to me. Here, I assume is testing for inference time performance and so the model has finished training, hence we are indirectly comparing the quality of base models trained by different methods? Is that correct?\n\nSince we integrate over 100-step, as indicated on the y-axis annotations, how do we trace out a curve for each specific method, i.e., what's the variation here between e.g., the pink triangle dots with different sizes? As a nit point, it'd be nice to visualize better so as to be able to compare different methods' parato fronts in a more informative way.\n\n===== *figure 4 right plots* =====\n\nFor the TB and FL-DB methods, why is there a clear increase in the ELBO gap as the number of discretization step increases from 2 to <10 in the plot? Intuitively the quality of approximation should improve as the discretization step increases, and here it seems that the results are suggestive of noise or something else going up that's not captured by the intuition. Can the authors provide some elaboration on this?\n\n===== *higher d benchmarks* =====\n\nIt seems that the manywell test case for whcih $d=32$ is the highest dimensional test case that the paper investigates. Would it be possible to benchmark in scenarios where the dimensions are even higher? It would also be nice to plot the level of speed up against $d$, so as to better understand the scaling behavior of different methods as a function of the dimension $d$. After all it is of practical interest to handle extremely high dimensional inputs, and understanding trend even for moderate sized $d$ is a great step forward.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PINHeF7GaH", "reviewer": "Reviewer_Yf67", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- Generally speaking, the paper is well written.\n- The first two sections present the background information in a synthetic and well-organized manner.\n- The convergence results, although based on existing results for Euler-Maruyama schemes, are nicely tailored to the objectives used in diffusion samplers.\n- The numerical results provide information on aspects such as rate of convergence and sensitivity to different types of meshes.\n- The numerical experiments cover various examples from the literature and are nicely presented.\n\n**Weaknesses:**\n- It would be helpful for the reader if the paper were more specific when citing references; otherwise, it is sometimes quite difficult to recover what the authors had in mind.\n- The theoretical results seem to be of limited interest for applications because they do not provide any insights in terms of rate of convergence or sensitivity to different types of meshes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies diffusion samplers, which have met great empirical success in various tasks pertaining to the field of generative AI. The main focus is on the connection between discrete-time approaches, which are used in practice (e.g., when combined with RL methods), and continuous-time models involving stochastic differential equations. The authors focus on two types of objectives: global in time and local in time. They show, under suitable regularity assumptions, that in both cases the discrete-time objectives converge to their continuous-time analogues. Numerical experiments on several sampling problems from the literature illustrate the advantages of using nonuniform time discretizations during the training of diffusion models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Here are a few points which need further clarification in the paper:\n\n1. Page 3, “can allow for greatly improved sample efficiency”: With respect to what baseline is the improvement supposed to be?\n\n2. Page 3, point (3): Could you please clarify how this is supported by Fig. 5 (and Fig. 11)? Here it is written “with nonuniform time discretizations” but it seems a bit misleading because in the experiments, you use not just “nonuniform” but more precisely *random* discretizations with a specific distribution, if I understand correctly. I suspect this is important because one could probably construct “nonuniform time discretizations” leading to very bad performance (e.g., if we put almost all the points close to time 0 and then just a big time interval until terminal time). Also, what do you mean by “similar performance to state-of-the-art methods”? Do you simply mean “similar performance as with uniform meshes”?\n\n3. Page 3, “The results in §3 extend classical results on SDE approximations to objectives for diffusion-based samplers”: Could you please be more specific? The literature on discretization of SDEs is very broad and it is, a priori, surprising that no existing result could be applied. On pages 2 and 3, only the book of Kloeden and Platen is cited, but there are many other results. It would be good to clarify which aspect(s) of the problem make it impossible to apply existing results.\n\n4. Page 3, “However, it is close to the setting of maximum-entropy reinforcement learning”: I would recommend writing clearly the control problem in a mathematical formulation (you write “We relegate formal definitions to Appendix B.2” but actually the problem is not really defined precisely even in the appendix), and comparing rigorously the definition with the more standard RL framework. If we include an entropy term in the reward, do we recover exactly your problem or only approximately?\n\n5. Page 6, equation (13): What is the definition of $dX_t$ in this situation? Is the stochastic integral well-defined or is it just an informal derivation? Similarly, could you please clarify the meaning of the last two integrals in (14)?\n\n6. Page 6, here and elsewhere in the paper, many references are given but not precisely enough for the reader to efficiently extract information. For example “A related result was derived by Richter & Berner (2024)”: Which result? Another example: on page 7, “The result can, e.g., be found in Kloeden & Platen (1992)”. More generally, when citing references for specific results or formulas, please quote the corresponding Theorem/Proposition number or equation number from that reference.\n\n7. Page 7, “more relevant for practical applications”: This is perhaps explained somewhere else in the paper, but could you please clarify what such results bring from the application viewpoint? (This is not a criticism, simply an honest question.)\n\n8. Page 8, “The convergence holds for the TB divergence with respect to any $c$”: What do you mean by “with respect to”? Is this comment simply saying that you can apply Proposition 3.3 with $f(x) = (x-c)^2$, or is there something more?\n\n9. Page 10, “While recent work on diffusion samplers has used a discretization with uniform-length time intervals for both integration and training, we vary the time discretization.”: It seems that the impact of using nonuniform time intervals is not discussed in the theoretical analysis of Section 3. For example, Propositions 3.2 and 3.4 only make an assumption on the largest time step going to 0, but they do not distinguish between uniform and nonuniform meshes. Is it possible to analyze theoretically the differences between such meshes to support your numerical findings?\n\n10. Page 19, Appendix B.1 (Assumptions): Could you please provide sufficient conditions ensuring that “all SDEs admit densities of their time marginals (w.r.t. the Lebesgue measure) that are sufficiently smooth such that we have strong solutions to the corresponding Fokker-Planck equations”? This condition is quite abstract and it is hard for the reader to check whether it holds for their model. Also, what condition do we need on the neural network such that these assumptions are satisfied when using a neural network for the drift in the SDE (as you mention below (9))?", "broader_impact_concerns": "Nothing specific.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)