Piecewise deterministic generative models
Authors: Andrea Bertazzi
Dario Shariatian
Umut Simsekli
Eric Moulines
Alain Oliviero Durmus
Venue: NeurIPS 2024 poster (2024)
Keywords: generative modelling
piecewise deterministic Markov processes
time reversals
Primary area: generative_models
TL;DR: We introduce a novel class of generative models based on piecewise deterministic Markov processes, a family of non-diffusive stochastic processes consisting of deterministic motion and random jumps at random times.

Abstract: We introduce a novel class of generative models based on piecewise deterministic Markov processes (PDMPs), a family of non-diffusive stochastic processes consisting of deterministic motion and random jumps at random times. Similarly to diffusions, such Markov processes admit time reversals that turn out to be PDMPs as well. We apply this observation to three PDMPs considered in the literature: the Zig-Zag process, Bouncy Particle Sampler, and Randomised Hamiltonian Monte Carlo. For these three particular instances, we show that the jump rates and kernels of the corresponding time reversals admit explicit expressions depending on some conditional densities of the PDMP under consideration before and after a jump. Based on these results, we propose efficient training procedures to learn these characteristics and consider methods to approximately simulate the reverse process. Finally, we provide bounds in the total variation distance between the data distribution and the resulting distribution of our model in the case where the base distribution is the standard $d$-dimensional Gaussian distribution. Promising numerical simulations support further investigations into this class of models.

Piecewise deterministic generative models
Andrea Bertazzi Dario Shariatian
Umut Simsekli", Eric Moulines 1.3 Alain Durmus'
École Polytechnique, Institut Polytechnique de Paris
INRIA, CNRS, Ecole Normale Supérieure, PSL Research University MBZUAI
andrea. a yeetaar edu
Abstract
We introduce novel class of generative models based on piecewise deterministic Markov processes (PDMPs), a family of non-diffusive stochastic processes con- sisting of deterministic motion and random jumps at random times. Similarly diffusions, such Markov processes admit time reversals that turn out to be PDMPs as well. We apply this observation three PDMPs considered in the literature: the Zig Zag process, Bouncy Particle Sampler, and Randomised Hamiltonian Monte Carlo. For these three particular instances, we show that the jump rates and ker- nels of the corresponding time reversals admit explicit expressions depending on some conditional densities of the PDMP under consideration before and after a jump. Based on these results, these characteristies and consider methods to approximately simulate the reverse process. Finally, we provide bounds in the total variation distance between the data distribution and the resulting distribution of our model in the case where the base distribution is the standard d-dimensional Gaussian distribution. Promising numerical simulations support further investigations into this class models.
propose efficient training procedures to learn
1 Introduction
Diffusion-basee generative models [Ho et al., 2020, Song et al., 2021] have recently achieved state- of-the-art performance in various fields of application [Dhariwal and Nichol, 2021, Croitoru al., 2023, Jeong et al., 2021, Kong et al., 2021]. In their continuous time interpretation [Song et al., 2021], these models leverage the idea that diffusion process can bridge the data distribution p. to base distribution и, and its time reversal can transform samples from π into synthetic data from p... Anderson [1982] showed that the time reversal of a diffusion process, i.c., the backward process, is itself diffusion with explicit drift and covariance functions that are related to the score functions of the mame-narginal densities of the original, forward diffusion. Consequently, the key element of these generative models is learning these score functions using techniques such as (denoising) score-matching [Hyvärinen, 2005, Vincent, 2011].
In this work we propose new family of generative models which use piecewise deterministic Markov processes (PDMPs) noising processes instead of diffusions. PDMPs were introduced around forty years ago [Davis, 1984, 1993] and since then have been successfully applied in various fields, including communication networks [Dumas et al., 2002], biology [Berg and Brown, 1972, Cloez, Bertrand et al., 2017], risk theory [Embrechts and Schmidli, 1994]. and the reliability of complex systems [Zhang et al., 2008J. More recently, PDMPs have been intensively studied in the context of Monte Carlo algorithms [Fearnhead et al., 2018] as alternatives to Langevin diffusion- based methods and Metropolis-Hastting mechanisms. This renewed interest in PDMPs has led to the development of novel processes, such as the Zig-Zag process (ZZP) [Bierkens et al., 201 2019a], the Bouncy Particle Sampler (BPS) [Bouchard-Côté al., 2018], and the Randomised Hamiltonian
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

and Sanz-Serna, 2017]. PDMPs offer several advantages com- pared Langevin- based methods, such as better scalability and reduced computational complexity in high-dimensional settings [Bierkens et al., 2019a]. In the context of generative modelling PDMPs offer several potential advantages over diffusion processes. key strength their ability to effec- tively model data distributions supported on constrained or restricted domains. By adjusting their deterministic dynamics, PDMPs can easily incorporate boundary behaviour, making them straight- forward to implement in such settings [Bierkens et al., 2018, Davis, 1993]. Similarly, PDMPs can model data on Riemannian manifolds by employing flows that respect the manifold's s geometry (see, e.g., Yang et al. [2022] for PDMP on the sphere). Moreover, PDMPs are well-suited for modelling data distributions that combine a continuous density and a positive mass on a lower dimensional manifold [Bierkens et al., 2022].
Monte Carlo (RHMC)
Our contributions are the following:
1) Leveraging the existing literature on time reversals of Markov jump processes [Conforti and Léonard, 2022], characterise the time reversal of any PDMP under appropriate conditions. It turns out that this time reversal is itself a PDMP with characteristics related to the original PDMP; Proposition 1.
2) We further specify the characteristics of the time-reversal processes associated with the three aforementioned PDMPs: 7ZP, BPS, and RHMC. For these processes, Proposition 2 shows the cor- responding time-reversals are PDMPs with simple reversed deterministic motion and with jump rates and kernels that depend on (ratios of) conditional densities of the velocity of the forward process before and after jump. In contrast to common diffusion models, the emphasis is distributions of the velocity, similar to the case of the underdamped Langevin diffusion [Dockhorn et al., 2022], which includes additional velocity vector akin to the PDMPs we consider. Moreover, the struc- ture of the backward jump rates and kernels closely connects to the case of continuous time jump processes on discrete state spaces [Sun et al., 2023, Lou et al., 2024].
3) We define our piecewise deterministic generative models employing either ZZP, BPS, or RHMC as forward process, transforming data points to a noise distribution of choice, and develop method- ologies to estimate the backward rates and kernels. Then, we define the corresponding backward process based on approximations of the time reversed ZZP, BPS, and RHMC obtained with the estimated rates and kernels. In Section 4 we test our models on simple toy distributions,
4) We obtain a bound for the total variation distance between the data distribution and the distribu- tion of our generative models taking into account two sources of error: first, the approximation of the characteristics of the backward PDMP, and second, its initialisation from the limiting distribution of the forward process; see Theorem 1.
2 PDMP based generative models
2.1 Piecewise deterministic Markov processes
Informally, a PDMP [Davis, 1984, 1993] on the measurable space B)R)) is stochastic process that follows deterministic dynamics between random times, while at these times the process can evolve stochastically on the basis of a Markov kernel. In order to define PDMP precisely, we need three components, called characteristics of the PDMP: a vector field th RR TR. - R.D,
which governs the deterministic motion, a jump rate X R4 x RD + R.. which defines the law of random event times, and finally a jump kernel Q : R4 x R x B(RD) - [0, 1), which applied at event times and defines the new state of the process. Let give informal description of the evolution of a PDMP Z., clarifying the role of the three characteristics. Suppose at time T E R4, the PDMP is at state E IR D that is ZT z. The deterministic motion of the PDMP is described by the ODE dZ = (T + s, Zr+e)ds for $ > O, with initial condition Zr- = 2. We introduce the differential flow 9o (t, s. 2) + ( which solves the ODE in the sense that (()), = (TT + s, (2)) for s > O. The process evolves deterministically according to ip until the next event time T + 7, where is a random variable with law PP(7 ZT 2) exp(- ((PT,T++())ddu)) i.e. the exponential distribution with non-homogencous rate + (())) We can, at least in principle, simulate 7 by solving
T T= inf ( 1>0: / X(T + u (z))dd > E
(I)

---

where E ~ Exp(1). The process is then defined on T by Zr+ = .T,TT+ (Zr) for t e (0, At time T + the process jumps to new state that is drawn from the Markov kernel Q, hence we set 7 Q(T + T, (T.T++(2). ). A realisation of the path of PDMP for given time horizon can then be obtained following this procedure (see also Algorithm in Appendix C.1 for pseudo-code). The formal construction of PDMP can be found in Appendix A.1.
Typically a PDMP has several types of jumps, which can be represented by a family of jump rates and kernels (A,, Qi).e A PDMP of such type can be obtained with the construction we have described by setting
Att, 2) E)(t,2) 2(t,2,d=))= E (e(t, 2(t,2)
Qi(t, 2,(t,,,dz')
(2)
An alternative, equivalent construction of PDMP with X, Q satisfying (2) is given in Appendix A.2. Finally, we say a PDMP is homogeneous (as opposed to the non-homogeneous case we have de- scribed) when the characteristics do not depend on time, that 4 RD + DD, : ED + R+, and Q : IR x _( - [0, IJ. In all this work, we suppose that the PDMPs that we consider are non-explosive in the sense of Davis [1993], that they are such that the time of the n-th random event goes to +oo as n + +o0, almost surely (see Durmus et al. [2021] for conditions ensuring this).
We now introduce the three PDMPs we consider throughout the paper. All these PDMPs are time- homogeneous and live on state space of the form E Rd x V, for V C Rdd assuming Vo E V. Then, Ze can be decomposed as Z, = (X,, Vi), where X₂ E R2 is the component of interest and has the interpretation of the position of particle, whereas V22 E V is auxiliary vector playing the role of the particle's 's velocity. In the sequel, if there is no risk of confusion, we take the convention that any E _2 x V, and we write = (x, v) for I E Rd and v E V. All the PDMPs below have stationary distribution of the form <(dx) v(d)) where π has density proportional to x. +> e e-((w), for 2b _ + F continuously differential potential, and is simple distribution on V for the velocity vector. In our experiments we take π to be the standard normal distribution, while v is the standard normal when V = R or the uniform distribution when V is compact set. Figure 1 shows sample paths for the position vector of the three PDMPs we introduce below.
Figure 1: Trace plots for ZZP (left), BPS (centre), RHMC (right). In all cases X 1 and Ty 10.
The Zig-Zag process The Zag process (ZZP) [Bierkens et al., 2019a] is PDMP with state space E² = Rd x [1. The deterministic motion is determined by the homogeneous vector 0) i.c. the particle moves with constant velocity v. For i E {1.....d) d} we
field 3 (²x,,0) define the jump rates (2 v) == (r:0n((z))+ + Ar, where (a)+ max(0, a). O denotes the i-th partial derivative, and X 3 0 is user chosen refreshment rate. The corresponding (deterministic) jump kernels are given by (2r v), (dy, duo)) = 6 (2, (dy, duc), where 6, denotes the Dirac measure E E. Here, 27 is the operator that reverses the sign of the i-th component of the vector to which it is applied, i.e. T.C
(11 Vu...., U;-1, -G,, U,t1.-.., Mta). The ZZP falls within our definition of PDMP taking X, Q as in (2). As shown in Bierkens et al. [2019a], the ZZP has invariant distribution
v, where v is the uniform distribution over {=1]d. Moreover, Bierkens
et al. [2019b] shows that for any X, > 0 the law of the ZZP converges exponentially fast to its invariant distribution e.g. when is a standard normal distribution.
The Bouncy Particle sampler The Bouncy Particle sampler (BPS) [Bouchard-Côté et al., 2018] PDMP with state space is EB_ XBB where VB Rd or VB = =dd11 := {v Rd ||e||| 1).
3

---

The deterministic motion is governed as ZZP by the homogeneous vector field defined for (I,v) v) E E by 40 (z, of random events: reflections and refreshments. Reflections enforce that m0(2, v) = =<)))()) is the invariant density of the process, where x (dx) x exp(- (Leerrr)) is a given distribution and v either a standard normal distribution when V IRd or the uniform distribution on 5 when V 5d-1. Reflections are associated to the homogeneous jump rate (z. v) + (Z v) (v, (z) while refreshments are associated to (z. e) + > (z. v) X for X > O. The corresponding jump kernels are QP v). (dy, dw)) = "ar.0Bo) u) (dy, dw) , Q7 2). (dy, dw)) = 5n (ddd)))(ddw) where BB V- (((o:.Ve()))//WWeo)))
0)". Now we introduce two jump rates which correspond to two types
(x)) The operator TB reflects the velocity e off the hyperplane that is tangent to the contour line of a passing though point z. The norm of the velocity is unchanged by the application of œ and this gives the interpretation that L is an elastic collision of the particle off such hyperplane. As observed in Bouchard-Cöté et al. [2018], BPS requires a strictly positive X, avoid being reducible, that is to make sure the process can reach any area of the state space. Exponential convergence of the BPS to its invariant distribution was shown by Deligiannidis et al. [2019], Durmus et al. [2020].
Randomised Hamiltonian Monte Carlo Randomised Hamiltonian Monte Carlo (RHMC) [Bou- Rabee and Sanz-Serna, 2017] refers to the PDMP with state space E" = Rd Rd which is characterised by Hamiltonian deterministic flow and refreshments of the velocity vector from the standard normal distribution. The flow is governed by the homogeneous vector field defined by (Ir, u) + 4 v) (u, V=) where &p is the potential of т. The jump rate coincides with the refreshment part of BPS, i.e,, it is the constant function AZ : (ar, v) + X 0 and jump kernel Q" ((x, u), (dy, dw)) 8 (dy) When the stationary distribution ι is a standard Gaussian, the deterministic dynamics (Iz, Ut):20 satisfy dz, vt du, solution 2, = Tg cos(t) + Un sin(t) and 1/ = -rg sin(t) + to cos(t), where (zo, vo) is the initial condition. It is well known that Hamiltonian dynamics preserve the density u(z,, e) (())(()) [Neal, 2010], where v is the standard normal distribution, while velocity refreshments are necessary to ensure the process is irreducible. Exponential convergence of the law of this PDMP to μ was shown in Bou-Rabee and Sanz-Serna [2017].
dt, which for t >> 0 has
Remark (Noise schedule) Similarly to diffusion models we can introduce a noise schedule 3(t) that regulates the amount of randomness injected at time t. This can be achieved using the time change of a given PDMP with characteristics (Ø, X, Q) as forward process, resulting in the PDMP with characteristics
( a. Ag, for (,(1, (()) 2) and >a(1, 2) = (())
2.2 Time reversal of PDMPs
In this section characterise the time reversal of PDMP. This key result, stated in Proposi- tion 1, is essential to be able to use PDMPs for generative modelling. The time reversal of a PDMP (Z))re(o.T,)) with initial distribution M4 is the process that at time E [0, Tyl has distribution Ho -r, where (4g P. denotes the law of Z,. It follows that the law of the time reversal at time T f is yo. which is the key observation in the context of generative modelling. Characterisations of the law of time reversed Markov processes with jumps were obtained in Conforti and Léonard [2022] and in the following statement we adapt their Theorem 5.7 our setting, showing that the time reversal of a PDMP with characteristics (#, A, Q) is a PDMP with reversed deterministic motion and jump rates and kernels satisfying (3).
Proposition 1 Consider a non-explosive PDMP (Z))g2 with characteristics (0) X, Q) and initial distribution (i on RDD In addition, let T be a time horizon. Suppose that Tb is locally bounded, 2t, X(t,2) 2) is continuous in both its variables, and TTy E[A(t, Zi)ltt < co. Assume the technical conditions H3, H4, postponed to the appendix. Then, the corresponding time reversal process is a PDMP with characteristics (Φ) X, Q), where T (1,,) (TT -, 2, and X, Q
are the unique solutions to the following balance equation: for almost all 1 E Ty).
p(yr X (t, v) y. dz) = (d= - 1, (T - t, 2, dy))
(3)
where PtPP stands for the distribution of Ze starting from po.
The proof is postponed to Appendix A.4. The condition H3 is standard in the literature on PDMPs
[Davis, 1993] and is verified for ZZP, BPS, and RHMC. H4 is a technical assumption on the do-

---

main of the generator of the forward PDMP and has been shown to hold e.g. for the ZZP. In the next proposition we derive expressions for the backward jump rate and kernel satisfying (3) corre- sponding to a forward PDMP with characteristics with the same structure as those of 77P, BPS, and RHMC. We state the result assuming the PDMP has only one jump type, but the generalisation to the case of & > jump mechanisms of the form (2) can be immediately obtained applying Proposition 2 to each pair (As, Qi) for E .J. We refer to Appendix A.6 for the details.
Proposition 2 Consider non-explosive PDMP (X,, Vi with characteristics (Φ, X, Q) and ini- tial distribution ug & 1 on R2dd. In addition, let T be time horizon. Suppose that o and X satisfy the same conditions as Proposition 1. in particular the technical conditions H3, H4 postponed to the appendix. Suppose in addition that for any t E (0, y the conditional distribution of V, given X2 has a transition density (zr,)) + (e(l))) with respect some reference measure p on R.
(I) (Deterministic jumps). Suppose Q(v, w) (dz, (ri,,v)) =5y(dz)0 (dd)) where for any y E R.., By : F + Rd is an involution which preserves pleg. i.e., R, B, and peer (d.Byw) p (dw). Then for almost all E [0. T and any (v, w) E ER2 such that pr, (wly) > 0 it holds that
PT,-o((Rywle) X(T, - (v,,,,))), (1,, w), (dx, dv)) = 5, (dzx) u (du). PT, (wly)(wly)
(2) (Refreshments). Suppose Q( w) (dx, du)) = (ddd))((dull)) where v is a transition kernel on TRd x B(R"),, and A(t, (v, (u,w)) w)) = X(t, v). Suppose also for any y E Rd, vl-ly) is absolutely continuous with respect to p Then for almost all [0, Ty and any (y. w) E RRdd such that PT, (wly) > 0 it holds that
(dr/an(a)\wlv)) A(T,-t.V). (tt, (v, (dr, du)) = 6y (()))) (dv). pry-((wly)
The proof is postponed to Appendix A.5. We remark that we consider that 1 is a distribution on Rd also when 14 (V) such that (V) = 1. Applying Proposition 2 we are able to derive explicit expressions for the characteristics of the time reversals of ZZP, RHMC, and BPS. The rigorous statements and their proof' can be found in Appendix A.7. For ZZP and BPS we assume the following condition on n the limiting distribution for the position vector of the forward process.
for V C RR, in which case the reference measure can simply be chosen
HI Recall x(x) x e v(a), It holds that vb E (R²) and supzer" [V < +00.
This assumption is satisfied e.g. by any multivariate normal distribution. For BPS and RHMC we suppose that for any / E (0, Ty), the conditional distribution of V. given X, has a transition density (z,u) + pe(o(ar) with respect to the Lebesgue measure. Moreover, for all samplers we assume H4.
Time reversal of ZZP In order to apply Proposition 2 we additionally assume that [(r (x) oc for all i I,...,d, d. We find that the deterministic motion is defined by T 2 (v,ww) w) = (-w,00 07T for any (y, w) E R2d, while the backward rates and kernels are for i == 1,...,, d and for all (y, w) E R2d such that pr, ( > 0.
(v. = Pr, Pr,-o((7 Pr, (wly)
X X7 R, T7 w), (dx, = (y (d),, v). (4)
Time reversal of BPS Whereas in Appendix A.7 we consider the case where the velocity of BPS is initialised on Sd-1, we can formally apply Proposition 2 to the case of y is the standard d- dimensional Gaussian distribution assuming that J VV(z))du, (x) < co. The drift of the backward BPS is clearly the same for the backward ZZP, while jump rates and kernels are for all t [0, Tyl and (y, w) E Rgdd such that pT, (r,y)( 0
PT,- (2B w/y) v) PT,
(t, w))
XP (y, BB))
P w), (dx, do)) .(v. un) (dz, du).
(w)-)(wly)
5

---

v(w)
(1,,)))))) Ar PT, p (wly)
( (y, w), (dr, - (1, (5)
Time reversal of RHMC. The deterministic motion of the backward RHMC follows the system of ODEs " (z, v) = (-v, V=(())" which, when the limiting distribution я is Gaussian, has solution 2t zo cos(t) Up (t)(t) and v 20 sin(t) + uo cos(t). The backward refreshment rate and kernel coincide with those of BPS as given in (5).
Remark (Variance exploding PDMPs) Similarly to the case of diffusion models [Song et al., 2.21], can define variance exploding PDMPs choosing (0(z) 0 for all Rd. that is when π (dx) is the Lebesgue measure. In this case, the deterministic motion of RHMC coincides with ZZP and BPS, and all three processes have only velocity refreshment events.
2.3 Approximating the characteristics of time reversals of PDMPs
In Section 2.2 we showed that the backward jump rates and kernels of ZZP, BPS, and RHMC, involve the conditional densities of the velocity vector of the forward process given its position vector at all times t E [0. Ty. These conditional densities are unavailable in analytic form, hence in this section develop methods to learn the jump rates and kernels of our time reversed PDMPs. In Appendix D we give the pseudo codes and more detailed descriptions of the training procedure for our models, together with a comparison with diffusion models.
Approximating the jump rates of the backward ZZP via ratio matching In the case of ZZP, we need approximate for any E {I...., d), the rates in (4). Since the terms X²(x, (f(()) are known, it is sufficient to estimate the density ratios (I, v, := pr v het/pppvvlp (v|z) for all states (z, v) such that Pt (v]2r) > O. To this end, we introduce a class of functions {s" Rd x 1- x [0, Ty] Rd : E θ} for some parameter set θ C R:de and aim to find a parameter 8, E O such that for any E dI, the i-th component of s denoted by 5i (-), is an approximation of r?? We then approximate the backward ZZP by using the rates (1, (Ir, v)) = 5 (r, v, T - 1) X (ar, 2Fv). To address the problem of fitting 0, consider different loss functions inspired by the ratio matching (RM) problem considered in Hyvärinen [2007].
From a discrete probability density P1 on {-1, RM consists in learning the d ratios + p+ (=) for E {1....,dl dd. This problem was motivated in Hyvärinen [2007] as means to estimate Р1 without requiring its normalising constant, similarly to score matching applied to estimate continuous probability densities [Hyvärinen, 2005]. In our context we are interested only in the ratios, hence opposed to Hyvärinen [2007] we do not model the conditional distributions (1r, v) + (c(wx)) but directly the ratios r Adapting the ideas of Hyvärinen [2007] to our context, we introduce the function G G (1 + r)) r)- - and define the Explicit Ratio Matching objective function
f-" TT
d
Σ E E V.tt) - G(ri(X, V,,t))}
(g(0)
dt (t)
(6)
(X, 2VV.. - ((. t))3
where Ty] + R. is probability density, and (Xe, Vi is ZZP initialised from 14, @ 11, This objective function considers simultaneously the square error in the estimation of both (z, (,, t) + 7,, v, t) and (z, e,t) + RV,, f), where the function G im- proves numerical stability, particularly when one of the two ratios is very small. Clearly (0) if and only if of (z. (,, r2(2, v, t) for almost all v, t and all i. Moreover, the choice of G allows us to optimise without knowledge of the true ratios, as shown in the following result.
Proposition 3 It holds that arg ming (0) arg ming for
d 6(0)= (:00-/" T w() [(()=((drr0 Σ EE(C (
- (7)
i=1
where Vi)ER, is 77P starting from p. & Unif([ (nif((-1,114) -1,11

---

Therefore we aim to solve the minimisation problem associated with 41 which has for empirical counterpart
N E
0 + N
Σ 1=
(s Van,
V T") G (VV, r")) 2G (X.i, V r"))
where {T"} alisations of the ZZP respectively starting at the n-th training data point with velocity Vo where (Vo" 1 Jn=1 are i...d. observations of 1ni
In=1 are i..d. samples from w, independent of {(XF. Vi") 1200 1 which are N iid. re-
Notice that the loss above has computational cost increasing linearly in d because d + evaluations of the model are needed for each datum. This can be improved considering an estimate for the ratio which does not take input the whole velocity vector (see Appendix D.1 for the details). This variation has computational cost that constant the dimension, but might have lower accuracy.
Approximating the characteristics of BPS and RHMC For BPS and RHMC, Proposition 2 shows that if we aim to sample from the backward process, we have to estimate both ratios of the conditional density of the velocity of the forward PDMP given its position at any time / E [0, Tyl. and also to be able to sample from such densities as prescribed by the backward jump kernel (5). In order to address both requirements, we introduce parametric family of conditional probability distributions {po : E Q of the form (x, U, t) + po(v|z, t), where e C Rd,, which we model with the framework of normalising flows (NFs) [Papamakarios et al., 2021]. The advantage of NFs lays in their feature that, once the network is learned, it is possible both to obtain an estimate of the density at given state and time, and also to generate samples which approximately from (z, v,t) + Pe (u(z). However, training conditional NFs can be challenging in high dimensions.
Focusing on BPS, we now illustrate how we can use NF' to learn the backward jump rates and kernels. We aim to find parameter 0 such that pep (r] I) a good approximation of p. the conditional density of the forward BPS with respect to the Lebesgue measure. We choose to optimise 0 following the maximum likelihood approach, which gives the theoretical loss
(0) -0)- L/'' dt (() (log Po (V)(,,,)),
(8)
where : [0, - R2 is probability density, and (X,, Vi)ego is BPS initialised from 14 @ v, with denoting the density the d-dimensional standard normal distribution. The optimal parameter O can then be found minimising the empirical counterpart of (0):
8 arg min
1 N
Σ
log po (V (x,,,"""))
where {T" alisations of the ZZP respectively starting the t-th training data point with velocity VJ", where {VT) J are i.i.d. observations from the multivariate standard normal distribution. Once we have obtained the optimal parameter 8B, we can define our approximation of the backward refresh- ment mechanism of BPS taking the rate X (t, (I) = X, x (t, (y. w), (dx, du)) = pas (vly. T (dr)dd. Similarly, we estimate the backward reflec- tion ratio of BPS as (1, v)) = XP (r, BBv) x pap /tt...,-t////.n
J== are i.i.d. samples from w. independent of {(XT, Vi") (2)) which are N i.i.d. re-
and the kernel
2.4 Simulating the backward process
We now discuss how we can simulate the backward PDMP with exact backward flow map (t, v) + (1,(2,,)) v) and jump characteristics and Q that are approximations of the jump rates and kernels of the time reversed PDMPs obtained discussed in Section 2.3. We recall that the backward rates have the general form X(t, (z, v)) = 50(x, v, T, f))(2, Rv), where sg is an esti- mate of a density ratio and E is a suitable involution. In principle such PDMP can be simulated following the procedure described in Section 2.1, but the generation of the random jump times via
(1) requires the integration of A(1, ( v)) with respect to 1. This cannot be achieved since is defined through neural network. A standard approach in the literature (see c.g. Bertazzi et al.
7

---

[2022, 2023]) is to discretise time and (informally) approximate the integral in (1) with finite sum. Here we focus on approximations based on splitting schemes discussed in Bertazzi al. [2023], adapting their ideas to the non-homogeneous case. Such splitring schemes approximate a PDMP with a Markov chain defined on the time grid (f.......... with la = 0 and i. = T The key idea is that the deterministic motion and the jump part of the PDMP are simulated separately in a suitable order, obtaining second order accuracy under suitable conditions (see Theorem 2.6 in Bertazzi et al. [2023]). Now, we give informal description of the splitting scheme that we use for RHMC, that is based on splitting DJD in Bertazzi et al. [2023], where D stands for deterministic motion and for jumps. We define our Markov chain based on the step sizes ( NY. where 6, = 1 1--1- Suppose we have defined the Markov chain on {E=))E( for n < N and that the state at time fn is (It,, Vt,,). The next state is obtained following three steps. First, the particle moves according to its deterministic motion for half-step, that we define an interme- diate state (Zt.,: it..) = / (/t Ut,)) Second, we turn our attention to the jump part of the process. In this phase, the particle is only allowed to move through jumps and there no deter- ministic motion. This means that the rate is frozen the value X(t., + 6n+1/2, (āt., ūe )) and thus the integral in (1) can be computed trivially. The proposal for the next event time is then given by Tn+1 ~ Ext(X(t,, + 1/22, (āt.. D)). If Tn+1 & 82+1, we draw w ~ Q(t, 6,+1/2, (āa.. Üt )) and update Ut,, w, else leave Vt,, unchanged. Finally we conclude with an additional half-step of deterministic motion, letting (Ir (tn.a) ))- = --- (Tt,, Ut,,). ). We refer to Appendix C.2 for detailed description of the schemes used for each process together with the pseudo-codes.
3 Error bound in total variation distance
In this section, we give a bound on the total variation distance between the data distribution μ., and the law of the synthetic data generated by PDMP with initial distribution x & V and approximate characteristics obtained, e.g., with the methods described in Section 2.3. We obtain our result com- paring the law of such PDMP to the law of the exact time reversal obtained in Section 2.2, that is the PDMP with the analytic characteristics of Proposition 2 and with initial distribution L(Xr,, Vr,), ). ie. the law of the forward PDMP at time T when initialised from μ. @ v. In Theorem 1 below we then take into account two of the three sources of error in our models: (i) the error introduced initialising the backward PDMP from the limiting distribution of the forward, (ii) the error due to the approximation of the backward rates and kernels. For simplicity we neglect the discretisation error caused by the methods discussed in Section 2.4.
We shall assume the following condition, which deals with the error introduced by initialising the backward PDMP from &
H2 The forward PDMP with semigroup (P)))20 is such that there exist 7, C >> 0 for which
|| @ p. @ v vPilITV & Ce
Informally, H2 is verified for some C < co when N is multivariate standard Gaussian distribution for ZZP and BPS if the tails of μ. are at least as light as those of オ, while for RHMC it is enough if μ. has finite second moments. We refer to Appendix E.1 for a more detailed discussion on this aspect. We are now ready to state our result.
Theorem Consider a non non-exxlosive PDMP (X,, Vi) with initial distribution μ., &v, stationary distribution N vv, and characteristics (9) X, Q). Let T be a time horizon. Suppose the assumptions of Proposition as well as H2 hold. Let (X V. ()eeoo.) be a non-explosive PDMP initial distribu- tion n & v and characteristics ( X, Q), where (t,, (z. v)) = TTT, t, v)) for all t E [0, T and (x, v) E 2d Then it holds that
l|.- - -CXX,), & Ce + 2E 1 - exp (-1(" To
gI, ( (x)dt( ).
1-exp
(10)
where
(x(x,v)= (x (X)( -(1,),)))))))000000))))))),) |
+ (1,,
X(4, (z,))) (11)
2
and 5, Q are as given by Proposition 1.
8

---

Dataset
i-DDPM
BPS
RHMC
ZZP
Checkerboard
2.49 = 0.98
1.96 1.51
4.27
0.81 0.19
Fractal tree
2.25 + 1.70
+
1.12 + 0.58
Gaussian grid
23.19
4.59
4.31
4.43 4.05
Olympic rings
2.03 = 1.60
2.07 + 1.19
2.41 2.24
1.43 + 0.86
Rose
6.77 5.81
1.92 1.57
2.16 1.59
0.90 = 0.35
Table 1: MMD J, in units of 1e-3, averaged over runs, with the corresponding standard deviations.
Fractal tree
7ZP
i-DDPM
Olympic rings
BPS
i-DDPM
Figure 2: Comparative results on two-dimensional generation of synthetic datasets.
The proof is postponed to Appendix E.2. The first term in (10) is caused by initialising the process from x @ v, while the second term represents the eπrо introduced by the approximate jump rate and kernel Q. For the sake of illustration we obtain a simple upper bound (10) in the case of ZZP (for the details see Appendix E.3). We assume the conditions of Theorem I are satisfied and also that the expected error of the learned rates X bounded by constant, in the sense that [[]r??(,, V,,
(X,, Vr, Ty - 1) (X,, X VV.]. & M for all E [0, [yy and {1....,d) d). The latter condition is similar to the standard assumption asked on the approximation of the score in diffusion models [Chen al., 2023]. Under these conditions we obtain the bound
||p. ((XT,)llTV Ce- dd.
(12)
Numerical simulations
In this section, we test our piecewise deterministic generative models on simple synthetic datasets.
Design We compare the generative models based on ZZP, BPS, and RHMC with the improved denoising diffusion probabilistic model (i-DDPM) given Nichol and Dhariwal [2021]. For all of our models, we choose the standard normal distribution as target distribution for the position vector, as well for the velocity vector in the cases of BPS and RHMC. The accuracy trained generative models is evaluated by the kernel maximum mean discrepancy (MMD). We refer to Appendix F for a detailed description of the parameters and networks choices.
Sample quality In Table report the MMD score for five, 2-dimensional toy distributions. We observe that the PDMP based generative models perform well compared to i-DDPM in all of these five datasets. In particular, ZZP and i-DDPM implemented with the same neural network archi- tecture, hence 7ZP appears to compare favourably to i-DDPM with the same model expressivity. The results of Table are supported by the plots of generated data shown in Figure 2, illustrating how 77P and BPS are able to generate more detailed edges compared to i-DDPM.
In Figure 4, we compare the output of RHMC and i-DDPM for very small number of reverse steps. We observe how in this setting the data generated by RHMC are noticeably closer to the true data distribution compared to i-DDPM. This phenomenon is observed also for BPS shown in Table 2, and is intuitively caused by the refreshment kernel, which is able to generate velocities that correct wrong positions. Respecting this intuition, ZZP does not perform as well BPS and RHMC for a small number of reverse steps since its velocities are constrained to 1,,, Nonetheless, ZZP generates the most accurate results in our experiments given large enough number of reverse steps.
Table 2 and Figure show that PDMP-based models require smaller computational time to gener- ate samples of a given quality compared to i-DDPM. This is the case because PDMP models require

---

considerably less backward steps than i-DDPM, although each step is more expensive (see Table 2). Additional results can be found Appendix F, including promising results applying ZZP to the MNIST dataset.
Table 2: MMD for various number of backward steps, Rose dataset.
Figure 3: MMD J. runtime (ms) per method, Rose dataset.
steps
i-DDPM
BPS
RHMC
ZZP
101
HDOPK
2
696.28
165.09
26.48
358.25
10-1
192.17
22.18
3.00
89.49
10
45.08
5.48
1.75
11.31
10
25
12.34
1.58
0.60
1.20
100
8.72
3.66
1.72
1.04
10-1
time/step(ms))
3.94
45.8
15.1
11.2
Total Compulational Time lims)
Gaussian grid
i-DDPM, 2 steps
i-DDPM, 10 steps
RHMC, steps
RHMC, 10 steps
Rose dataset
i-DDPM, 2 steps
i-DDPM, 10 steps
RHMC, steps
RHMC, 10 steps
Figure 4: Comparing RHMC and i-DDPM for small number of reverse steps.
5
Discussion and conclusions
We have introduced new generative models based on piecewise deterministic Markov processes, de- veloping theoretically sound framework with specific focus three PDMPs from the sampling literature. While this work lays the foundations this class methods, it also opens several direc- tions worth investigating in the future.
Similarly to other generative models, our PDMP based algorithms are sensitive to the choice of the network architecture that is used to approximate the backward characteristics. Therefore, it is crucial to investigate which architectures are most suited for our algorithms order to achieve state of the art performance in real world scenarios. For instance, in the case of BPS and RHMC it could be beneficial to separate the estimation of the density ratios and the generation draws of the velocity conditioned on the position and time. For the case of 77P, efficient techniques to learn the network in a high dimensional setting need to be investigated, while network architectures that resemble those used approximate the score function appear to adapt well the case of density ratios. Moreover, there are several alternative PDMPs that could be used as generative models and that we did not consider in detail in this paper, as for instance variance exploding alternatives.
10

---

Acknowledgments and Disclosure of Funding
AB, EM, AD are funded by the European Union (ERC-2022-Sye 101071601). US and DS are funded by the European Union (ERC. Dynasty, 101039676). Views and opinions expressed are however those of the authors only and do not necessarlly reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. US is additionally funded by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d' a'veni"" program, reference ANR-19 (ERR-19P31A-000) (PRAIRIE 3IA Institute). The authors are grateful to the CLEPS infrastructure from the Inria of Paris for providing resources and support.
References
Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Ap- plications, 12(3):313-326, 1982. ISSN 0304-4149. doi: https//woo..ap////0000000001111) 90051-5.
Howard Berg and Douglas Brown. Chemotaxis in escherichia coli analysed by three- dimensional tracking. Nature, 239(5374):500-504 1972.
Andrea Bertazzi, Joris Bierkens, and Paul Dobson. Approximations of Piecewise Deterministic Markov Processes and their convergence properties. Stochastie Processes and their Applications, 154:91-153, 2022.
Andrea Bertazzi, Paul Dobson, and Pierre Monmarché. Piecewise deterministic sampling with split- ting schemes. arXiv:2301.02537, 2023.
Joris Bierkens, Alexandre Bouchard-Côté, Arnaud Doucet, Andrew B. Duncan, Paul Fearnhead, Thibaut Lienart, Gareth Roberts, and Sebastian J. Vollmer. Piecewise deterministic markov pro- cesses for scalable monte carlo on restricted domains. Statistics & Probability Letters, 136:148- 154, 2018. The role of Statistics in the era of big data.
Joris Bierkens, Paul Fearnhead, and Gareth Roberts. The zig-zag process and super-efficient sam- pling for bayesian analysis of big data. Annals of Statistics, 47, 2019a.
Joris Bierkens, Gareth O Roberts, and Bvere--Adrtt Zitt. Ergodicity of the zigzag process. The Annals of Applied Probability, 29(4):2266 2301, 2019b.
Joris Bierkens, Sebastiano Grazzi, Frank van der Meulen, and Moritz Schauer. Sticky PDMP sam- plers for sparse and local inference problems. Statistics and Computing, 33(1):8, 2022.
Nawaf Bou-Rabee and Jesús María Sanz-Serna. Randomized hamiltonian monte carlo. The Annals of Applied Probability, 27(4):2159-2194 2017.
Alexandre Bouchard-Côie Sehastian J. Vollmer, and Amaud Doucet. The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method. Journal of the American Statistical Association, 113(522)8555-867, 2018.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, 2023.
Cloez, Bertrand, Dessalles, Renaud, Genadot, Alexandre, Malrieu, Florent, Marguet, Aline, and Yvinec, Romain. Probabilistic and Piecewise Deterministic models in Biology. ESAIM: Procs, 60:225-245. 2017.
Giovanni Conforti and Christian Léonard. Time reversal of markov processes with jumps under finite entropy condition. Stochastic Processes and their Applications, 144:85-124, 2022. ISSN 0304-4149. doi: https://doi.org// ort./0..///000/ 1ttg//1..01....1.
1ttp:/0..11j...pa22.000 (/11///pp..221.1.. 20.0.00.0..
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor lonescu, and Mubarak Shah. Diffusion models in vision: A survey. IEFE Transactions Pattern Analysis and Machine Intelligence, 2023.
11

---

M. H. A. Davis. Stochastic Models. Journal of the Royal Statistical Society. Series B (Methodological), 46(3): 353-388, 1984.
Markov Processes: A General Class of Non-Diffusion
M.H.A. Davis. Markov Models & Optinization. Chapman & Hall/CRC Monographs Statistics & Applied Probability. Taylor & Francis, 1993. ISBN 9780412314109
George Deligiannidis, Alexandre Bouchard-Côté, and Arnaud Doucet. Exponential ergodicity of the bouncy particle sampler. The Annals of Statistics,
06 2019.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-based generative modeling with critically- damped langevin diffusion. In International Conference Learning Representations, 2022.
Vincent Dumas, Fabrice Guillemin, and Philippe Robert. Markovian analysis of additive-ineres increase multiplicative-decreas. algorithms. Advances in Applied Probability, 34(1):85-111, 2002. doi: 10. 20.1239/aap/10 10tpa///119919911..11
Conor Durkan, Artur Bekasov, lain Murray, and George Papamakarios. Neural spline flows. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Alain Durmus, Amaud Guillin, and Pierre Monmarché. Geometric ergodicity of the Bouncy Particle Sampler. The Annals of Applied Probability, 20(5):2069-209) 10 2020.
Alain Durmus, Arnaud Guillin, and Pierre Monmarché. Piecewise deterministic Markov processes and their invariant measures. Annales de l'Institut Henri Poincaré, Probabilités et Statistiques, 5743):1442 1475, 2021.
Paul Embrechts and Hanspeter Schmidli. Ruin estimation for a general insurance risk model. Ad- vances in Applied Probability, 26(2):4044422, 1994.
Paul Fearnhead, Joris Bierkens, Murray Pollock, and Gareth O. Roberts. Piecewise deterministic markov processes for continuous-time monte carlo. Statistical Science, 33(3):386-412, 08 2018.
Jonathan Ho. Ajay Jain, and Picter Abbeel. Denoising Diffusion Probabilistic Models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840-6851. Curran Associates, Inc., 2020.
Aapo Hyvärinen. Estimation of normaazzd statistical models by score matching. Journal of Machine Learning Research, 6(24):695-709, 2005.
Aapo Hyvärinen. Some extensions of score matching. Comput. Stat. Duta Anal., 51:2499-2512.
2007.
M. Jacobsen. Point Process Theory and Applications: Marked Point and Piecewise Deterministic Processes. Probability and Its Applications. Birkhäuser Boston, 2005. ISBN 9780817642155
Myeonghun Jeong, Hyeongju Kim, Sung Jun Cheon, Byoung Jin Choi, and Nam Soo Kim. Diff- TTS: A Denoising Diffusion Model for Text-to-Speech. In Proc. Interspeech 2021, pages 3605- 3609, 2021. doi: 10.21437/mierspeech.222 2021-469.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference Learning Representations (ICLR), San Diega, CA, USA, 2015.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv:2310. 16834, 2024.
12

---

Radford M. Neal. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 54:113-162, 2010.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 166-8171.
PMLR. 18-24 Jul 2021.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lak- shminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57): 1-64, 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Yang, Zach De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library, 2019.
Franccois Rozet et al. Zuko: Normalizing flows in pytorch, 2022.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio matching under the breg- man divergence: unified framework of density ratio estimation. Annals of the Institute of Statistical Mathematics, 64, 10 2011. doi: 10. 10.10077/1046--01-0034434-
Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, and Hanjun Dai. Score-based continuous-time discrete diffusion models. In The Eleventh International Conference on Learning Representa- lions, 2023.
Pascal Vincent. connection between matching and denoising autoencoders. Neural compu- tation, 23(7):1661-1644 2011.
Jun Yang, Krzysztof Latuszyíski, and Gareth O Roberts. Stereographic Markov chain Monte Carlo. arXiv preprint arXiv:2205.12112. 2022.
Huilong Zhang, Franccois Dufour, Y. Dutuit, and Karine Gonzalez. Piecewise deterministic markov processes and dynamic reliability. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 222:545 551, 12 2008. doi:
13

---

Appendix
The Appendix is organised as follows. Appendix includes the details and the proofs regarding Section 2.1 and Section 2.2. Appendix B contains details and proofs regarding the framework of density ratio matching. Appendix C gives details and pseudo-codes for the exact simulation of our forward processes (see Appendix C.1), and also for the splitting schemes that are used to approxi- mate the backward processes (see Appendix C.2). Appendix details the algorithms used to train our generative models and outlines the computational differences with the popular framework of denoising diffusion models. Appendix E contains the proof for Theorem and some related details Finally, Appendix F contains the details on the numerical simulations, well as additional results.
A PDMPs and their time reversals
A.1
Construction of PDMP
Here describe the formal construction of PDMP with the characteristics (Φ, X, Q). To this end, consider the differential flow ip (t, s, 2) + Pr.++(2), which solves the ODE, d = d(t s, 2t++))dd ds for e i.c. 2t++ (Ze). We define by recursion on n E N the process on (Z)) on [0, Tw] and the increasing sequence of jump times (In)eeN starting from an initial state Zo and setting To = 0. Assume that (T:))e(o......) and (Zi)cEjoTT.) are defined for some n E N. We now define (Z)),E(T..TT.HI)) First, we define
Tn+1 = inf { 1> L M(T, + u, PI,, (TT,))do (2t,))t d Entij
inf
(13)
where En+11 1 ~ Exp(1), 1) and set the n + 1-th jump time T = Tn + Tu+l- The process is then defined on (Tn; Tu+1) by ZT, PT.,.T.+ee (Tr,) ). for & E [0, Tn+1). Finally, we set Z ~ Q(Tx+1:
).). The process (Z))20 Markov process by [Jacobsen, 2005,
Theorem 7.3.1].
A.2 Construction of PDMP with multiple jump types
In this section we describe the formal construction of non-homogeneous PDMP with the charac- teristics (0, X, Q) where X, Q are of the form
x(1 - Σ x(()z), Q(422dz))= Σ (r(t,z) 21(1),dz)). x(1,))
(14)
Recall the differential flow &0 : (1,, 8, 2) + (2),+(()), which solves the ODE, dat+s +( + 2t+o ds for 8 3 0, i.e. Ztte = (1,,4+ Similarly to the case of one type of jump only, we start the PDMP from an initial state Zo. assume is defined as (Zz)ce0oTT.) on [0, Tn] for some n E N, and we now define define (Z))te(T...T u]. First, we define the proposals (T++ for next event time as
The = inf far(t> {>
0: L >;(T, + U. PT.. (ZT, 2 E
where E Exp(1) for fl,...,(). .). Then define 1 arg argminie Fati and set the next jump time to
Tut1 Tn + -ra The process is then defined on [Tn, Tnt1) by ZT,+ TT (ZT,) for E [0, Ta41). Finally, we set ZT... ~ Qi- (Tn+1, PPT,, Tn+4741 )..))
A.3 Extended generator
In order to obtain the generator of PDMP, Theorem 26.14 of Davis [1993] requires "standard conditions" on the characteristics (see conditions (24.8) in Davis [1993]). We state these conditions for non-homogeneous PDMP in the next assumption.
14

---

H3 The non non-homogeneous characteristics (.. X, Q) satisfy the following conditions:
1. & is locally Lipschitz and the associated flow map up has infinite explosion time;
2. À is such that + M(Fr.(+((z)) is integrable on [0, [0,=(z, =(x,t( t)) for some e(x,t) t) > 0 and all
(t, x) E RR+ E.
3. Q is measurable and such that Q 2, {x}) for all (t, R4 X
4. Let (Tw)ne(o.....) be the random sequence of event times of the PDMP and define N₂ k== LISTT It holds that [Nd] < for all (t, x) IR+ X E.
Notably, the PDMP required to be non-explosive in the sense that the expected number of random events after any time starting the PDMP from any state should be finite. These conditions are veri- fied for all the three PDMPs consider as forward processes. Assuming HB we can apply Theorem 26.14 14 in Davis [1993] to the homogeneous PDMP obtained including the time variable, which gives that the extended generator of the non-homogeneous PDMP with characteristies (Φ, X, Q) is given
by
/ (()))
LT((2) = (b(() V<]())) + x(1,))
(15)
for all functions f E (t) that the space measurable functions such that
Mf = f(Z.) f(Zo) I. L.f(Z.)ds
a local martingale. We also introduce the Carré du champ g.(f.g) = Li(fg) - fflgg gLf, with domain (T', {f.g: : fg E dom({2) which in the case of PDMP with generator (15) takes the form
(=) / (f(y) f()))(((v) g(t))
A.4 Proof of Proposition
In order prove Proposition we apply Conforti and Léonard [2022, Theorem 5.7] and hence in this section we verify the required assumptions. Before starting, we state the following techni- cal condition which omitted in Proposition and is assumed in Conforti and Léonard [2022, Theorem 5.7].
H4 It holds (R C. (o)( any t E R..
We now turn to verifying the remaining assumptions in Conforti and Léonard [2022, Theorem 5.7]. The "General Hypotheses" of Conforti and Léonard [2022] are satisfied since we assume the vector field & is locally bounded, the switching rate (t, 2) + A(t, is continuous function, and the jump kernel Q is such that Q(t. .). is probability distribution. In particular these assumptions imply that
/ (LA 2t) dyy < sup 1E(O.T).I= / p
sup 1E]
x(t,z) < for all p 0.
Then, Conforti and Léonard [2022, Theorem 5.7] requires further integrability condition, which is satisfied when
(LA - wF)uop(((e)MM33331 < 00.
It is then sufficient to have that
L' E[A(L, (.)d. < 00
(16)
Finally, Conforti and Léonard [2022, Theorem 5.7] requires some technical assumptions which now discuss. Introduce the class of functions that are twice continuously differentiable and com- pactly supported, denoted by C (IR)). and for f C² (Rd) consider the two following conditions:
S." ( < co,
(17)
15

---

J." J (oP co
for all CRR)).
(18)
We define F {ff E (E) (17), (18) hold J. We need to verify that J (E). Let start by considering (17): we find
No 2dde))
<f (oPi(dz)) [K((1,)), V[)()) + x(1,=) [ [u(u) u(=)]Q((), 2,)) dl.
Since f E (E) we have that (t, 2). V=())) is compactly supported and hence integrable, while the second term is finite assuming 1f Z))]dt cc. Under the latter assumption, (18) can be easily verified.
A.5 Proof of Proposition
Let us denote the initial condition of the forward PDMP by μo nD @ pp. First of all, notice that, for PDMP with position-velocity decomposition and homogeneous jump kernel, the flux equation (3) becomes
Po Pr(dy, dw) X (t, (v. w)) (t, (v. w), (dx, du)) Mnu Pzdzz, do)A(t, (z, ()) u), (dy. dw))
where 7 = T Moreover, since the jump kernel leaves the position vector unchanged we obtaín that this is equivalent to
Po (ddwly) (t. (y, w)) (t, (y, w), (dæ, de)) = (oPi(duly) (v, e))Q((z v), (dy, dw)),
where HoP (dwlly is the conditional law of the velocity vector given the position vector time with initial distribution po-
Suppose first that Q((,, w). (dz, du)) (dx)), (du) for an involution By Then find
(dwayy 5 (t, (y. w)) Q (t, (y, w), (dx, de)) = (l (y, R,),), (d (dv)
where used that die, (du) 5 dee, (dw) since By is involution. Under our assumptions have
No PodB pr( C, (er no PoPaldwly) pr(wll))
since assumed Me (dw) MYes w) Hence find for any (v. w) R2ad such that pi(wly) y) > 0
[t, (y. w)) (t, ( (v, w). (dz,de))
Pa(B,wly) X(E, (y. (dx) (dv). pr(wly) y)
This can only be satisfied if
(1,, = wl( x(i, (v. R,)), Q(t, (y, w), = 6y(dr)d (dv). pr(wly)
Consider now the second case, that Q( w), (dx, do)) = (ydd))u(drl)) and A(t. (y,, w)) = A(t. v). The flux equation (3) can be rewritten as
no ((dwlly))) 5 (y, w) ( (y. w), du)) = nn P:((lly))A(L (ddbl)))))(.,)6,((dd)((((ba)
Under our assumptions have v(dw)j y) (wl/dd))))wvv))) (dw) and no (
Pa(w)p YY (dw) for some measure We Hence for any (2. w) I2dd such that pr(wlv) 0
we obtain
Te, (us @1, (v, w), (dx, (d)) (dw/dxYa))wlw) X(1, (ddd
Pe(wly)
This is satisfied when
A (t, (y, w)) = (dv/dnYe) (E,
T(t, (y. w), (dx, du)) = (d(((dr)))
pr(wly)
16

---

A.6 Extension of Propasition 2 to multiple jump types
Proposition 2 considers PDMPs with one type of jump, while here we discuss the case of charac- teristics of the form (14), which is e.g. the case of ZZP and BPS. In this setting we can assume the backward jump rate and kernel have similar structure, that
2 T1(1,2) (1(,,,,d'),
-(t,))=
()) (t,,,,='') xx(t,))
Σ ((,,2)
1=1
in which case the balance condition (3) can be rewritten as
No pr Σ Je w) = no no Pr,-((d)) Σ - 1, (1
1=1
is then enough that
PoPry-e(dy) X (t, -
holds for all i E (l...., .....,,) f}. follows that it is sufficient to apply Proposition 2 to each pair (Axs Qi)
obtain (5, ;, Q:) such that (3) holds.
A.7 Time reversals of ZZP, BPS, and RHMC
In this section we give rigorous statements regarding time reversals of ZZP, BPS, and RHMC. For all samplers rely Proposition and hence focus verifying its assumptions. In the cases of ZZP and RHMC we assume the technical condition H4 since proving it rigorously is out of the scope of the present paper. We remark that this be proved with techniques as in Durmus et al. [2021], which show H4 in the case of BPS.
Proposition 4 (Time reversal of ZZP) Consider a TIP (X,, V,) with initial distribution NA @ U, where (n and invariant distribution a 8 v, where has potential 1p satisfy- ing H1. Assume that H4 holds and that J p.. (dx))))::bbb() < oc for all 1,.... Then the time reversal of the TZP has vector field
(2,, v)x =
(-o,0))
and jump rates and kernels are given for all (y, w) E Rddd such that Pr,-i(wly) > 0 by
77,1,, w)) = (r,-1((f w())
pr,-((uly) X7 (y. 1710, (7, w), (dx, v)) = (V. w) (dx; v)
for = 1....,d
Proof We verify the conditions Proposition 2 corresponding to deterministic transitions and rely Appendix A.6 to apply the proposition to each pair (X4, Qf). First notice the vector field 2(z,v) = (v. 0)T is clearly locally bounded and (t,z) + A(z, v) is continuous since is continuously differentiable. Moreover, the ZZP can be shown to be non-explosive apply- ing Durmus et al. [2021, Proposition 9]. Then, we need to verify (16). First, observe that (X, Vo) < (X,)], Then
E(a((X))) = E (oo((X)) + 1'' -o, V.r +(X, - Xo)))
(((0, + [ 1Xxx Xo. v + s(X, Xo)) |s
where e; is the i-th vector of the canonical basis. Notice that X, Xol & W.d. Thus we find
E((2((X)))] < B[(a,((Xo)] + ttd sup 2 (2((z)]
mER4
17

---

and therefore
Ty Va sup IV9)(=))) 2
[ Vi TT
[X,)l
ØERe
Since B,, (x)) < co and because we are assuming HI, we obtain (16). Finally, notice that Pt (ddo())) is absolutely continuous with respect to the counting measure on {1. -11", which is clearly invariant with respect to S.
Proposition 5 (Time reversal of BPS) Consider BPS (Xt, Vi)LEJo,T,1 with initial distribution
μ4, Vv where v Unif(Sd-1), and invariant distribution x v, where x has potential vb satisfying
HI. Assume that E,, [I (X)][ < co. Then there exists density pe(w pe(ly)) == (SGooP))(dll))//ttt Moreover, the time reversal of the BPS has vector field
(2, (-v.0)²
while the jump rates and kernels are given for all t. w E [0. Ty] x Rdd such that pr, pr, (e(wl)) by
P (y, w)) = Pr(3BBulu wlll)
(y. w)) 2 w), (dx, duj) = 8(2 28 (dr, du),
Pi(w(g) y)
(1,)))))))= X pr, (wly) where i = =TT- -t
(1, (y.w), w) (dx; dv)) no no (duly) (dr))), (19)
Remark 3 Under the assumption that v is the uniform distribution on the sphere, it is natural to take piet =V, which gives that dufap%s When is the d-dimensional Gaussian distribution, the natural choice is to let p be the Lebesgue measure and hence we obtain a rate as given in (5).
and hence the backward refreshment rate is in (19).
Proof We verify the general conditions of Proposition 2, then focusing on the deterministic jumps and the refreshments relying on Appendix A.6. The BPS was shown to be non-explosive for any initial distribution in Durmus et al. [2021, Proposition 10]. Since X(1, v)) = A(x,
(u, +x))))++ with a similar reasoning of the proof of Proposition 4 we have
E[A(X,, [O) = E (1V. + S'' (V,, V2 (Xo + 8(, XX)
-xo)((x-Xx)ddd Xo))ds)
Taking advantage of [V,] = 1 we have [X, Xol < l and thus we find
B[A v4) < E (Xo)) + / V2 v(x + s(X, - Xx)
E
(X))) + sup | (t)) xER
This is sufficient to obtain (16) since V < oo and we assume HI. Moreover, H4 holds by Durmus et al. [2021. Proposition 23]. Finally notice that (d is absolutely continuous with respect to Mzel U '), which satisfies Uref All the required assumptions Proposition 2 are thus satisfied.
V
B (#2 (x)u)) = (Te(()) for all x,, E Idd x Sd-1
Proposition 6 (Time reversal of RHMC) Consider a RHMC (X,, Vo)GEJ,;T with initial distribu- tion μ. & v. where v is the d-dimensional standard normal distribution, and invariant distribution क @ v, where n has potential vi E C² (Rd)). Suppose that H4 holds and that for any y E Rd, Pi(dwly) is absolutely continuous with respect Lebesgue measure, with density pe(wly). Then the time reversal of the RHMC has vector field
TH = (-v, V((()))
while the jump rates and kernels are given for all (y, w) E Ig2d such that pT, -e > 0 by
T (t, (y. w)) > "T,-r(wly))
v(w)
T (t, (y. w), (dx, du)) ( (dx)de.
18

---

Proof First of all, RHMC is non-explosive by Durmus et al. [2021, Proposition 8). Then O is locally bounded and (16) trivially satisfied. Finally, we can take M to be the Lebesgue measure.
B Density ratio matching
B.1 Ratio matching with Bregman divergences
We now describe a general approach to approximate ratios of densities based on the minimisation of Bregman divergences [Sugiyama et al., 2011], which as discuss is closely connected the loss of Hyvärinen [2007].
For differentiable, strictly convex function f we define the Bregman divergence B By(r, f(r)- f()) (r) s). Given two time-dependent probability density functions Rdd p. wish to approximate their ratio r(z, u, t) = (((x,))//((t.)) for t E [0, Ty with a parametric function 8g Rd x Rd x 1y R2 by solving the minimisation problem
min / Fy ( Vz, (), No (X,, Vr, ()) dt,
where the expectation is with respect to the joint density de (xt v), that is (Xt. V/) while w is probability density function for the time variable. Well studied choices of the function / include e.g. f(r) r log r r, that is related to a KL divergence,
fr) = (r -1)² related to the square loss. f(r) log (1 + r) log(1 r), which corresponding to solving a logistic regression task. Ignoring terms that do not depend on 0 we can rewrite the minimisation as
min ( 0
('w(0) ((t (Ep, (s Vr, ()) V2, f(s Ve, k]] - - (so Ve, ())) dt.
Notably this independent the true density ratio and thus it a formulation with similar spirit to implicit score matching. Naturally, in practice the loss can be approximated empirically with Monte Carlo average.
B.2
Details and proofs regarding Hyvärinen's 's ratio matching
B.2.1
Connection Bregman divergences
In the next statement, we show that the loss & defined in (6), or equivalently its explicit counterpart Eg (see Proposition 3), can be put in the framework of Bregman divergences.
Corollary Recall G(r) = (1 + r1 and ler f(r) (r-1)//2. The task mino (x()) is equivalent to
min
Ep,
By By(G(s) V. V2,1), G(n((., V.,)))
+ ( t)), G(r 3 LVV f)))
Proof
The result follows by straightforward computations.
B.2.2 Proof of Proposition 3
The proof follows the same lines as Hyvärinen [2007, Theorem 1]. We find
(p( C+
w(t) E
RVv,.)
-2G(ri(X, -.0.:44Hh..000022200000000020001 (V,,))
19

---

Algorithm 1: Pseudo-code for the simulation of a homogeneous PDMP
Input : Time horizon T, initial condition (xr, v).
Set f = 0. (Xo, Vo) (z, v);
while < do
draw E' ~ Exp(1);
compute the next event time, that is R4 such that So ((pu(Xe, Vi)) E;
it + 7 > T then
set ZT = (r-)(Z));
else end
simulate Zet ~ (Z),,
set = 1 +
end
where is a constant independent of Then plugging in the expression of G we can rewrite the last term
Ep.
G( (I V2)
(2
Σ
R. 1)) Pe(x, v) Dr (2,
dx
=E(+1]a
,
Ep. G (X,
(X,, V),t))
pe pr( Vt. + pc (X,; R.....
Therefore we find
lE(0) C S T wtt) Σ Epp G2 V.,)) G2 RV v.))
2G Vr, t)) Pa (X,, Vj))
2G V., P.V.)
dt
Pr( Vi) + Pa (X,, Vi)
(X,, Vi) + Pt X1 Ro
= C f w(t) Σ Ep. (X) V2,t)) G² (s, R./V 2G (6(5)((, (x, ()) dt.
C
Simulation of forward and backward PDMPs
C.1 Simulation of the forward PDMPs
The forward PDMPs that we consider can all be simulated exactly by solving the integral (13), at least when the limiting distribution is the multivariate standard normal. This is possible because for each process
easily simulate the random event times well as their deterministic dynamics. The general procedure for the simulation of a homo-homogensuus PDMP up to a fixed time horizon T given in Algorithm 1. In the remainder of the section give additional details the simulation of each process.
The case of RHMC is trivial since the random events have the exponential distribution with constant parameter X, and the deterministic dynamics are given by x, = xg cos(!) + ug sin(t) and -o sin(t) + cos(t), where (æo, vo) is the initial condition. Hence all the steps in Algorithm can be performed and the state (XT, VT) can be casily obtained.
RHMC:
Notice the event rates of ZZP are of the form ( v) = (eę2;)+ when the stationary distri- bution the standard normal. In this case we find that each coordinate of the ZZP is independent, that is the evolution of ((x,):, (V)):) is not affected by ((Xt), (V2),) for i, = 1, ...... d with i + j. Therefore we can simulate each coordinate of the 7ZP in parallel following the procedure in Algo- rithm I. Let us illustrate how one obtain the next event time of the i-th coordinate when the
ZZP:
20

---

process is at (2,, 4.). We have that r solves + + the following quadratic equation for т:
E = for E ~ Exp(1). This gives
T
1
(-1522) - FE =
2
which has solution
(u:a)) __ + 2E.
7 U:3
BPS: In the case of BPS one has to simulate a proposal for both the next reflection event, Q6 and for the next refreshment event, Tr, then accepting the smallest of the two event time. Obtaining the proposal for the next refreshment time is straightforward since the rate is constant. The proposal for the following reflection time
be obtained similarly to the case of ZZP, but noticing that in = 0 for E ~ Exp(1). Noticing that it must be To > (w.n)/1of, this gives the following quadratic equation for Tb:
this case the event rate is \xx, v) (v, +) Then % solves fo (fu, 2)) + [=|
1 2
1 マ (- (1,,,))) E
+
(u, a)
which has solution
((,,x) (v. .) 2/u|2 76
|0|2
C.2
Simulation of time reversed PDMPs with splitting schemes
Here we discuss the splitting schemes we use to discretise the backward PDMPs. For further details on this class of approximations we refer the reader Bertazzi al. [2023].
RHMC: We have already discussed the splitting scheme DJD for RHMC in Section 2.4, and give the pseudo-code in Algorithm 2.
Algorithm 2: Splitting scheme DJD for the time reversed RHMC
Initialise either from (Xu, Vo) & (Xo, Vo) rddx) (dulæ, Ty);
for 72 , N
1 do
(X, V) = pt1 1nt1/2 ( bn vin);
7 7= T in Dat1
Estimate ratio: 8 u(V) p VI 7,;;
Draw proposal T ~ Exp(3dr):
TT+11 & 8n+1 then
Draw ~ po (11X, T);
end
(X V ((n-)) = pll 1/ (X, V):
end
ZZP: For ZZP we apply the splitting scheme DJD discussed in Section 2.4, with the only differ- ence that we allow multiple velocity flips during the jump step similarly Bertazzi al. [2023]. Algorithm gives a pseudo-code.
In the case of BPS, we follow the recommendations of Bertazzi et al. [2023] and adapt their splitting scheme RDBDR, where stands for refreshments, for deterministic motion, and for bounces. We give pseudo-code in Algorithm 4. We remark that an alternative is to use the scheme DJD for BPS, simulating reflections and refreshments the part of the splitting. This choice has the ad advantage of reducing the number of model evaluations.
BPS:
21

---

Algorithm 3: Splitting scheme DJD for the time reversed ZZP
Initialise (Xo, Vo)
π 8 V;
for n 0,.... . 1 do
X X An V.
V Vin
= T in 6n+1
Estimate density ratios: 80 (X, V,t); io 1..., do
With probability (1 - exp(ep((8ap (X.VIT A(X (XXV)VV))) set V R.V
end
X Xtn11 X V
6u+1 V;
end
Algorithm 4: Splitting scheme RDBDR for the time reversed BPS
Initialise either from (Xu, Vo)
@ or (Xo, Vo) m(xx) ( (x, Ty);
for 71 O...., N 1 do
t. V= Estimate density ratio: $2 v -VV/po- (i Ty-ta);
With probability (1 exp(( - X, 6nts )) draw V ~ po- ( Ne,, Ty - tn)
X = X T in 1
1 I.
Estimate density ratio: $1 Po (* v x, 7) 1/ po- V x,
With probability (1 - exp( - 8, 8nt181 3
set V = 38 V:
X x 6.+1 V;
2
Estimate density ratio: 82 v(V)/1g(( vl X 1 tm++1)
With probability (1 - exp(- - 82 Snti )) draw V ~ pa- ( 1x,,, Ty - bn+1):
V fat1 V
end
D Training the generative models
In this section, we present the algorithmic procedures used train our generative models, and outline computational differences with the popular framework of denoising diffusion models. In Table we list the backward rates and kernels of the time reversal associated with each forward PDMP introduced in Section 2.1. In Appendix D.1 we give the procedure used for ZZP: while in Appendix D.2 we focus on RHMC and BPS, which can be trained with the same approach. In Appendix D.3 we compare the training phase of PDMP- based and diffusion baasudd generative models.
D.1 Fitting the b.Z.-based generative model
In the case of ZZP, we only need to approximate the jump rates X7 of the backward process, since we have access the true backward kernel. To this end, we introduce class of functions {s" R2d {-1,1]d x [o,Tyl + P$ 0 E Q} for some parameter set t c dd such that for any i {I....,d) d). the i-th component of 5 (v, f), denoted by sf (v. w. t), is approximation of
Pr, (1877(597
Pry-r
22

---

Process
Backward Rates
Backward Kernels
for I.....d
for i I....,
77P
(1, Pr, (3f (1))
(z, sef
( ((( w)), (dz, (di, = (v..0% (dr, do)
PT,-t (10 13)
RHMC
("t,,s. (v. A,
v(w)
TH(( (t. (y w), (dz, = PT,- (v) (dz)
Pr, Pr, [(w
PT, PT, (18
40 (v)w))
Ap (Pw)
BPS
PT 9)
क. ((v. (dz, (v. (dxx, (t. (v. (dx,, du)) pT, dy(dr)
(y, X, (10 8)
v(w)
Table 3: Time Reversal Characteristics of ZZP, RHMC, BPS.
We learn S by minimising the empirical counterpart of the implicit ratio matching loss & given in (7). We define such empirical loss l as the function & 0 Li (Xp,, Vi., 7 "")m) for
Li(so, (Xpm, Vin 1
GG (s (") (
(G (",,
(20)
J m= which are N i.i.d. re- alisations of the ZZP respectively starting the n-th training data point XE with velocity Vii". where { (Vo' [V n=1
where [T")
In=1 are i...d samples from w, independent of {(x;, Vi")
i.i.d. observations of Unif([-1, 1)d). Algorithm shows the pseudo code for
our training algorithm. We remark that the simulation of the forward ZZP follows the guidelines explained in Appendix C.I.
Algorithm 5: Training loop for 77P-based generative models
Input: Time distribution [0, Ty) model 8
while has not converged do
Get random data batch ( Sample {-") B
Sample {V/' n= ~ (
for n = 1 to B do
L Simulate (X?., V.A by rumning a ZZP from (XB, Vi");
Compute loss (1) + Li(s", (X",, Vi, (=,)) Perform optimization step (0);
Output: trained model 8
computationally efficient model for the ZZP. The computation of i, in each iteration of Al- gorithm 5 requires evaluating d+1 times the model 8 for each data-point, Indeed, the loss function (7) requires evaluating s" for each component flip of the velocity vector. Hence the computational cost of the algorithm scales linearly in the data dimension d.
In order overcome this computational burden build an alternative model leveraging the fol- lowing observation:
(r(vaaa, v-;) = po(p)lz),
(21)
where u- denotes the vector containing all components of other than the i-th. The approximation in (21) is motivated by the fact that we expect the components of the velocity be nearly indepen- dent conditional on the position vector. This is because the position of the process holds most of the information regarding the velocity of each coordinate. Under (21) we find that a good approximation for the ratio r!' (I, t) is given by
(x, t), Pc (-0,))))
Pe (1(v:()
23

---

Hence it is reasonable to estimate the simplified ratios 57 rather than the true ratios (z, v, t). In particular this can be achieved with model s" that requires the current position and time, and only the i-th velocity component rather than the full vector v.
Following the reasoning above we build an alternative model which takes as input the position vector and the time variable and outputs a 2d-dimensional vector, that is {s : R x [0, Tyl - R
)). We introduce the following notation for the output of the neural network s::
(z, t) + (z,, t), s (z, E R
(22)
where o and a are both vectors in Rdd and denote the two, d-dimensional blocks in the output of 8", The model will be trained in such way that sh. t), that is the i-th component of the vector sf (xx (), approximates r (1r, +1,1), i.e. in the case 0; = +1. Similarly, we estimate r (2r, 1,
with 8 (z,
Let us now describe how we can train this model in such a way that the computational cost remains constant in the dimensionality of the data. For any w E {1,-1}, we introduce the projection operator IT defined for any w E [+1] i (1,..., d}. y Rd,, t Ty) as
II, s " (y. t) 2 (w)n) ). (y.t).
(23)
where have sign(w:) + when w; +1 and (g:) = when wa = -. In words, IT (y. t) selects either s4 or s in (22) based on sign(w;) and returns the estimate of the i-th ratio at (y, corresponding to the velocity wy Using the operator I we can re-formulate the loss (7) as
Tr (0) f'dt dt (t)
Σ E G² (1 o (X,, + V. t)) 2G s
The associated empirical loss is &g 0 +s L iSimplo ( (Xp,, Vn V m)) were
i Simple (s" (x", " (t") 1))=
N d MΣC ( (x" (")))")) +
(24)
where {r") >= and ((X,, Vi" ) are obtained as in (20). This simplified model can then be trained using the same procedure shown in Algorithm 5, but where the loss above is used instead of the loss (20). The computational cost for the training of this model is clearly constant in the data dimension d, since only single evaluation of the model 8 is needed for each data-point.
D.2 Fitting the RHMC and BPS-based generative models
In the case RHMC and BPS both the backward rate and backward jump kernel are characterised by (v(z)) We introduce a parametric family of conditional probability distributions (po 0 E θ} of the form (x, f) + pe (u|z, t). where θ C Rds, which we model with normalising flows (NFs) [Papamakarios al., 2021]. as it permits both obtaining an estimate of the density, at a given state and time, and also to generate samples according to this distribution. To train our model, we rely on the maximum likelihood population loss method, as derived in (8), and use its empirical counterpart (9). The final BPS and RHMC training algorithm is given in Algorithm 6. The simulation of the forward BPS and RHMC follows the methods listed Appendix C.I.
D.3 Computational comparison with diffusion models
We provide a short comparison of the computational complexity of our PDMP generative methods with diffusion models, as each generative method admits the same design: a fixed forward process (Xe JogIeT, ran from time 0 to Ty, with XI, approximately distributed as a standard Gaussian N (0 (using the Sariance-Preservee
process in the case of diffusion models [Song et al., 2021]), corresponding backward process (Xx JosTeT, and a generative process (Xf)ogegT) being an approximation to the true backward process, initialized from X (O)
24

---

Algorithm 6: Training loop for RHMC and BPS based generative models
Input:
Time distribution w on [0, Tyl, model po
while 0 has not converged do
Get random data batch {Xx)) mm
Sample {-" ==1 ~V w
Sample (Vo ===1 (2) for n to do
L Simulate (X²,, V running RHMC/BPS from (Xy), Vo");
I 6- - Σ Po (x",,")) (V")
Perform optimisation step on L";
Output: trained model po-
Using conventional notations for diffusion models [Song al., 2021] we denote by (or)ogreT, the variance-preserving noise schedule, such that the distribution of the forward process any time i is given by
x, Va, + v1 -a,G,
(25)
where G. (N (d). The generative process {x is typically defined by denoiser neural network eg, Rdo, trained with the denoiser loss
[ox(X..))
a VI āq
dillusion
where is the time parameter. We display the training algorithm for diffusion models in Algorithm 8. For the sake of comparison we give in Algorithm 7 the general procedure used for PDMP-based generative models.
Algorithm 7: Training loop for PDMP-based
Algorithm 8: Training loop for diffusion- based generative models
generative models
Input: Time distribution w on [0, Tyl, model 8
Input:
Time distribution w on [0, Tyl, noise
schedule {ār)e model e"
while 0 has not converged do
while 0 has not converged do
Get random data batch {XP) Jmzi
Get random data batch {X Jm=1
Sample
{r F2 m==1 &
Sample [-")==> ~ @B
Sample [Vrr ]== ~ (nif([ [=1]
Sample [e"]=]] n== ~ (O,
for 1 to B do
for 1 to B do
Simulate (XPr, V.h by running
Compute
PDMP from (Xp. V());
X Vann X + v1 - Opp e":
Compute loss I
Compute loss
I - B
| |
T²) e=I
Perform optimization step on L";
Perform optimization step on L
Output: trained model "
Output: trained model "
Computational differences in training the models
We compare the training algorithms:
The simulation of the forward process (X, losest, more efficient in the case of diffusion models, since the distribution of X, given Xo is explicitly characterizable as given in (25), whereas the case of PDMPs the complexity scales with the expected number of random jumps in [0, Ty}.
Computing the loss function has the same computational cost (number of evaluations of the network relative to the dimension of the data) for PDMP and diffusion models (assuming we using the simplified model for ZZP, introduced in Appendix D.
25

---

Computational differences in the generative processes Since we are using the splitting scheme for our PDMPs, simulating the backward processes admits a computational complexity growing lin- early with the chosen number of backward steps N, alike diffusion models [Song et al., 2021]. J. The latter models require only single network inference per backward step. However, as measured in Table 2, each backward step is costlier in the case of our PDMP samplers. We provide an explanation for each sampler:
ZZP Using the simplified model introduced in Appendix D.I and Algorithm 3, only one network inference is required per backward step, to approximate the backward rate. How- ever, this approach requires slight modification to the neural network architecture, in- volving doubled channel output and adjustments for the element selection mechanisms (projection operator (23)), resulting in a higher cost per step than diffusion models.
RHMC Using Algorithm 2, one likelihood computation from the normalizing flow is needed, to approximate the backward rate. Additionally, at most random variable must be drawn from the normalizing flow, which approximates the backward kernel.
BPS Using Algorithm 4, three likelihood computations are required from the normalizing flow, approximate backward rates. Moreover, at most two random variables need to be sampled from the normalizing flow, which approximates the backward kernel. The final cost per step depends on the chosen normalizing flow architecture, but is higher for BPS than for all other samplers.
should be noted that, in our experiments, each generative PDMP model requires less backward steps than the diffusion model, leading to smaller overall computational time for equal generation quality, as showed in Figure
E
Discussion and proof for Theorem 1
F.
Discussion on H2
In this section we discuss H2 in the case of ZZP, BPS, and RHMC. For all three of these samplers, existing theory shows convergence the form
||5 (a..)) Pt vllv & C'e CVe v),
(26)
where V : 2d - oo) is positive function and || == sup of (v))(( is the V-norm. When the initial condition of the process is μ4, 8 v, we obtain the bound
||, & uP: - π & ullv & Ce uu @ v(V),
which translates to a bound in TV distance, since we assume > 1. Conditions on и ensuring (26) can be found for ZZP in Bierkens et al. [2019b], for BPS in Deligiannidis et al. [2019], Durmus et al. [2020], and for RHMC in Bou- Rabee and Senz-Serna [2017]. Observe that we can set the constant C in H2 to C C' 14 & vVV) Clearly, C is finite whenever 14, @ v(V)
xc. Since -o, showing C is finite requires suitable tail conditions on the initial distribution fl, & v. Inspecting the results of the papers mentioned above, one can verify that u. @ v(V)= < oo when æ is a multivariate standard Gaussian distribution as long as: (i) the tails of μ, are least light those of for ZZP and BPS, (ii) μ., has finite second moments for RHMC.
V is such that lim s/ V(z)
E.2
Proof of Theorem
First notice that
(27) hence we focus on bounding the right hand side. Under our assumptions, the forward PDMP (Xe, V.),), admits time reversal that is a PDMP (X,, V Vi)eooTTT with characteristics ( x, Q) satisfying the conditions in Proposition 1. Therefore, it holds μ. &V L(Xr, Vr,) and so (27) can be written as
||,, - C(XT,)l & l|u. @ L(XT,, VT,)lITv
||. - L(XT,, & (II VT,) VI,)lIV
26

---

We introduce the intermediate PDMP (Xe (,eto,,)) with initial distribution L(Xr,, VI,) and characteristies (,, In particular, (Xe, (o)T) has the same characteristics as (X,, V.)E)T,.T)) but different initial condition By the triangle inequality for the TV distance find
ll.. ((Xr,) < II (0r))-((s) -(xr, Vr, Mrv [| Vi,) ((X,, lIvv.
VT,))
Applying the data processing inequality to the second term, we find the bound
1/44 - ((XT)) < | VT,)) - L(XT,, VIy + ( VT,) - x 6 u||TV. (28)
The second term in (28) can be bounded applying H2, hence it is left to bound the first term. We introduce the Markov semigroups Pt, P. P. R x Tk22 x B(Red) [0. defined respectively
as P,((2r,)) v). == ((X,,)((e,, Vo) E .). (,((2,)),) = P (c.,0) ((X V.) E .). and Pr( v), == P(a,v) ((x,, Vi) E -). Recall that for any probability distribution 1y on ( )), np((.) Saee (dr, P.(r,,( u),,)) and similarly for np(r-) and n P.(.). Finally, to ease the notation we denote QT, L(XT, VI,) = (w. Pr, Then we can rewrite the first term in (28) as
B(R"d))
|Ix(XTT VT,)= f(XT,, Vr, l|l 1QT, Pr, Pr, Pr, Qr, (dxr, d PI, 6(a,v) PTyl (29)
< /
Therefore we wish to bound [](z.)) Pry 0(z,0) Pr,, ||v A bound for the TV distance between two PDMPs with same initial condition and deterministic motion, but different jump rate and kernel was obtained in Durmus et al. [2021, Theorem 11] using the coupling inequality
||6 (a.e) Pr, (a,,) Pr,lIrv & (a.w))) (Kx,,) V VT,) + (Xr, Vr,)):
and then bounding the right hand side. Following the proof of Durmus al. [2021, Theorem 11] have that a synchronous coupling of the two PDMPs satisfies
(-10"
To aixxe V Vosat) ]
P(z,,)) (Exry, Bei,,,
VI,)+ + KKr,VVr,) < (m,))
1 1cxxp
where
(e(x, v) 2 (1,, (2, Ael, (z, u))
(o,, (1,
(1,,, (z, v)) - T(t, (z, ())
Since c(x,, V.) L(XT, VT,-11 for E [0, Tyl, we can rewrite this bound as
1 - exp (- (-(²" 9T, 9T, Vo)at) ]
P(r,)) (IXT,,, T,, # (XT,, Vir)) < (k,2)
Plugging this bound in (29) we obtain
1- e-exp exp (-1%) T,
01,- Vildt) ]
| VI,) L(TY, Vr 1 & 2E
This concludes the proof.
E.3 Application to the ZZP
Here we give the details on the bound (12), which considers the case of ZZP. First, we upper bound the function gr defined in (1)). We focus on the first term in (11), that is
(T2 A [l, (z, v)) || 2 v). Q² (2, lTv.
gt (z. v)
27

---

We find
(2, (1,
(t (t, (z, .lvv sup
Σ 11 (o)E
(T (2,
(2,,))))))
A
T7(1 (c. X7 (x, T2(1, (x,
+ X7 (2,,0)) (1,
< sup
T2(1, (x, u))
(2,
(1, (z, v) - X²(1, (xs, (2, 7 (z,
(t, (x,,)) - 2 (x, 77 (z,
In the last inequality we used that X is non-negative. Therefore we find
Σ X70,, (z, (z, (2)))00)) + + -((,))))))((( (c,
de (c, & 2
(t, (2. - (z,,,
Noticing that
(77 (z,)) 177(0,,,) - (((,,,tt) (2, P.
we find
9x (x, 1) & 2
Σ
r (x) (a,,,, (xr,
1=1
Finally, we use the inequality
1
< 2, which holds for = > 0, conclude that
TT
E
1
exp
Sr, OX,,, (X)d)
rT'
E
10 (X,: V1, T, sx((,, V2, T, +) 20, 22
V.)dt
i=1
Experimental details
We run our experiments on 50 Cascade Lake Intel Xeon 5218 16 cores, 2.4GHz. Each experiment on single CPU and takes between I and 5 hours to complete, depending the dataset and the sampler at hand.
For each forward PDMP, we take time horizon Ty equal to 5, and set the refreshment rate A, to 1. For training, we choose the uniform distribution Uniform [0, Ty]) as the time distribution w. For the simulation of backward PDMPs with splitting schemes, steps, that is (8,)ne
quadratic schedule for the time given by 8, = =T x ((*/N)² (=-
For i-DDPM, we follow the design choices introduced in Nichol and Dhariwal [2021] and in partic- ular we use the variance preserving (VP) process, the cosine noise schedule, and linear time steps.
E1 Continuation of Section 4
2D datasets In our experiments we consider the five datasets displayed in Figure 5. The Gaus- sian grid consists of a mixture of nine Gaussian distribution with imbalanced mixture weights {.01, .02, .5,, .05, .1, .15, 2, .3.. We load 100000 training samples for each dataset, and use 10000 test samples to compute the evaluation metries. We use batch size of 4096 and train our model for 25000 steps.
28

---

refresh rate
0.0
0.1
0.5
1.0
2.0
5.0
10.0
process
BPS
0.286
0.069
0.048
0.052
0,045
0,048
0.072
RHMC
0.324
0.040
0.041
0.033
0.040
0.047
0.072
ZZP
0.040
0.045
0.040
0.036
0.038
0.035
0.057
Table 4: Mean of Wasserstein W2 +. on Gaussian grid dataset, averaged over 10 runs.
time horizon
2
10
15
process
BPS
0.031
0.040
0.040
0.041
HMC
0.025
0.036
0.036
0.033
ZZP
0.031
0.033
0.030
0.046
Table 5: Mean 2- Wasserstein W2 | for different time horizon, averaged over runs.
Detailed setup For ZZP and i-DDPM we use a neural network consisting of eight time- conditioned multi-layer perceptron (MLP) blocks with skip connections, each of which consisting of two fully connected layers of width 256. The time variable passes through two fully connected layers of size 1 32 and 32 32, and is fed each time conditioned block, where it passes through an additional 32 x 64 fully connected layer before being added stement-wise to the middle layer. The model size is 6.5 million parameters. For ZZP, we apply the softplus activation function + 1/3 log(1 + exp(3z)) to the output of the network, with 3 = 1, to constrain it to be positive and stabilise behaviour for outputs close 1.
In the case of RHMC and BPS, we use neural spline flows [Durkan et al., 2019] to model the conditional densities of the forward processes, as it shows good performance among available archi- tectures. We leverage the implementation from the zuko package [Rozet et al., 2022]. We set the number of transforms to 8, the hidden depth of the network to 8 and the hidden width 256. To condition on æ, I, we feed them to three fully connected layers size d x 8, 8 x 8 and 8 x 8, where d is either the dimension of X,, are then concatenated and fed to the conditioning mechanism of zuko. The resulting model has 3.8 million parameters.
= 1 in the of the time variable. The resulting vectors
We take advantage of the approaches described in Section 2.3 to learn the characteristics of the backward processes.
All experiments are conducted using PyTorch [Paszke al., 2019]. The optimiser Adam [Kingma and Ba, 2015] with learning rate 5e-4 for all neural networks.
Additional results In Table 4 we show the accuracy in terms of the refirehment rate, while in Table we show different choices of the time horizon. In both cases, consider the Gaussian mixture data and we use the 2- 2-Wasserstein metric to characterise the quality of the generated data. Figure shows the generated data by the best model for each process.
F.2 MNIST digits
We consider the task of generating handwritten digits training the ZZP on the MNIST dataset. We use the simplified loss function given in Equation (24) in Appendix D.I and use the simplified model and loss. In Figure we show promising results obtained with the design choices described previously in Appendix F.1, apart from the differences that follow. The optimiser is Adam [Kingma and Ba, 2015] with learning rate 2e-4. We use U-Net following the implementation of Nichol and Dhariwal [2021], where the hidden layers are set to [128, 236, 256, 256], where we fix the number of residual blocks to 2 at each level, and add self-attention block resolution 16 x 16, with heads. We duplicate the channel of the penultimate layer, and make each copy go through separate MI.Ps to obtain the two vectors (. (,,),, d (:))) 2d2d (as (22)). We use an exponential moving average with a rate of 0.99. At every layer, we use the silu activation function, while we apply
29

---

Checkerboard
Z7P
RHMC
BPS
i-DDPM
Fractal tree
ZZP
RHMC
BPS
i-DDPM
Olympic rings
ZZP
RHMC
BPS
i-DDPM
Rose
77P
RHMC
BPS
i-DDPM
Gaussian mixture
ZZP
RHMC
BPS
i-DDPM
Figure 5: Generation for the various datasets.
30

---

3
7
6
7
9
2
2
4
2
8
6
D
0
4
S
9
2
6
4
7
7
2
o
4
9
7
C
8
7
Figure 6: Generation for the ZZP trained on MNIST.
the softplus to the output of the network, with 8 0.2. We train the model for 40000 steps with
batch size 128.
31

---

NeurIPS Paper Checklist
The checklist designed to encourage best practices for responsible machine learning research, ad- dressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evalu- ation. While "[Yes] is generally preferable to "[No] it is perfectly acceptable to answer "[No] provided a proper justification is given (e.g.. "error bars are not reported because would be too computationally expensive" or "we were unable to find the license for the dataset general, answering "[No] or "[NA] not grounds for rejection. While the questions are phrased in binary way, acknowledge that the true answer is often more nuanced, so please just your best judgment and write a justification to elaborate. All supporting evidence can appear either the main paper the supplemental material, provided in appendix. If you answer [Yes]] to question, in the justification please point to the section(s) where related material for the question can be found.
used"). In
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We believe the abstract and introduction reflect the contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
fine to include aspirational goals motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We believe made the limitations of our work clear.
32

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community, Reviewers penalize honesty concerning limitations.
will be specifically instructed
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: To the best of our abilities, we made sure all theoretical statements have sound proofs and precise sets assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Ys]]
Justification:
We have indicated all the design choices that we used to obtain the results of experiments.
Guidelines:
33

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibilityy which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors welcome to describe the partieular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
provide all the necessary codes to reproduce our experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https: cc/
for more details.
While we encourage the release code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips... wtt.ccccplllggguucccoee
CoodeSumieiiic ss onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
34

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification:
All the design choices are specified in the paper or in the supplementary
material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the standard deviations to our tables, or report when the results are based on a single run.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide information the type of compute workers that were used.
Guidelines:
The answer NA means that the paper does not include experiments.
35

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g.. preliminary or failed experiments that didn't tt make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics attps:///nt h/tp:r///..e./.ee//// httpprpppppppppppppppppee httpppulilc//hchshuuieeeee
Answer: [Yes]
Justification:
The research in this paper respects the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymitt (e.g., if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work does not have immediate societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
36

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite all the packages that are used in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms of use pack- age should be provided. For popular datasets, paperswithcode thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine li- cense of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
Justification:
The codes used to obtain the numerical simulations are provided and are accessible.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsoureing experiments and research with human subjects, does the pa- per include the full text instructions given participants and sereenshots, applicable, as well as details about compensation (if any)?
37

---

Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA that the paper does involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
38


Reviews:
[{"review_id": "lAYKjddlbl", "reviewer": "Reviewer_FgP1", "summary": "The paper proposes using Piecewise Deterministic Markov Processes (PDMPs) for generative modelling applications, by using the property that PDMPs also admit time reversals that themselves are PDMPs. There are three major contributions in my understanding - \n\nBy characterizing certain families of PDMPs, i.e. Zig-Zag processes (ZZP, the Bouncy Particle Sampler (BPS), and the Randomised Hamiltonian Monte Carlo (RHMC), in terms of their jump rates and kernels, this paper shows how to obtain tractable closed-form and approximations for the jump rates and kernels of the time reversed PDMPs.\nTheoretically, this paper then proposes a total variation bound between the “learnt” data distribution and the true data distribution when the base distribution is a Gaussian. This is a useful property, quite similar to bounds that have been proposed before in the literature (for example, for the Ornstein-Uhlenbeck process in [1, Theorem 5.2.]).\nFinally, the paper proposes two empirical techniques to learn the time reversals akin to score-matching. First, for the ZZP process, inspired by score-matching techniques, the authors propose a ratio-matching technique. Secondly, for the BPS and RHMC processes, the authors learn normalising flows for the time reversal. They then show promising results in low-dimensional and MNIST generative modelling applications as a proof-of-concept.\n\n[1] Dominique Bakry, Ivan Gentil, Michel Ledoux, et al. Analysis and geometry of Markov diffusion operators, volume 103. Springer, 2014.", "questions": "I’ve summarised a few of my questions in the Weaknesses section, but I list a few more questions here I would love some clarity on - \n\nYou mention that [2] originally suggested time reversal theorems for jump Markov processes. I found it really difficult to figure out what the exact distinctions are between the theoretical formulations of [2], and the specific novelty of this paper. I’d love clarifications on what the additional mathematical frameworks introduced are, that adopt this framework to PDMPs.\nI would really like a better discussion on Equation 9, talking about what the different components of the inequality correspond to, and if there are any insights to gain about why PDMPs perform better or differently to something traditional like an OU process.\nIn Line 238-239, you mention that it’s easy to subsample the loss across dimensions. Is this still an unbiased, consistent estimator for the continuous time loss?\nYou mention that PDMPs typically have several types of jump rates and kernels, and provide Equation 2 as a parametric family of possible jumps and kernels. Is this the only possibility for constructing jumps and kernels?\nAt the end of each section describing one of the 3 PDMPs, the authors mention that there has been work showing exponential convergence to invariant distributions for all the PDMPs. How does this exponential convergence compare to the OU process? Is there actually faster convergence due to the inherent irreversibility of PDMPs?", "limitations": "Yes, they have addressed any potential limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "I quite like the structure and formulation of the paper. The main goals and approach is elucidated quite clearly, and the mathematical preliminaries, while dense, seem correct for me. \n\nIn my understanding, it is a known fact that all PDMPs admit an equivalent time-reversal, but these are quite hard to calculate in general. In this paper, building on theory involving jump Markov Processes in [2], the authors derive expressions for time-reversal jumps and kernels for 3 different PDMPs. In general, deriving the backwards time reversal and then also designing an empirical scheme with a neural network architecture and loss function would be a substantial contribution, but this paper has many additional contributions on top of that. The numerical experiments seem compelling, even if a little small scale. However, this paper seems like a proof-of-concept on the use of time-reversed PDMPs for generative modelling, and I think the theoretical contributions along with the design of the loss functions and training paradigms are a pretty significant contribution already.\n\n[2] Giovanni Conforti and Christian Léonard.   Time reversal of markov processes with jumps under a finite entropy condition. Stochastic Processes and their Applications, 144:85–124, 2022.", "weaknesses": "Fundamentally, I think the paper lacks a convincing argument about why generative modelling with PDMPs would fundamentally be more useful than traditional generative modelling. I understand that there were some arguments made in the introduction of the paper, namely Lines 35-36 (“such as better scalability and reduced computational complexity in high-dimensional settings”). However, it is really unclear to me how this argument actually translates to the empirical score-matching (or normalising flow training) objectives that the authors formulate, vs an approach like DDPM. The experimental section is quite lacking in details and comparisons about how the PDMP approach improves along any number of axes, beyond the qualitative plots. For example, I can think of many axes of improvement that could be discussed - \nsample efficiency (how many training datapoints are needed to learn the time-reversal given that the process is partly deterministic), \nmixing rates towards the Gaussian for their time reversal. Usually, SDEs such as the Ornstein-Uhlenbeck process are quite quick at mixing towards a Gaussian, making them quite nice to use when reversing a Gaussian distribution as the base. For partly deterministic processes, is this easier or harder to do?\nAre there any comparisons to regular Markov process methods that can show that having an irreversible Markov process is beneficial here? I believe that this is a big factor in why PDMPs are alluring, and their irreversibility makes them mix faster and use less data [3]. Any experiments showing sample efficiency and mixing rates would be really beneficial here.\n\nI am worried that there are many subtleties in the training and sampling procedures of diffusion models, and indeed there are many papers focusing solely on the empirical training tricks that can improve generative modelling, and comparing to a vanilla DDPM model doesn’t properly ablate the technique. I would be hesitant to rely on these empirical results as a surefire sign of improved modeling, which is frustrating, as theoretically, the paper does seem to point to this being the case, and I really do want to believe. \n\nFurthermore, I think the paper could benefit from being a lot clearer about the specific advantages of PDMPs vs other stochastic processes for generative modeling. This is barely mentioned, but does form the crux of the empirical results. This made it difficult for me to read through the theoretical developments, proofs and theorems without knowing the reason why we would really want to do this in the first place.\n\nI also think the paper can also benefit from being more explicit in how their developed score matching and normalizing flow training differs from traditional methods (maybe an algorithm block), as this would be something really interesting to practitioners looking to adopt existing codebases to using PDMPs instead.\n\n[3] Bierkens, J., Fearnhead, P., and Roberts, G. (2019), “The zig-zag Process and Super-Efficient Sampling for Bayesian Analysis of Big Data,” The Annals of Statistics, 47, 1288–1320. DOI: 10.1214/18-AOS1715", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable review. We hope the following responses will answer their main remarks.\n\n* We will include a paragraph in the introduction motivating the PDMP approach, based both on theoretical and empirical aspects. We expect PDMPs to be successful at modelling data distributions that are supported on restricted domains, since it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially when compared to diffusions. Similarly, data distributions that are a mixture of a continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained taking advantage of flows which do not leave the manifold. \n* We acknowledge that the current focus of our paper has been on establishing the theoretical foundations and validating our approach with toy datasets. We underline that developing the theoretical foundations for this class of methods is non-trivial, which is our main contribution in this paper. The experiments on 2D data show faster convergence to the data distribution in terms of the number of reverse steps (Table 2 in the paper), which also translates in better computational time (see the PDF attached to the rebuttal, Figure 1). As the reviewer suggested, we will work on empirical studies to highlight the specific advantages of PDMPs over other methods, focusing in particular on sample efficiency. Regarding the mixing rates of PDMPs, improved convergence over traditional, reversible methods has been observed first in the physics literature [4], while quantitative estimates on the rate of convergence to the stationary distribution were obtained using the hypocoercivity approach in [5], hence these can be compared to existing results for diffusions for a given limiting distribution. Moreover, the recent work [6] shows that PDMPs can at most achieve a square root speed-up compared to the overdamped Langevin diffusion in terms of relaxation time, and that this can be achieved in some cases. \n* Contrary to what was erroneously written in the manuscript, we compared our model to the improved DDPM framework of [7], which already improves over the standard technique. While it is clear that this does not represent the current state of the art, we also found it to be a reasonable comparison, since our method is novel and, as the reviewer pointed out, there are many engineering tricks improving the performance of diffusion based models. \n* We agree with the reviewer that it is beneficial to give clearer explanations on the training phase of our models. We will provide a dedicated ‘Training’ section in the appendix of the revised paper, that will include algorithm blocks for each sampler and will highlight how our training procedures differ from well-established methods. We will also give more details on other aspects as for instance the simulation of the forward process.\n\n* *On the novelties compared to [8]* The work of [8] characterises the time reversals of a wide class of Markov processes with jumps. However, the conditions of [8] are abstract and stated in a “language” that differs from the standard in the context of PDMPs. Our contribution is twofold: (i) in Proposition 1 we give simple assumptions on the characteristics of the PDMP under which the process admits a time reversal, for which we give the backward characteristics; (ii) in Proposition 2 we consider the two abstract types of transition kernels that are used in the PDMP literature, namely deterministic mappings which change the velocity vector only and velocity refreshments, and give explicit formulas for the corresponding backward rates and kernels. Moreover, in Propositions 4,5, and 6 in the appendix we give clean statements for the time reversals of ZZP, BPS, RHMC, where the only assumption we require is a technical condition on the domain of the generator.  Therefore, while the abstract machinery for time reversals was developed in [8], in our paper we specialise it to PDMPs pruning the statement to the cleanest form possible. \n\n* We will provide a clearer explanation on the different terms in Equation (9) in the next version of the paper. The first term corresponds to the error caused by initialising the backward process at the limiting distribution instead of at the law of the forward process at time $T_f$. The second term is a consequence of using approximate backward rates defined with the estimated density ratios.\n\n* *On lines 238-239* If we draw uniformly at random a subset of the components $i=1,\\dots,d$, then the expectation of the empirical loss coincides with the theoretical loss $\\ell_I$ given in Proposition 3. Hence the estimator is unbiased, although its variance will increase.\n* In the classical framework of PDMPs, a PDMP is indeed constructed by specifying the rates and transition kernels for each type of jump.\n* The papers we cite at the end of the description of each PDMP on page 3 do not give quantitative upper bounds on the rate of convergence to the limiting distribution, hence they cannot be used to compare these PDMPs to e.g., the OU process. As mentioned above, there is both theoretical and empirical evidence that PDMPs can give faster convergence to a given probability distribution.\n\n[1] Bierkens, et al. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. \n\n[2] Bierkens et al. (2023). Sticky PDMP samplers for sparse and local inference problems.\n\n[4] Michel et al. (2015). Event-chain Monte Carlo for classical continuous spin models. \n\n[5] Andrieu et al. (2021). Hypocoercivity of piecewise deterministic Markov process-Monte Carlo. \n\n[6] Eberle et al. (2024). Non-reversible lifts of reversible diffusion processes and relaxation times. \n\n[7] Nichol et al. (2021). Improved Denoising Diffusion Probabilistic Models.\n\n[8] Conforti et al. (2022). Time reversal of Markov processes with jumps under a finite entropy condition.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5p2pEepoFP", "reviewer": "Reviewer_LXdU", "summary": "This paper considers the development of generative models based on piecewise deterministic Markov processes. The key idea proposed in the paper is to use piecewise deterministic Markov processes instead of diffusions as the \"noising process\" of the generative model. This relies on the fact that time reversals of PDMPs are themselves PDMPs. Three specific instances of PDMPs are considered. The authors also derive a bound (in total variation distance) between the data distribution and the distribution of the generative model. The methodology is illustrated with some simple experiments.", "questions": "Are there particular data characteristics that these processes would be most suitable for? It seems that images with sharp boundary transitions are better modeled by the authors proposed approach. Would you say that is the case?\n\nDo these methods have issues like being only able to generate from one mode if the data distribution is multimodal?\n\nWhy does RHMC do so well for such a small number of steps?", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I enjoyed reading the paper and I like the idea of considering alternative noising processes in the context of generative models. The paper covers both theory and provides an example showing the viability of these methods. The examples are sufficient and certainly the area seems worthy of further investigation.", "weaknesses": "To me, the descriptions of approximating the process characteristics with normalizing flows are unclear. This part should be written more with more details, perhaps in the supplement, as this is core to being able to reproduce the results. I do appreciate that the authors provided a description of the experiment in E.1, but it is not enough to put things together. I would be interested in replicating at least the simple experiment, but I don't think I can do it as the paper stands.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time invested in our paper and the relevant questions. Below, we believe we address all the raised concerns.\n\n* *To me, the descriptions of approximating the process characteristics with normalizing flows are unclear. ...*\n\n    We understand that we did not provide sufficient details on our methodology to obtain approximations of the characteristics of BPS and RHMC via normalising flows. We will clarify our implementation further both in the main text and in the supplement of the updated version of the paper, including pseudo-algorithms and step by step guides to reproduce the experiments.\n\n    * Let us give a brief overview of the methodology here. Our procedure to approximate the characteristics of the backward BPS and RHMC is built on minimising an empirical counterpart of the (maximum likelihood based) loss displayed between lines 252 and 253. Such empirical loss is obtained with the standard approach of using Monte Carlo estimates for the inner expectation as well as the integral with respect to the probability distribution on the time variable. Concerning the time variable, we draw a time $\\tau$ independently from $\\omega$ and use it as the time horizon for the forward PDMP, obtaining $(X_\\tau,V_\\tau)$. \n\n    * Then, the normalising flows framework is used to model the conditional density that appears in the loss. Essentially, we define a normalising flow that takes as input a $d$-dimensional standard normal random vector and outputs another $d$-dimensional random vector. The architecture of the normalising flow is described in Appendix E.1. Then, the normalising flow must be conditioned on the position and time variable since it models a conditional density. We achieve this by embedding the position and time via MLPs (with size given in Appendix E.1) and then injecting the outputs in the NF architecture at different points. The output of the NF is then a random vector which is approximately distributed according to the conditional distribution of v given x and t. The obtained conditional NF defines an invertible deterministic mapping $ v \\mapsto T_{(x,t)}(v)$ and hence gives an explicit expression for the modelled conditional density by the change of variables formula. Therefore, this can be used to compute the empirical loss.\n\n* *Are there particular data characteristics that these processes would be most suitable for? It seems that images with sharp boundary transitions are better modeled by the authors proposed approach. Would you say that is the case?*\n\n    This is a very important question, which will be best answered through extensive experimentation. As the reviewer noticed, our numerical simulations indeed suggest that sharp boundary transitions are modelled well by our algorithms compared to DDPM. We also expect PDMPs to be successful at modelling data distributions that are supported on restricted domains, since it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially if compared to diffusion processes. Similarly, data distributions that are a mixture of continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained leveraging flows which do not leave the manifold (see e.g. [3] for a PDMP on a sphere).\n\n* *Do these methods have issues like being only able to generate from one mode if the data distribution is multimodal?*\n\n     While the non-reversibility of PDMPs can lead to improved convergence, it is not yet clear whether this is beneficial in the context of multimodal distributions. Nonetheless, it is worth mentioning our 2D experiment with the Gaussian grid, which was designed to test the robustness of the different generative methods with respect to mode-collapse. The Gaussian grid at hand is an unbalanced multimodal dataset (see line 686), and we can see that the PDMP samplers all have better performance than DDPM with respect to the MMD metric. One can visually check on Figure 2 that the data coverage of RHMC compares favourably to DDPM. This suggests that PDMPs can generate from multimodal distributions.\n\n* *Why does RHMC do so well for such a small number of steps?*\n\n    This is an interesting and perhaps surprising behaviour. Our intuitive explanation is that velocity refreshments can guide the position vector of the process towards the data distribution quite efficiently assuming the last step is performed close enough to the time horizon and assuming the conditional distribution of the velocity vector given the position vector is learned accurately enough. This intuition is supported by the fact that this phenomenon is not observed for the Zig-Zag process when the refreshment rate is set to 0, in which case the model requires a larger number of reverse steps to give good performance. We will add a comment about this in the paper.\n\n[1] Bierkens, J., Bouchard-Côté, A., Doucet, A., Duncan, A. B., Fearnhead, P., Lienart, T., Roberts, G., & Vollmer, S. J. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. Statistics & Probability Letters, 136, 148-154. \n\n[2] Bierkens, J., Grazzi, S., van der Meulen, F., & Schauer, M. (2023). Sticky PDMP samplers for sparse and local inference problems. Statistics and Computing, 33, 8. \n\n[3] Yang, J., Łatuszyński, K., & Roberts, G. O. (2024). Stereographic Markov Chain Monte Carlo. arXiv. https://arxiv.org/abs/2205.12112", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0EbGfeMLlV", "reviewer": "Reviewer_RL5x", "summary": "This interesting paper on the popular topic of generative models introduce a new family of generative models which builds on the so-called piecewise deterministic Markov process (Zig-Zag process, Bouncy Particle Sampler, Randomised Hamiltonian Monte Carlo). In contrast to many of the existing models this family is not based on diffusion models. The paper includes a through analysis of the construction \nand it propose training procedures and methods for approximate simulation of the reverse process.", "questions": "* You provide convincing examples on simple synthetic datasets. Is you family of methods able to generate data also in more challenging real-world examples? \n* When you compare the results of the various methods you show results for \"steps\". Is the a fair way of comparing the methods? Would it not be better to have some kind of computational cost on the x-axis? My reasoning here is that a \"step\" is not a well defined concept and it could mean very different things to different methods. In any case it would be really interesting to see the performance with the computational cost on the x-axis.", "limitations": "Real-world examples missing.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "* A new family of generative models is proposed. \n* Thorough analysis of the properties of the proposed construction is provided.\n* Simple examples provided.", "weaknesses": "* Missing real-world examples\n* The is a big jump in  the style of writing between Section 1 and 2. Do not get me wrong here, the technical developments are most interesting, but many readers would be helped by a more gentle transition between these sections. Space for this can be created by moving more of the technical details into the supplemental material.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time invested in our paper and their feedback. We believe we addressed all the raised issues below.\n\n* *There is a big jump in the style of writing between Section 1 and 2...*\n    * We agree the description of a time inhomogeneous PDMP is technical. In order to alleviate this difference in style between the two sections, in the next version of the paper we will add at the beginning of Section 2 the trace plots that can be found in the pdf attached to the rebuttal (Figure 2). These plots show the dynamics of the three PDMPs we discuss in the paper and hence help the reader develop a clear idea of the noising processes and their time reversal. \n    * In addition, one option that we are considering is to first present our methodology for the specific case of RHMC, discussing the forward process and its time reversal, before introducing PDMPs in full generality. RHMC is perhaps the simplest of the three processes and the characteristics of its time-reversal are relatively straightforward to present and can be intuitively interpreted. We could then briefly outline the main ideas of the training procedures to estimate these characteristics, leaving the most technical details to a new appendix. If the reviewer thinks this is a good idea, we would gladly make this addition to the revised version of our paper.\n    * Finally, in the supplement, we will add a section dedicated to the simulation of the forward PDMPs, with special emphasis to the case in which the limiting distribution is a multivariate standard normal. We think these additions will give the reader a more gentle, visual introduction to PDMPs and their time-reversals.\n\n* *You provide convincing examples on simple synthetic datasets. Is you family of methods able to generate data also in more challenging real-world examples?*\n\n    We are happy the reviewer found the experiments on synthetic datasets convincing.  The numerical experiments, albeit on simple datasets, indeed suggest that it is worth conducting more extensive studies and comparisons on real-world datasets. For the moment, this paper laid the theoretical foundations to this new class of methods. Obtaining further results on more complex datasets is an important future direction that we wish to work on.\n\n* *When you compare the results of the various methods you show results for \"steps\". Is this a fair way of comparing the methods? ...*\n\n    We agree with the reviewer, and have added to the pdf attached to the rebuttal a plot (see Figure 1) which reports the computational time on the x-axis and the performance on the y-axis, for the “rose” dataset. This plot clarifies the computational cost of our algorithms as compared to DDPM, and we indeed observe clear improvements. We plan to make the same type of comparisons on the other datasets that we consider and add the resulting plots to the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y3Cu594V4g", "reviewer": "Reviewer_a5JX", "summary": "The paper explores generative models utilizing PDMPs, a type of stochastic process characterized by deterministic motion interspersed with random jumps at random times. These models offer an alternative approach to diffusion-based generative models, which have become very popular in the AI community in recent years. The authors focus on three specific PDMPs: the Zig-Zag process (ZZP), the Bouncy Particle Sampler (BPS), and Randomised Hamiltonian Monte Carlo (RHMC). The authors leverage the existing literature on the time reversals of Markov jump processes, characterizing the time reversal of any PDMP under appropriate conditions. They show that the time reversal of a PDMP is itself a PDMP with characteristics related to the original process. The authors also specifically outline the characteristics of the time-reversal processes for ZZP, BPS, and RHMC. The jump rates and kernels of these time reversals admit explicit expressions based on the conditional densities of the PDMPs before and after jumps.\n\nThe authors provide bounds on the total variation distance between the data distribution and the distribution of their generative models, considering errors from the approximation of the backward PDMP’s characteristics and its initialization from the forward process’s limiting distribution. Some initial but promising numerical simulations on simple toy distributions are presented, showcasing the potential of PDMP-based generative models. The results support further investigation into this class of models on more challenging data structures.", "questions": "- I've outlined quite a few points in the weaknesses section that the authors may wish to address as questions.\n- Could the authors include computational time with their simulations?\n- Under H2, the authors state that the condition is satisfied when \"the tails of $\\mu_*$ are sufficiently light.\" Could this be clarified as to what \"sufficiently light\" means? I'm interpreting this as lighter than Gaussian. \n- A small query, in Proposition 2, on Line 166, it states that $\\mu_0^X \\otimes \\mu_0^V$ on $\\mathbb{R}^2d$. But how does this work for ZZP when $V=[-1,+ 1]^d$ (sorry - I can't get the curly brackets to work)?", "limitations": "NA", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Originality: The paper introduces a class of generative models based on piecewise deterministic Markov processes (PDMPs). This is a very novel idea which departs from the widely used diffusion-based models. In my opinion, this is a fresh perspective on generative modelling and opens up new avenues for exploration and potential improvements in various application areas. The key idea behind diffusion-based models is the derivation of the reverse time and process, and following a similiar line of thinking, the authors of this paper characterize the time reversal of PDMPs, which is particularly original. While time reversal in diffusion processes is well-studied, applying these concepts to PDMPs and providing explicit expressions for their jump rates and kernels is a significant contribution.\n\nQuality: The paper presents a comprehensive theoretical framework for PDMP-based generative models. The mathematical rigour in characterizing time reversals, deriving error bounds, and proposing training procedures leads to a complete and high-quality piece of work. The thorough analysis of the three specific PDMPs (ZZP, BPS, RHMC) and the detailed exploration of their time-reversal characteristics demonstrate a deep understanding and careful consideration of the underlying processes. The numerical simulations provide supporting empirical validation of the proposed models, adding credibility to the theoretical claims. The use of multiple toy distributions (extras are in the appendix) to test the models in the paper strengthens the evidence for their potential efficacy in practice.\n\nClarity: The paper is very well-structured, with a logical flow from the introduction of PDMPs to the detailed theoretical contributions and empirical results. Each section builds on the previous one, making it easier to follow the progression of ideas. The explanations of the PDMPs, their time reversals, and the derivation of jump rates and kernels are clear and very detailed. The inclusion of propositions and their proofs in the appendices provides a robust foundation for the proposed algorithms and establishes their convergence properties. The use of toy distributions to demonstrate the numerical simulations helps illustrate the practical application of the models, aiding in the reader’s understanding and also provides some nice opportunities for the authors to clearly articulate the advantages of their approach over the diffusion-based alternative approach.\n\nSignificance: The proposed PDMP-based generative models have the potential to be applied in a wide range of fields, from machine learning and statistics to physics and biology. This broad applicability enhances the significance of the work. By offering an alternative to diffusion-based models, this paper advances the field of generative modelling. The potential advantages of PDMPs, such as better scalability and reduced computational complexity, could lead to significant improvements in high-dimensional data generation. The theoretical insights and empirical results lay a strong foundation for future research in this area.", "weaknesses": "Originality: While the introduction of PDMP-based generative models is novel, the paper primarily focuses on theoretical aspects and toy datasets. There is a limited exploration of how these models can be applied to more complex, real-world problems, which could showcase their true originality and practical utility. Although PDMPs offer a new approach, the paper does not extensively compare these models with a wide variety of existing generative models, even on the simple toy examples that were considered here. This makes it difficult to fully appreciate the originality and benefits of PDMPs over other state-of-the-art methods.\n\nQuality: The paper relies on several technical assumptions and conditions (e.g., H3, H4). While these are necessary for the theoretical results, their practical applicability might be limited and the authors do not discuss whether or not these assumptions hold for the toy examples which they consider. The paper could be strengthened by discussing the feasibility of these assumptions in real-world scenarios. The empirical validation is primarily limited to simple toy distributions. While these are useful for initial validation, the lack of experiments on more complex datasets (e.g., image or text data) reduces the overall impact and persuasiveness of the empirical results. It would be beneficial to include comparisons on more challenging benchmarks.\n\nClarity: The paper uses dense mathematical notation and detailed proofs, which may be challenging for readers who are not specialists in stochastic processes or PDMPs. This is going to be challenging for the authors as the high-level of technical detail provided in the paper does lead to a very robust paper. However, perhaps more intuitive explanations or visual aids could help make the content more accessible, if not in the main paper then in the appendix. The practical implementation details, particularly regarding the training procedures and simulation methods, are somewhat sparse (even though there are more details in the appendix). Providing a step-by-step guide or pseudocode could help practitioners better understand how to apply the proposed methods. Given the space constraints, this would have to be added to the appendix. This is covered in the case of splitting schemes, but could perhaps be modified to be more user-friendly to people new to this area of research. \n\nSignificance: The paper’s significance is somewhat limited by the focus on theoretical and synthetic examples. Without demonstrating the effectiveness of PDMP-based models on real-world data, it is challenging to gauge their practical significance and potential impact in applied settings. While the paper claims that PDMPs offer better scalability and reduced computational complexity, there is limited empirical evidence to support these claims. Benchmarking the computational performance against existing generative models would provide a clearer picture of the advantages and limitations in terms of scalability and efficiency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and for taking the time to read and appreciate the full extent of our paper. We value your detailed feedback and have addressed your concerns below.\n* *Weaknesses (originality, significance)*\n\t* Our paper focuses on establishing theoretical foundations and validating our approach with toy datasets. We highlight that our main contribution in this paper was proposing this novel methodology together with the theory for this class of methods. We believe this was a significant contribution, as deemed by the reviewer. \n\t* In the numerical experiments, PDMPs show promising results, with faster convergence to the data distribution in terms of computational time (see the pdf attached to this rebuttal, Figure 1). We agree that demonstrating their performance on more complex, real-world problems would showcase practical utility more effectively, and this is an important future direction we wish to work on. \n\t* We expect PDMPs to be successful at modelling data distributions supported on restricted domains, as it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially compared to diffusion processes. Similarly, data distributions that are a mixture of continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained leveraging flows which do not leave the manifold (see e.g. [3] for a PDMP on the sphere).\n\t* As the reviewer suggests, in the next revision we will work on comparisons with other generative methods (e.g., variational autoencoders and normalising flows) on the datasets we consider.\n* *Weaknesses (quality)* and *Questions: Under H2, the authors state that the condition...*\n\n\tWe understand the remarks asking for more clarity regarding the assumptions that we require, and we agree that it is important to discuss whether these hold in practice. We will add these clarifications to the revised paper. We stress that H2 is the only assumption that depends on the data distribution. Let us comment on them.\n\t* H1 is about the chosen stationary distribution of the forward PDMP and is satisfied when this distribution is the multivariate standard normal, but also in the “variance exploding” variant, that is when $\\psi(x) = 0$ for all $x$.\n\t* H2 is only required to prove Theorem 1 and is discussed in Appendix D.1. H2 holds when the stationary distribution of the PDMP is Gaussian and the data distribution has tails at least as light as those of a Gaussian in the case of ZZP and BPS, or in the case of HMC when it admits finite moments up to order 2. H2 is then satisfied in all our toy examples and in general when the data distribution is supported on a bounded set, as e.g. when the dataset is composed of images. We will give more information on this aspect in Appendix D.1 of the revised paper. We remark that in the case of ZZP and BPS much weaker conditions on tails of the data distribution can certainly be obtained by developing theory which is tailored for the context of generative modelling. \n\t* H3 is verified for the three PDMPs we consider, as mentioned in Appendix A.2. We will add comments on it in the appendix.\n\t* H4 is very technical, but is about the dynamics of the forward process rather than about the data distribution. It was verified in previous work for a specific version of BPS and can be verified for ZZP and RHMC after some technical arguments. Since this is out of the scope of our submission, we leave this verification for future works dedicated to this technical question.\n* *Clarity* \nWe thank the reviewer for having found our paper theoretically solid. We agree that more intuitive and visual explanations could make the content more accessible. In order to improve the clarity of our paper, we will make the following changes. \n\t* Include a sample trajectory for the forward process of each sampler; you can find them in Figure 2 of the PDF attached to this rebuttal. \n\t* Add a dedicated ‘Training’ section in the appendix, which will be self-contained and include a step-by-step guide for each of the three PDMP considered. This will clarify the overall training procedures and make them easy to implement. \n\t* Another option we are considering is to start Section 2 with a description of RHMC and its time reversal, using this simple process to illustrate the main ideas and intuition in a more gentle and visual way. We could then outline the main ideas of the training procedures for the characteristics of RHMC, leaving the most technical details to a new appendix we plan to add. If the reviewer thinks this is a good idea, we would gladly make this addition to the revised version of our paper.\n* *Could the authors include computational time with their simulations?* \nAs we previously mentioned, we include in the attached pdf a plot (Figure 1) displaying performance for each generative method on the Rose dataset, as a function of computational time. In addition, we plan to make the same type of comparisons on the other datasets that we consider, and add the resulting plots to the paper.\n\n* *A small query, in Proposition 2...*\nWe consider that the uniform distribution on $\\{ \\pm 1 \\}^d$ is also a distribution on $\\mathbb{R}^d$, hence the proposition applies. Based on your comment we will add a clarifying remark.\n\n[1] Bierkens, J., Bouchard-Côté, A., Doucet, A., Duncan, A. B., Fearnhead, P., Lienart, T., Roberts, G., & Vollmer, S. J. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. Statistics & Probability Letters, 136, 148-154. \n\n[2] Bierkens, J., Grazzi, S., van der Meulen, F., & Schauer, M. (2023). Sticky PDMP samplers for sparse and local inference problems. Statistics and Computing, 33, 8. \n\n[3] Yang, J., Łatuszyński, K., & Roberts, G. O. (2024). Stereographic Markov Chain Monte Carlo. arXiv:2205.12112", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t5qbsjThh4", "reviewer": "Reviewer_jjQr", "summary": "This paper proposes a new generative model on continuous random variables similar to diffusion-based generative models, but this method uses a different family of perturbation schemes. In particular, for the forward processes, which transform the clean data to a stationary distribution, the authors propose using piecewise deterministic Markov processes (PDMPs), which consist of a combination of discrete diffusions and ODEs. Specifically, the initial value, i.e., data, is transformed via simple ODEs, whose velocity term is often chosen to be a constant. However, the velocity term (also called vector fields) is updated at any random time with a constant rate $\\lambda$. The occurrence of a new event is independent, and the time interval of the new event since the last one is exponentially distributed. Moreover, the new velocity will be assigned by user-chosen schemes; for example, one can flip the sign of the velocity, or another one can sample from the standard normal distribution. Thus, while the forward process transforms the data with a simple ODE, it randomly changes the ODEs, hence, the title of the paper.\n\nIn addition, the paper shows that under some mild assumptions, the time-reversed processes of the PDMPs are also PDMPs, and their transition probability distribution and update rate can be written in terms of the forward processes. Therefore, we can consider modeling such terms with parametric models similar to popular diffusion-based generative models.\nThe paper proposes the explicit ratio matching objective function, as in Equation 6, to train such terms.\n\nMoreover, the authors suggest using three popular PDMPs for the forward process: Zig-Zag process (ZZP), bouncy particle sampler (BPS), and randomized Hamiltonian Monte Carlo (RHMC). Consequently, the paper shows the time-reversed PDMPs and potential parameterizations that fit each case.\n\nFinally, the paper demonstrates the efficacy of the proposed method through a few toy experiments.\n\n----------------------------------\nUpdate the rating from 5 to 7 after the authors' rebuttal.", "questions": "N/A", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "One of the paper's key contributions is the introduction of a novel generative model that uses PDMPs as the forward processes. Moreover, the authors propose a training method for the time-reversed PDMPs.", "weaknesses": "The proposed method is novel and interesting. However, the paper didn't characterize some potential drawbacks well. For example, due to the nature of PDMPs, the training of the proposed method inherits the problem of discrete diffusion models. In particular, the ratio matching should be done for each dimension independently, and this costs a lot for high-dimensional data compared to the denoising score matching and others for previous methods. This problem could be the reason that the current submission didn't include the experiments on popular high-dimensional datasets. In this regard, the paper suggests that Monte Carlo estimates can reduce the computational cost, but it trades off the variance of the learning signals, which would be critical for large-scale experiments.\n\nI consider that a new method doesn't always need to be better than previous methods. Nevertheless, proper information about the proposed method in practice would be important for potential readers to evaluate the significance of the paper. Thus, some additional discussions related to this problem need to be added.\n\nMoreover, given that many alternative choices exist other than the proposed method, in-depth discussions on the proposed method's merits would be appreciated for evaluating its importance.\n\nIn addition, I find that the presentation should be improved. For example, many statements contain multiple prepositional phrases, which are difficult to parse and understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and the time you have taken to provide valuable feedback on our paper. Below are our responses to your comments.\n\n* *Regarding the first two points mentioned in the Weaknesses*\n\n    We agree with the remarks and we will add clearer discussions on the limitations of our models in the next version of the paper. As the reviewer observed, the computational cost of the ratio matching loss in the case of ZZP grows linearly in d, which indeed makes the algorithm costly for higher dimensional datasets. As pointed out by the reviewer, subsampling over dimensions as suggested in lines 238-239 increases the variance of the unbiased estimator of the loss function. On the other hand, notice that the cost of the loss function associated with BPS and RHMC does not increase with the dimension, since the velocity vectors lie in a continuous space and the whole vector is updated at the same time, instead of flipping each component separately. Nevertheless, it is challenging to tune conditional normalising flows when the dimension increases, hence this constitutes the main limitation for BPS and RHMC.\n\n* *Moreover, given that many alternative choices exist other than the proposed method, in-depth discussions on the proposed method's merits would be appreciated for evaluating its importance.*\n\n    We agree with the reviewer that it is important to motivate our novel generative approach and we will add a paragraph to the introduction to improve this aspect. In general, we expect PDMPs to be successful at modelling data distributions that are supported on restricted domains, since it is very simple to adapt their deterministic dynamics to incorporate boundary terms (see e.g. [1]), especially if compared to diffusion processes. Similarly, data distributions that are a mixture of continuous density with point masses can be modelled adapting the ideas in [2], while extensions to data on Riemannian manifolds can be obtained leveraging flows which do not leave the manifold (see e.g. [3] for a PDMP on a sphere). \n\n    We also believe that further empirical studies will be beneficial to highlight the specific advantages of PDMPs over other methods. In the revised paper we plan to include comparisons with other generative models on the synthetic datasets we considered.\n\n* *In addition, I find that the presentation should be improved. For example, many statements contain multiple prepositional phrases, which are difficult to parse and understand.*\n\n    Concerning this remark by the reviewer, we interpret that it refers to sentences such as those in lines 136-138, or 316-318. We will rewrite these and similar sentences to make the presentation more streamlined and easy to read. In addition, we will modify the paper extensively in order to improve the general presentation, taking into consideration the feedback that we received by all reviewers. For example, we will improve the exposition of Section 2 by introducing intuitive explanations and visual aids such as trace plots of the forward processes. We will also consider starting Section 2 with a description of RHMC and its time reversal, using this simple process to illustrate the main ideas to the reader in a more gentle way before discussing PDMPs in full generality. We will work on improving other sections, for instance including better discussions of our assumptions and additional material on the implementation of our algorithms, emphasising the main differences with diffusion models.\n\n\n\n[1] Bierkens, J., Bouchard-Côté, A., Doucet, A., Duncan, A. B., Fearnhead, P., Lienart, T., Roberts, G., & Vollmer, S. J. (2018). Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains. Statistics & Probability Letters, 136, 148-154. \n\n[2] Bierkens, J., Grazzi, S., van der Meulen, F., & Schauer, M. (2023). Sticky PDMP samplers for sparse and local inference problems. Statistics and Computing, 33, 8. \n\n[3] Yang, J., Łatuszyński, K., & Roberts, G. O. (2024). Stereographic Markov Chain Monte Carlo. arXiv. https://arxiv.org/abs/2205.12112", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their constructive feedback, which will certainly help us improving the quality and exposition of our contribution. We have responded individually to each review to address their specific questions and remarks.

As part of our answer to some of the reviewers, we have attached a PDF document including two figures:
* *(Figure 1)* We provide a figure displaying performance on the Rose dataset as a function of *computational time*, for each generative method. We re-adapt the results listed in Table 2, where performance is given as a function of reverse steps. The figure illustrates that PDMPs improve over DDPM when the computational time is considered. We plan to make the same analysis on the other datasets that we consider and add the resulting plots to the next revision of the paper. 
* *(Figure 2)* We provide sample trajectories for each PDMPs considered in order to visually illustrate their respective dynamics. These plots are part of our efforts to make the presentation in Section 2 more accessible for the reader, taking advantage of visual aids to introduce this class of generative models.


Please let us know if we have adequately addressed your comments and questions. We remain at your disposal for further clarifications. Thank you again for your engagement.

Meta-review: The reviewers, some of whom are experts in topics intimately related to PDMPs such as MCMC methods, and others of whom are experts in generative modeling, unanimously recommend acceptance. I concur.
Decision: Accept (poster)