Hamiltonian Score Matching and Generative Flows
Authors: Peter Holderrieth
Yilun Xu
Tommi Jaakkola
Venue: NeurIPS 2024 poster (2024)
Keywords: Hamiltonian dynamics
score matching
generative models
diffusion models
flow matching
Hamiltonian Monte Carlo
Primary area: generative_models
TL;DR: Novel score matching and flow-based generative models are introduced by learning velocity predictors of Hamiltonian dynamics.

Abstract: Classical Hamiltonian mechanics has been widely used in machine learning in the form of Hamiltonian Monte Carlo for applications with predetermined force fields. In this paper, we explore the potential of deliberately designing force fields for Hamiltonian systems, introducing Hamiltonian velocity predictors (HVPs) as a core tool for constructing energy-based and generative models. We present two innovations: Hamiltonian Score Matching (HSM), which utilizes score functions to augment data by simulating Hamiltonian trajectories, and Hamiltonian Generative Flows (HGFs), a novel generative model that encompasses diffusion models and OT-flow matching as HGFs with zero force fields. We showcase the extended design space of force fields by introducing Oscillation HGFs, a generative model inspired by harmonic oscillators. Our experiments demonstrate that HSM and HGFs rival leading score-matching and generative modeling techniques. Overall, our work systematically elucidates the synergy between Hamiltonian dynamics, force fields, and generative models, thereby opening new avenues for applications of machine learning in physical sciences and dynamical systems.

Hamiltonian Score Matching and Generative Flows
Peter Holderrieth
Yilun Xu
MIT CSAIL phold@mit edu
NVIDIA
yilunx@nvidia.com com
Tommi Jaakkola
MIT CSAIL
tommi @csail.maaiii..... edu
Abstract
Classical Hamiltonian mechanics has been widely used in machine learning in the form of Hamiltonian Monte Carlo for applications with predetermined force fields. this work, we explore the potential of deliberately designing force fields for Hamiltonian ODEs, introducing Hamiltonian velocity predictors (HVPs) as tool for score matching and generative models. We present two innovations constructed with HVPs: Hamiltonian Score Matching (HSM), which estimates score functions by augmenting data via Hamiltonian trajectories, and Hamiltonian Generative Flows (HGFs), novel generative model that encompasses diffusion models and flow matching as HGFs with zero force fields. We showcase the extended design space of force fields by introducing Oscillation HGFs, generative model inspired by harmonic oscillators. Our experiments validate our theoretical insights about HSM as a novel score matching metric and demonstrate that HGFs rival leading generative modeling techniques.
1 Introduction
Hamiltonian mechanics a cornerstone of classical physics, providing a powerful framework for analyzing the dynamics of physical systems [33, 4]. The Hamiltonian formalism has been widely applied in machine learning and Bayesian statisties via Hamiltonian Monte Carlo (HMC) [14, 36, 8]. In this setting, the goal is to sample from a probability distribution π whose density x(x) is known up to normalization factor. In HMC, one interprets V log π (xx) force function and plugs it into Hamiltonian ODE construct fast-mixing Markov chain exploring the data space quickly. This makes HMC one of the most powerful sampling algorithms to date [36, 23].
In generative modeling, the density Tzx) is unknown, only data samples IT, .i,..., En ~ π are given, and the goal is to learn to generate novel samples from π. Current state-of- fittthh-rtt models are based
on diffusion [41, 42, 45, 21] and enjoy widespread success image generation [39], molecular generation [1]]. and robotics [10]. Diffusion models learn the score function V log Fo (x) for range of noise scales via denoising score matching (DSM) [48]. This enables one to subsequently generate high-quality samples by following a stochastic differential equation [45].
In light of the success of HMC, it is natural to ask whether the Hamiltonian formalism can also improve generative models or provide novel insights into their construction. Previous works have exploited the connection between Hamiltonian physics and generative modeling for specific force fields [13]. However, these works usually consider particular (fixed) force fields and stay within the diffusion framework.
More recently, flow-based generative models such as flow matching have enabled scalable training of continuous normalizing flows (CNFs) [31, 32]. These ODE-hased models allow craft first-order ODEs transforming arbitrary distributions from one to another. Inspired by these successes, we consider the Hamiltonian ODE Neural ODE [7]. We show that marginalize out velocity
38th Conference Neural Information Processing Systems (NeurIPS 2024).

---

distributions and then follow backward ODEs that faithfully recover data distributions. Importantly. provide associated theorem that holds for any force field. With the growing success of generative models in physical sciences [11, 49, 11. is striking that most approaches do not use existing known force fields sometimes leading to physically implausible results [1]. A framework that allows to reason natively about force fields in the context of generative models would be very promising for such applications [2, 12]. This work aims to build towards such a deeper integration.
We explore the intricate relationship between Hamiltonian dynamics, force fields, and generative models. Specifically, we make the following contributions:
1. Hamiltonian velocity predictor: We introduce the concept of Hamiltonian Velocity Predictor (HVP) and show the utility of HVPs for score matching and generative modeling (Section 3).
2. Hamiltonian score discrepancy (HSD): We introduce and validate Hamiltonian score discrepancy (HSD), a novel score matching metric based on HVPs and a corresponding score matching method (Section 4).
3. Hamiltonian generative flows (HGFs): We show that the location marginal of a Hamilto- nian ODE is generated via the Hamiltonian Velocity Predictor (Section 5). This leads to novel generative model generalizing diffusion models and flow matching (Section 6).
4. Oscillation HGF': As special HGFs, we study Oscillation HGFs, simple generative model rivaling the performance of diffusion models due to in-built scale-invariance (Section 7).
2 Background
2.1 Hamiltonian Dynamics
The inspiration of using Hamiltonian dynamics machine learning comes from considering data point x E Rd coordinates of object in RR.. Such an object also has velocity u E Rd Let be probability distribution with density function æ : IRd - Rza- Assuming unit mass, the energy of such an object is given by its Hamiltonian [33, 4], defined by:
H(x, v) = (x) + ||/l/
(I)
where U : Rdd >> IR is a potential function. The key idea behind using Hamiltonian dynamics in the context of probabilistic modeling and sampling is to set the potential function as negative log-likelihood, i.e., (xx = - log x(x). One defines the Boltzmann- Gibbs distribution TBG then as:
BB ग 6 (a) TB v) = exp(- (x, =x)) 0, La),
Le. the product distribution of the data distribution and normal distribution. In particular, can easily draw sample z from Ba by sampling x ~ т,, N/ Ia) and setting (z, v).
Hamiltonian dynamics describe how an object described by z (xo, vo) evolves over time. defined by the ODE [4]
di 10, d u()) (v(),-VU(<(())) = (o((), V log (o))()()))), di (z(0), v(0)
(2)
ie. the change of location is the velocity and the change of velocity is the force (here, equals acceleration as we assume unit mass). Let y g2d R + Rdd (z, + (2)(2) be the corresponding flow, ie. the function + (z) is a solution to the above ODE with starting point z.
As one would expect from physics, Hamiltonian dynamics 9r preserve the energy of a system, ie.. H (2, v)) = Hf (z, v) for all f, x, (see proof in Appendix B. This physical intuition translates into the fact that Hamiltonian dynamics preserve the Boltzmann Gibbs distribution, i.e.
~ ~~C = (Z) ~ TBG for all 2 0
We include derivation of this well-known statement in Appendix C.I as it is so central to this work.

---

2.2 Score Matching
The goal of score matching is to learn the score function log n from data samples #1,-.. In ~ и. As the score function naturally appears in the Hamiltonian ODE (see Equation (2)), we interpret it as a force function and denote a parameterized score model by F6 R + R. A natural approach to fitting Fo is to minimize the mean squared error between F6 and the true score weighted by their likelihood under æ.. This leads the explicit score matching loss [25] given by
Lesa (8; π²) Ezaz
Fo(z))1
This loss cannot be minimized directly as one does not have access to V log π and various score- matching methods differ in how they circumvent not having access V log (see Appendix A for detailed overview).
different approach to score matching is to slightly modify the objective by adding Gaussian noise
to the data distribution to get the noisy distribution To (x:) = f 2x; o² (a)) [48]. The objective can then be expressed as denoising score matching:
Tutss (0; na) Bewnr [1Fo(( + ac)
[ = Lesm
Lesm (0: To) + C.
(6)
for a constant C.. Noised data distributions naturally appear in diffusion models, and the denoising score-matching objective. therefore, became the ofae-oof-theart method to train diffusion models. However, denoising score matching suffers from high variance leading to long training times as well computationally expensive sampling [44, 52]. In addition, it would be unreasonable choice for an application where it is important to learn the original data distribution.
3 Hamiltonian Velocity Predictors
The Hamiltonian ODE has been widely used in Bayesian statisties in the form Hamiltonian Monte Carlo [14, 36]. However, in such settings, is assumed that one has access to the score V log π (=force), and the goal is to sample from (by sampling from TBG). In machine learning tasks, the inverse true. For such tasks, one has access data samples æ1, "i и and the goal is (I) leam V log π (score matching) or (2) create new samples I ~ æ (generative modeling) or both.
Parameterized Hamiltonian ODEs (PH-ODEs). This inspires the definition of parameterized Hamiltonian ODE (PH-ODE). A PH-ODEs consists of two components:
1. Initial distribution II: The starting condition (z, v) II is distributed according to joint location-velocity distribution II such that its marginal over x x:
f [[2 v)du T(x)
In the default cause, IT equals the Boltzmann- -Gibbs distribution Ta. i.e. II #8,(00, Ia).
2. Force field: The evolution is governed by a parameterized force field Fg : RRd - Rd via:
r(), d (()) = (v(1), Fo(a(t), dt (z()), v(0) = (Do, vo) =
d
(8)
(9)
As we consider II part of the definition of a PH-ODE, we write with a slight abuse of notation
1 (f()) = (z) (=) (af, x()
(10)
for the solution of the ODE assuming 2 (xp, vo) II. We note that while Fe might refer 0-parameterized neural network, we can also set to a fixed known vector field. Both cases will be important.
3

---

Velocity prediction is all you need. The crucial idea of this work is that one can use PH-ODEs for both score matching and generative modeling by predicting velocities. For this, family of functions Va R2d IR IR, here usually neural network. Let us consider the following
auxiliary
velocity prediction loss:
Lv(o)0,t) ==(,,)) V (a - e
(11)
By minimizing the above loss over Ø, we train Va to predict the velocity given the location after running the Hamiltonian ODE with starting conditions defined by II. For a sufficiently rich class of functions Vg. the minimizer Vø- of the above loss is the expected velocity conditioned the location (see Appendix D):
Ve (z, f) = E[uf< x]
(12)
As we will see, this quantity "all you need" for generative modeling and score matching. We call V-- the Hamiltonian velocity predictor (HVP). We note that many of flow-based works also implicitly learn to predict velocities [31], although they do not consider them separate states.
(a) Density (2)
(b) Learnt score F's
(c) Learnt velocity predictor Vi
Figure I: Results of training HSM on a Gaussian mixture. The score vector field faithfully recovers gradients the density. The optimal velocity predictor is zero everywhere.
Hamiltonian Score Matching
In this section, we provide a novel score-matching method using PH-ODEs and velocity predictors. We will first connect the score function preservation property of Hamiltonian systems (Sec- tion 4.1), and then introduce a new score-matching objective derived from this property (Sections 4.2 and Section 4.3).
4.1 Characterizing Scores with Hamiltonian Dynamics
The conservation of the Boltzmann-Gibbe distribution BG (see Equation (4)) is the crucial property that enables Hamiltonian Monte Carlo as it allows for proposals of distant states enabling fast mixing of a Markov chain. The inspiration of this work was to take the inverse perspective: rather than considering the preservation of x BG useful property of the Hamiltonian ODE, we ask whether is the defining property of the score function. In other words, is any vector field that preserves the Boltzmann- Gibbs distribution under a PH-ODE automatically the score? And if yes, could we train for this property to learn score? As we will show, we can characterize the score with an easier-to to trairain preservation property solely depending the velocity predictor.
Theorem 1. Let > and Fo(x)) a force field. statements are equivalent:
TBG π 2 (0., Ia)- The following
1. Score vector field: The force field Fe equals the score, iee. Fo(x) = V. logoo (x) for
π a-amosst every x E Rdd
2. Preservation of Boltzmann-Gibbs The PHIODD with Fe preserves the Boltzmann-Gibbs distribution TBG.
3. Conditional velocity is zero: The velocity given the location after running the PH-ODE with Fg is zero if starting conditions = (Io, vo) are sampled from nnC:
TBC
[[o((2))2 = for all 0 <t<T
(13)

---

A proof can be found in Appendix C and is based on the fact that test functions linear in have vanishing expectation if Equation (13) holds. We note that the equivalence of conditions (1) and (2) is well-known in statistical physics.
4.2 Hamiltonian Score Discrepancy
Condition (3) in Theorem naturally motivates a new way of training F9 to approximate score. Specifically, our goal is train the force field Fg such that its optimal velocity predictor is zero. By Theorem I, it necessarily holds Fe V log in this case. Unfortunately, such bilevel optimization not tractable with stochastic gradient descent in general, as it contains two different objectives.
However, a simple trick allows us to convert the above into a single objective. For this, we define the Hamiltonian Score Matching loss:
(14) where C(O, = E|[|u[11?] As the value of C(0, 1) is constant in d it holds that the optimal velocity predictor is also the unique minimizer of the Hamiltonian Score Matching loss (o10,, 1). However, while the argmin the same, the actual obtained minimum value is drastically different as the next proposition shows.
Lusm @ 0, =Ez~<nG (=
(V,
= Lv(o)),t) - C(O.t) -(),tt))
Proposition 1. For a sufficiently rich class of functions (Vo)ger, it holds that
Dhsm (0]t, πT) == min Lism (0100 t) = Ex-wsa ||u]|>10]]]1 [=]]]]]]
(15)
ΦE!
The proof relies on plugging the identity of Va. (see Equation (12)) into Equation (14)) and can be found in Appendix E. By condition (3) in Theorem (see Equation (13)), we want to minimize Dnam (0 T) order to learn scores. For this, let's define a distribution À with full support over [0,
for T E IR U {2o)). With this, we define the Hamiltonian score discrepancy (HSD) as
Dasm (0 T) Ex~2 [0|s
(16)
Note that the discrepancy defined for an arbitrary (regular) vector field F/ not restricted scores of probability distributions. By Theorem 1, the discrepancy fulfills all properties that we would expect from a discrepancy to hold: (00l=) > 0 for all I and (0) = 0 if and only if Fe V log We summarize the findings in the below theorem.
Theorem Minimization of the Hamiltonian score discrepancy results learning the score V log n fr = arg min Brrm (f) () = 80- = V log π
(17)
The full proof is stated in Appendix F. The Hamiltonian score discrepancy gives natural measure of how far vector field is from the desired score vector field. However, first, it seems rather abstract. The following proposition shows that minimizing this measure has a very intuitive interpretation. In fact, it is closely connected to the explicit score matching loss Lesm (see Equation (5)).
Proposition 2 (Taylor approximation of HSM loss). There exists error term E() such that
D O1t, T) = 21 Less (0; T)) + =(t)
(18)
and lim t-0 27 (e()) + O.
A proof can be found in Appendix G. Intuitively, minimizing the Hamiltonian score discrepancy, therefore, consists of pushing the parabola in Equation (18) down onto the x-axis. The above theorem also indicates the optimal choice of T: one should choose 7 high enough to have a loss value high enough to give signal but low enough to avoid errors due to numerical integration of the ODE.
4.3 Hamiltonian Score Matching
Beyond its theoretical value, we can explicitly minimize the HSD, method we coin Hamiltonian Score Matching (HSM). To minimize the HSD, two networks V. and Fo can jointly optimize Equation (14). There are two difficulties coming along with this: (1) One has to simulate trajectories. This can be done via Neural ODEs [7] with constant memory. (2) One has to run min-max optimization. Here, a big toolbox developed for GANs for training stabilization can be used [35, 19]. On the other hand, we hypothesize that HSM has two advantages: (1) every trajectory of HSM gives several points of supervision effectively augmenting our data and (2) we can learn the original ("unnoised"") data distribution π. However, please note that we do not propose Hamiltonian Score Matching as a replacement for denoising score matching in diffusion models. Rather, it is a scalable alternative to score matching methods that learn the original (" ('unnoisedd") data distribution
5

---

5 Hamiltonian Generative Flows
Next, show that training general velocity predictor of Hamiltonian ODE is useful even if Fo F V log x. This leads to a generative model that we coin Hamiltonian Generative Flows (HGFs). As Fg F is fixed and not trained here, write (Ig, (4) (a of") for the solution of the PH-ODE. Let us denote [1 v, as the distribution of (It, Ut) at time t and the location marginal
J [(x, v, Dd (x;
(19)
The location marginal describes a probability flow starting from our data distribution π T(; (). turns out that the optimal velocity predictor is exactly the vector field that generates π(,, t).
Proposition 3. Let тT be the distribution such that 2² wT. Let V be the Hamiltonian Velocity Predictor (see Equation (12)). Then by sampling 2T ~ TT and running the velocity predictor ODE
d x(t) dt
Vo- (x, x00)
(20)
backwards in time, we will have s=0) n. i.e. can sample from the data distribution x. More specifically, the optimal velocity predictor Vo- generates the probability path (T,
The proof uses the fact that the vector field G(æ, v) = (v, Fo ()) is divergence-free to show that Vo- fulfils the deterministic Fokker-Planck equation (see Appendix H). The above proposition allows us to build a generative model by training HVP. We coin this model Hamiltonian Generative Flows (HGFs). To make this framework tractable, we need two criteria to be fulfilled:
1. (C1) Forward ODE efficiently computed: For efficient training, we need to be able to compute x of efficiently either via an analytical expression or ODE solvers.
2. (C2) Initial distribution should be approximately known: In order to be able to sample the initial point of the ODE faithfully, we need to (approximately) know π (2x T).
m(s) A'0.
m(a) Uy ~ M(L
-20
(a) Diffusion
(b) Flow matching
(c) Oscillation HGFs.
Figure 2: Evolution of various HGF's joint coordinate-velocin space from = 0 (blue) /o T
(red) with trajectories (black). Data distribution (xx) = 0.4 (N 1) + 0.6 (2 1). Diffusion models and flow matching have zero force fields, i.e. the velocity does not change. Diffusion models do not converge in finite time (here, T = 3). The coupled distribution in FM allow for a convergence for T = 1. Both distort the joint distribution. Oscillation HGFs only rotate the distribution.
6 Diffusion Models and Flow Matching as HGFs with zero force field
6.1 Diffusion Models as HGFs
We can recover diffusion models with variance- (VP-SDE) noising process [45] as special case of HGFs. If we simply set IT mBG and (x:)) 0- no force applied. In this case, get that 24 = x + tu and i4 = v leading to the training objective:
E E (o.
(x,,,))) t) vz -EE Vo(z + te,t) =
which equals denoising matching (Equation (6)) with model -tVo(z, V log TT(t) with ott) t. In this case, Hamiltonian Generative Flows correspond to training diffusion model

---

and the velocity predictor corresponds to a denoising network (often denoted as ea in DDPMs [2I]. It then holds ØT ит (o0,, (() and the velocity predictor ODE then reduces to the well-known probability flow ODE formulation of diffusion models with noise schedule (t) = t:
d
2T AT ~ (0, of (t)La),
di -=(() - ((t))==o())(((Vaa) (o (z)
())()),t)
In fact, the above is a universal way of modeling diffusion models [26]. In other words, diffusson models are special models correspond to different time rescaling and preconditioning of the network. The location marginals (2, t) only fully converge to Gaussian in the limit of + co (see Figure 2).
of HGFs for the zero-force field. In this perspective, different diffusion
6.2 Flow Matching as HGFs
Flow matching with the CondOT probability path [31], a current state-of-the-art generative model, can be easily considered HGF model. As in diffusion, let consider the zero force field F/ and let's consider a coupled initial distribution II:
n. P=6-I
(O Ia)
(21)
Similarly, the velocity prediction loss corresoonds to the OT-flow matching loss:
==== Vll t)a + te, t) - (e - (e-x))
(22)
and flow model corresponds to the velocity predictor ODE. Therefore, diffusion models and OT-flow matching are both HGFs with the zero force field - the difference lies in coupled construction of the initial distribution (see Figure 2). The coupled construction allows OT-flow matching to have straighter probability paths, leading to improved generation quality for the same number of steps [31].
7 Oscillation HGFs
So far, we studied optimal velocity predictors V& for two extreme cases: either Fg 0 or Pg V log æT Finally, we want investigate a different choice of F9 to construct HGFs. Here, we study Oscillation HGFs that correspond to a natural extension. In Appendix I, we give another example that we coin Reflection HGFs.
A simple design of force field is to use the physical model of harmonic oscillator, i.e. set Fo(x) = -a²x with II BB and a > 0. The flow of the ODE then becomes:
(Iz, Ut) (cos(at)
sin(at)e; a sin( cos(at)o)
(23)
Le. condition (C1) is fulfilled as can simply compute the ODE analytically. Setting T T= (2a)).
it holds that (I,, vt) = (v, -az). In particular, = (O, Ia/a²) ) condition (CI) is easily fulfilled.
Therefore, the above choice gives natural generative model based on harmonic oscillators that we coin Oscillation HGFs. To summarize, they have the following simple training objective:
Pe<k,u~W00... ( sin(al)
u,
[- sin(at) cos(at)))))
(24)
natural choice for a is to set a = (including n 0) of the inputs and outputs in the training objective is constant in time (see Figure 2), Le. for all O:
With this, the scale of the n-th derivative
d" 1 xz 11 duy
[I
B E
aa a-2dd
d v
and
(25)
In the context of dramee Langevin diffusion [13], it was already observed that a constant scale in velocity space leads to improved training and better performance. Here, extend this idea of a constant scale from the velocity to the n-th derivative.
8 Related Work
Assessing and training energy-based models. Stein's discrepancy [16] is a well-known measure to assess the quality of energy-basee models based on Stein's 's identity [46]. Based this metric,
7

---

[17] developed a method that is similar to ours where a critic is optimized to assess the quality of an energy-based model via Stein's 's discrepancy and jointly trained with the energy model via min-max optimization. [25] introduced score matching as a method by showing that the explicit score matching loss (see Equation (5)) can be implicitly trained if one computes the trace of Hessian of the energy function an expensive step. To expedite this, [34] introduced curvature propagation for an unbiased Hessian estimate, while [43] used Hutchinson's 's Trick to estimate the trace. In practice, both methods suffer from high variance due to their underlying Monte Carlo estimators.
Flow matching and Stochastic interpolants. As already seen for special case in Section HGF's are strongly connected to Flow Matching [31] and Stochastic Interpolants [3]. They construct probability paths that fulfill the continuity equation by predicting derivatives of flows (i.c. velocities) in the same way how in this work, The differences of these works lie in the design perspective: HGFs consider 2nd-order ODEs in an extended state space Rd x Ed with simple initial velocity distribution (here, (O, Ia)), while FM considers 1st-order ODE paths in Rd converging to a simple final location distribution. Flow matching conditions on final states (usually at = 1). while our framework conditions on the velocity of the current state (arbitrary t) and is centered around forces. This work arrives at the ideas of conditional velocity predictors via the search of properties that conserved under Hamiltonian dynamics (see Theorem I). We elucidate the mathematical connection in more detail in Appendix J.
predict velocities as marginals of extended state space.
Generative models and Hamiltonian physics. V-Flows also consider augmenting the state space with velocities deriving ELBO objective for CNFs [6]. [13] extend diffusion models to joint state-velocity samples that converge to a joint normal distribution. One difference is that we only need to the backward equation in state space Rd as opposed extended state-velocity space Rd x Rdd Though rather unmotivated, Oscillation HGFs could, in principle, also be derived as an EDM model with preconditioning [26]. Finally, several works have, like this, explored generative models based on specific physical processes, e.g. Poisson flow generative models [50, 51J. A few works also combined Hamiltonian physics with deep learning. For example, [18] use conservation of energy as an implicit bias to learn networks for physical data. Conversely, deep learning was also used to accelerate HMC sampling [15], e.g. by training custom MCMC kernels [30] or correct for complex geometries via flows [22]. Very recently, score matching approaches were also designed to leverage existing force fields as part of diffusion model that samples from an energy landscape [2, 12].
Acceleration Generative Model (AGM). The AGM model [9] also uses constructions in phase space (joint position and velocity space) and 2nd order ODEs. While AGM focuses learning the force field, our approach primarily focuses on learning the optimal velocity predictor. While we also consider optimizing the force field by minimizing the norm of the optimal velocity predictor, this happens in the "outer loop" of the maximization the inner loop optimizes the optimal velocity predictor. Further, ATM focuses on bridging two desired distributions by posing a stochastic bridge problem in phase space. We do not consider the problem of bridging distributions. In contrast, our framework centers around energy preservation and divergence from that preservation (for optimal velocity predictors that are not zero). Specifically. we establish a connection to Hamiltonian physics and property of the preservation of energy. This allows us to introduce further bi-level optimization and the possibility of joint training for score matching.
9 Experiments
9.1 Hamiltonian Score Discrepancy
As we introduced Hamiltonian score discrepancy as a novel score-matching metric, we first empir- ically investigate our theoretical insights Gaussian mixtures (see Figure 1). As one see in Figure 3a, the Hamiltonian score discrepancy is highly correlated with the explicit score matching loss. Further, we can validate empirically that the Taylor approximation derived in Proposition 2 is pretty accurate for large (see Appendix 1). Overall, these results indicate that the Hamiltonian score discrepancy is a natural metric to assess score approximations.
Further, we investigate whether explicitly minimizing the Hamiltonian score discrepancy leads to accurate score approximations We jointly train velocity predictors and score networks as described in Section 4. As one can see visually in Figure this approach can faithfully learn score vector fields. In addition, we investigate the signal-to-noise ratio for gradient estimation. As shown in Figure 3c, the gradient estimates of HSM have significantly lower variance compared to denoising
8

---

score matching at lower noise levels σ. The reason for that is that we allow for supervision across full trajectory locations for the data points effectively acting data augmentation. Of course, this comes at the expense of simulating the Hamiltonian trajectories for ~ 5 steps.
sec Hoar bagreaau
0.5éé
60
Bid
Eu (
(R:)
Timet
Mean
(a) ESM loss HSD for net- works trained epoch
(b) Empirical HSD Taylor ap- proximation (see Proposition 2)
(c) Std absolute of deriva- live of param. of score network.
Figure 3: Empirical investigation of Hamiltonian score discrepancy (HSD), (a) The Taylor approxima- tion is good approximation. (b) Hamiltonian score discrepancy strongly correlated with explicit score matching loss. (c) Signal-to- ratio is significantly better for HSM vs DSM for low a.
Figure 4: Image generation examples based on Oscillation HGFs for FFHQ.
9.2 HGF experiments Image Generation
In the form of diffusion models and flow matching, HGFs have already been tensively optimized and achieved state-of-the-art results. Instead, we investigate whether also other non-zero force fields, specifically Oscillation HGFs, can lead generative models of high quality.
We focus image generation benchmarks.
Specifically, we train a Oscillation HGF on CIFAR- 10 unconditional and conditional. As two central benchmarks, we use the original SDE formulation of diffusion models [45] as well as the EDM framework [26]. a highly-tuned optimization of diffusion models. Our hypoth- esis is that Oscillation HGFs should work well out-of-the-bont the-hhhe.... the scale of their inputs and outputs stay around constant (c.f. Eq. 25). There- fore, we remove any preconditioning optimized for diffusion models (scaling of inputs and out- puts and skip connections) [26] and train on the default DDPM architecture [21] (see details in Appendix I.). Our results encouraging: with- out hyperparameter tuning. Oscillations HGFs can sample high-quality images and surpass most previous methods (see Table 1) measured by Frechet Inception Distance (FID) [20]. While they still lack behind the EDM model, this dif-
Table 1: Sample quality (FID) and number of func- tion evaluation (NFE).
METHOD
FID
NFE $
CIFAR-I0 (uncundiiooal):3222
StyleGAN2-ADA
2.92
[2]]
3.17
LSGM [47]
2.10
147
PRGM [50]
2.48
TEESEE[G [45]
3.77
35
VP-SDE
EDM [26]
1.98
35
FM-OT (BNS) [40]
2.73
Oscillation HGF (ouns)
2.12
35
CIPAR-10 (class conditional).12233
VE-SDE [45]
3.11
35
VP-SDE [45]
2.48.
35
EDM [26]
1.79
35
Oscillation HGF (ours)
1.97
35
FFHQ (4mcondiiooaa)-64r6d
VE-SDE [45]
25.95
VP-SDE [45]
3.39
79
[26]
2.39
Oscillation HGF (ours)
2.86
79
ference might well be explained by the fact that architectures and hyperparameters have been optimized for diffusion models over several works that hard to replicate.

---

To investigate whether similar results can be achieved similar performance at higher resolutions, we perform another benchmark on the FFHQ dataset 64x64 resolution. Here, our results are similar: Oscillation HGFs improve upon original diffusion models with a small performance margin to the EDM model. They can generate high-quality faces that appear realistic (see Figure 4).
10
Conclusion
Our work systematically elucidates the synergy between Hamiltonian dynamies, force fields, and generative models - extending and giving a new perspective on many known generative models. We believe that this opens up new avenues for applications of machine learning in physical sciences and dynamical systems. However, several limitations remain. Minimizing the Hamiltonian Score Discrepancy (HSD) via a default mix-max algorithm is scalable but requires adversarial optimization. Future work can focus on adapting the HSD framework, e.g. to develop denoising Hamiltonian score matching that could allow for guaranteed convergence. For HGFs, the extended design space presented has only been explored for data without known force fields (image generation, here). Future work can focus specific applications that require domain-specifte force fields, e.g. for molecular data. Further adaptions might be required in such settings as such data often lie on manifolds. further challenge is that HGFs not necessarily converge to a known distribution for more complex force fields. Therefore, we anticipate that future work will focus on adapting HGF' and related models to this challenge design Nomain-spectie models.
Acknowledgments and Disclosure of Funding
This work was funded in part by GIST-MIT Research Collaboration grant (funded by GIST), the Machine Learning for Pharmaceutical Discovery and Synthesis (MLPDS) consortium, the DTRA Discovery of Medical Countermeasures Against New and Emerging (DOMANE) threats program, and the NSF Expeditions grant (award 1918839) Understanding the World Through Code.
P.H. would like to thank Gabriele Corso, Ziming Liu, and Timur Garipov for helpful discussions during early stages of the work.
References
[1] J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, J. Ballard, J. Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold Nature, pages 1-3, 2024.
[2] Akhound-Saae Sadegh, Rector-Brooke A. J. Bose, S. Mittal, P. Lemos, C.-H. Liu, M. Sendera, S. Ravanbakhsh, G. Gidel, Y. Bengio, et al. Iterated denoising energy matching for sampling from boltzmann densities. arXiv preprint arXiv:2402.06122 2024.
[3] M. S. Albergo, N. M. Boffi, and E. Vanden Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.0879) 2023.
[4] V. I. Amol'd. Mathematical methods of classical mechanics, volume 60. Springer Science & Business Media, 2013.
[5] N. RabeR and J. M. Sanz-Serna. Geometric integrators and the hamiltonian monte carlo method. Acta Numerica, 27:113-206 2018.
[6] Chen, C. Lu, B. Chenli, J. Zhu, and T. Tian. Vflow: More expressive generative flows with variational data augmentation. In International Conference on Machine Learning, pages 1660-1669. PMLR, 2020.
[7] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[8] Chen, E. Fox, and C. Guestrin. Stochastic gradient hamiltonian monte carlo. In International conference on machine learning, pages 1683-1691. PMLR, 2014.
[9] T. Chen, J. Gu, L. Dinh, E. A. Theodorou, J. Susskind, and S. Zhai. Generative modeling with phase stochastic bridges. arXiv preprint arXiv:2310.07805, 2023,
10

---

[10] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprini arXiv:2303.04137,
20233 2023.
[11] G. Corso, H. Stärk, B. Jing, R. Barzilay, and Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprint drXiv:2210.01776, 2022.
[12] V. De Bortoli, M. Hutchinson, P. Wirnsherger, and A. Doucet. Target score matching. arXiv preprint 24XX::24022086667, 2024.
[13] T. Dockhorn, A. Vahdat, and K. Kreis. Score-based generative modeling with critically-damped langevin diffusion. arXiv preprint urXiv:2112.07066, 2021.
[14] S. Duane, A. D. Kennedy, J. Pendleton, and D. Roweth. Hybrid monte carlo. Physics letters B, 195(2):216-222, 1987.
[15] S. Foreman, X.- Y. Jin, and J. C. Osborn. Deep learning hamiltonian monte carlo. arXiv preprint (0IV:2205.034/88 2021.
[16] Gorham and L. Mackey, Measuring sample quality with kernels. In International Conference on Machine Learning, pages 1292-1301 PMLR, 2017.
[17] W. Grathwohl, K.-C Wang, J.-H. Jacobsen, D. Duvenaud, and R. Zemel. Learning the stein discrepancy for training and evaluating energy-based models without sampling. In International Conference on Machine Learning, pages 3732-3747. PMLR, 2020.
[18) S. Greydanus, M. Dzamba, and J. Yosinski. Hamiltonian neural networks. Advances in neural information processing systems, 32, 2019.
[19] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
[20] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[21] Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[22] M. Hoffman, P. Sountsov, J. V. Dillon, I. Langmore, D. Tran, and S. Vasudevan. Neutra- lizing bad geometry in hamiltonian monte carlo using neural transport. arXiv preprint arXiv:1903.03704, 2019.
[23] M. D. Hoffman, A. Gelman, et al. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res., 15(1):1593-165 2014.
[24] M. F. Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in StutisticsSimulaate and Computation, 18(3): 1059- 1076, 1989.
[25] A. Hyvärinen and P. Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
[26] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35:26565-2657 2022.
[27] T. Karras, M. Aittala, J. Hellsten, S. Laine, J. Lehtinen, and T. Aila. Training generative adversarial networks with limited data. ArXiv, 0ew/2006.06676 2020.
[28] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 81 10-8119,, 2020.
[29] U. Köster, J. T. Lindgren, and A. Hyvärinen. Estimating markov random field potentials for natural images. In International Conference on Independent Component Analysis and Signal Separation, pages 515-522. Springer, 2009.

---

[30] D. Levy, M. D. Hoffmmnn, and J. Sohl-Dickstein, Generalizing hamiltonian monte carlo with neural networks. arXiv preprini (tt:::/711.0026... 2017.
[31] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. arXiv preprint arXiv:22 10.02747, 2022.
[32] X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint wrXiv:2209.03003 2022.
[33] E. Marsden and T. S. Ratiu, Introduction to mechanics and symmetry: basic exposition of classical mechanical systems, volume 17. Springer Science & Business Media, 2013.
[34] Martens, I. Sutskever, and K. Swersky. Estimating the hessian by propagrapgg curvature. In International Conference on Machine Learning, 2012.
[35] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint 180ivv/802.05992 2018.
[36] R. M. Neal et al. Meme using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011.
[37] B. Øksendal and B. Øksendal. Stochastic differential equations. Springer, 2003.
[38] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[39] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[40] N. Shaul, Perez, R. T. Chen, A. Thabet, A. Pumarola, and Y. Lipman. Bespoke solvers for generative flow models. arXiv preprint 20. 19075, 2023.
[41] J. Sohl-Dickstein, F. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics.. In International conference on machine learning, pages 2256 2265. PMLR, 2015.
[42] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[43] Y. Song, S. Garg, Shi, and S. Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pages 574-584. PMLR, 2020.
[44] Y. Song and D. P. Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.03288 2021.
[45] Y. Song, J. Sock-Dickste,n, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 2020.
[46] C. Stein. A bound for the error in the normal approximation to the distribution of sum of dependent random variables. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory, volume 6, pages 583-603. University of California Press, 1972.
[47] A. Vahdat, K. Kreis, and J. Kautz. Score- generative modeling in latent space. In Neural Information Processing Systems, 2021.
[48] P. Vincent. A connection between score matching and denoising autoencoders, Neural compu- tation, (3(7):1661-167) 2011.
[49] J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahern, A. J. Borst, R. J. Ragotte, L. F. Milles, et al. De novo design of protein structure and function with rfdiffusion. Nature, 620(7976): 10::: 108900991111 1100, 2023.
[50] Y. Xu, Z. Liu, M. Tegmark, and T. Jaakkola. Poisson flow generative models. ArXiv, abs/2209. 1 1178, 2022.
12

---

[51] Y. Xu, Z. Liu, Y. Tian, S. Tong, M. Tegmark, and T. Jaakkola. Pfgm++: Unlocking the potential of physics-inspired generative models. In International Conference Machine Learning, 2023.
[52] Y. Xu, S. Tong, and T. Jaakkola. Stable target field for reduced variance score estimation in diffusion models. arXiv preprint 20000 2023.
13

---

A Score matching methods
To make score matching tractable, express the explicit score matching loss via [25]
Lism (8: T) = Exnax V Fo(a) + =1 = Lesm (0; +
(26)
where V Fo is the divergence of the vector field and the constant C. is independent of 0. While this loss has been widely used [29]. this objective is hard to optimize with neural networks as the divergence requires to backpropagate d times. Still, the divergence can be approximated via Hutchinson's trick [24] leading sliced score-matching [43].
B Hamiltonian ODE: Conservation of energy and volume
In this section, we prove the fundamental properties of the flow fi of the Hamiltonian ODE (see Equation (2)). As these properties are used throughout this work and usually presented in the context of the physics literature, we include the proofs here for completeness, solely expressing it in the language of probability theory. Throughout this section, let J E 2dx be the matrix defined by:
1d
- ld 0
B.1 Preservation of energy.
Proposition 4. For all t E I2 it holds that o ipt H
Proof. We follow [5, theorem 2.2]. For all R22d one has Jz) 0. Hence
d dt H()((=)) = (x))))) d
dt p(())) (VH() = which implies that H (x)())) = (vpo==))))))))))))) H()))
B.2 Preservation of volume.
Proposition (Symplectic property). For all = (xx, v) E 2d and the Jacobian Dipe (z)
satisfies the following equation:
((p())))
(27)
In particular, pdetDipal = 1, L.e. the is volume-preserving.
Proof. Here, follow [33, proposition 2.6.2]. First of all, it can be easily seen that Equation (27) is equivalent to the statement that the bilinear form 3(u, v) = (u, JTT)) on 2d fulfils
8(u, v) ((Drr (z)o) Vz, v E 2d
(28)
By taking derivatives of the right-hand side and afterwards using the identities B(Jx, (J²r, = - (z, 2) and 3(x, Jy) (z, l Jy) (a, v). one gets:
d di (D)x,))) (p((2))) 3 (2p:(z)) Dipr(=)o) + B(Dp(())) मे D))(2)j0)
di
d u, ((( dt Pi(2j))
=3(D dt
H (2)) htt.::/ww...............ooooonnn
= (p: Dep(==))) --
-0
where in the last step have used that the Hessian is symmetric. One can conclude
3 (=),, (=)o) = ((Doo(=) Dyo(=)u) = (u,,
Since |det J/ = 1, immediately gets |detD 1.
14

---

C Proof of Theorem 1
C.1
(1) (2): Score preserves Boitzmann-Gibbs distribution
This section gives proof that the Hamiltonian ODE defined with the score preserves the Boltzmann- Gibbs distribution. This implication is well-known, and the proof is included for completeness following [5].
For all E IR and Borel sets D, it holds by Propositions and 5 and change of variables:
TBC(f( =z-
D((D) exp(-H)dz
(29)
=2-1 /
LO((D) o sp exp (-H o pp)llettDpppldz
(30)
=z- / |DD
(31)
=D)
(32)
Note that the symplectic property is crucial, e.g. consider the simple pendulum (U(z) 22²²) and g(z) (II=|], | 0). This function fulfills TB)(g(=)) x BG (z) but it does not leave the distribution invariant (it is not symplectic, either).
C.2 (2) (1): Invariance under Boltzmann-Gibbe uniquely defines score
We write ()(2) = (x U for the solution with force field -U. Let's pick an arbitrary sufficiently regular test function f R2dd - IR and let's define
(t Ez~ (+]
(t) = Ex (+))
As the Hamiltonian dynamics with force network - VU preserve the Boltzmann- Gibbs distribution, FoU must be a constant function, i.c. derivative zero. Therefore, we can compute:
(()
di
dt ExnaBa [f()(()))]
de [((((t))
V=f((p(())) d dt Pc(2)
-EgNTnG
V=(((p(=)))" (- (-vU(a)" (21))
(2)
Lee. at time f. = O:
d
[ef(E)7 (-vür=))
Fvu (t) = (o) EE:
(33)
dt
As of also preserves the Boltzmann- distribution, can follow the same computation to also get that:
d (t) = E==(r,))=ne
v.)())T v [Fo(o))
(34)
dl.
Substracting Equation (33) from Equation (34), we get that:
0 Vzf(=)² (o(()) + ( +v
0 = (x=
(35)
15

---

Let's set f(2)= = fx = (Fo(2) + (U(x)). Then
(0" OF (DP()))))) (= vVU(2))
v=f(2,0)=
(36)
where DF denotes the Jacobian and V the Hessian. Then inserting Equation (36) into Equa- tion (35) we get that:
0 (DF(())) + T + +
(16(0) +v)ca)
0 -B (2,
= =B;=(a,)) ~NG
((()))))
-BoNs |1 -=-VI
This implies that Fo(x)) VU(()) v log ग for x-almost every x.
C.3
(3)+s (1)
Finally, we show that condition (3) is equivalent to condition (1), Note that if (xf, ug) TB @ (0, Ia), then uf is independent of oy and it holds that Elvy [=]] = O. Therefore, condition (2) trivially implies condition (3). As we have already have shown that condition (2) is equivalent condition (1), it sufficient to prove that (3) implies (1). The proof is similar the proof for (1).
Let's s assume that the following condition holds for ~ ABG
0 = E[v x x] for all x
(37)
where for brevity we write (x), of") = (2). Then for f(z) = f(x,v) = = (F((((z) + (U())) as
selected above, we equally derive:
(t) - EEzwinG [f(((
EEf((2 uf']
(I((c))((FF((f))) V
=E (Fo(()))) -U(=)))vV((=)))
-E(O]
-0
In the same way as above, we can now deduce that Fo(() = Similarly to above, we can complete the proof by using Equation (36):
d E. B
d BINTeal((P((()))
0
di
di
Vzf(2)² [o(z)++U(a))
-EEAT [F(((())) - [U))]
Again, we can deduce that Fo(x)) = VU)(x) V log (2)) for alllmost every x.
D Proof of Equation (12)
We formally proof the identity in Equation (12). We will use the following well-known characteriza- tion of the expectation:
Lemma 1. Let X E Rd be random variable. Then:
E[X] arg min [[[] X -
(38)
mERU
16

---

Proof. Let c E[x] E R then
B||l X ml|
= =||| c c ml|²]
= [|| X cl2 + + - -c) m) + ||c- - mll²]
= [I|| X
-)) - -m]] + mm|
=B [[| cl + + ml|
> _ cll3]
This implies the statement.
Therefore, we can apply Lemma conditionally on x. to see that
(x, = ETuul=> =
E Proof of Proposition 1
We can derive:
Lism (0l0, t) =1E =EgwKBd
(xf, + +lVVo t]1
(39)
= -E Ezaznn
|||/||| (v t)a ut + || ( [)13] BEwano [||x(]]]]
(40)
E Ezaana
[[[Vo(z).) [1a] Egwnna [100.111]
(41)
(42)
As BE [|] is a constant in Φ, we can see that
arg min Lasm (ø)0, arg min Ez-ana V
Hence, the Hamiltonian velocity predictor is the minimizer of the above objective. Inserting Equa- tion (12) into Equation (39), get
max Lhsm @00, -Las ( t),))
E Eganna
22 |- 7
E Egaana
[2 [ uf [-]] - -1E
EzwaBa
[2 (0 - ||
Egon
[2|[B[]|| | (1]
=Egoree aa
[|
where have used that the conditional expectation B[ug [x]]] constant conditioned on xf This finishes the proof.
Proof of Theorem 2
As X is a distribution with full support over [0, T), it holds that E if and only if for very 0 s < T (up to measure zero)
Drom (0] ())
[[ [1 = 0
E [ = 0
By Theorem 1, this is equivalent to Fg V log h. Hence, Dnsm (0) 0 if and only if Fg V log
However, remark that maximizing across t leads to the same result for all 0 < t < under reasonable regularity conditions. More specifically, assuming that æ is a smooth density with full support, the and x. Therefore, as long as (Vo)øer covers all
Remark. Technically speaking, maximized Vo for fixed / in Proposition
map (xr, + E[oe] ! = x]] continuous continuous function in t and z, the result above is the same.
17

---

G Proof of Proposition
The proof relies Taylor approximation of Tnsm (0, around = 0. To finish the proof, have to show the following three equations:
Dnsm (0)t, |= = 0
(43)
d
nmm (f]t, ())t,x))r=) r=)==0 =0
(44)
dt
d²
D nsm (0]t, x)),, = 4. xt) (l V log (x)
(45)
d't
Proof of Equation (43) Note that at time = 0, holds that og = N(, Iz) and x. x Therefore.
E[of = E Ee- (1 =0
Therefore, by Proposition
Dsm (6)0, 0) [E| 1 ||||0|
Proof of Equation (44)
We can compute:
d
d Dasm (flt, -E
2E|E[// u dt
[E
Elo x1
dt
dt
So at time t == 0:
Eloe [=) = 2k|0" d Eley x 1
π) (= [[E[E[((x] Dusm (
dt
dt
dt
Proof of Equation (45) Let's take the second derivative:
d² d2t
Tham (0 1t) =2 d E
d
F(v? w0]1]
dt
dt
d²
-2
2Ell|
Biel (1
det
dt
by the product rule. And at time = 0:
d2
d² d't
d 2æl1
Dhsm (0), 1= F0"
R
2B Blof
de 1|o
1o
d2t
(46)
Let n.. Rdd Rd + R be the density (xf, uf) and compute:
T,( v) du Biuf|ag =x] xx]=| un (v(2)dd /
(47)
m7 (x)
d
d (x, du
Bloelag | x) v dt TŪ (x)
(48)
dt
(1)) d nt (x, u)
n (x, u) a (x))
do
(49)
(x)(
Let G( (v, (o be the Hamiltonian vector field. By the deterministic Fokker-Planck equation [37], we can derive that:
d
V
[m[ - VG
dt
Note that the Jacobian of F is given by
DG(x, ( 0
0
L
(FF(z)
18

---

In particular, G' is divergence-free, i.. V G = tr(DG) = 0. Therefore,
d 8 GJ v
dt "
and for particular 2; u IRC:
d
T (oo(x))
di ai (z, 1) 1) -
VTB
(50)
As To TBG by construction, we can derive that
(ng v) = V Taa (2, v)
z V [[xpp(- (U(x) - [|o||]]]]
Z VU(x) exp(-U((2)
1 -
(U((x) TBC(S,0)
Inserting this into Equation (50), we get
d
v (o(t)) -(1b()) ) (TU(#)
Ti (2,
uo TBC(0,0)
dl
-F=(F(( + VU()))
And hence,
d
/ f
d
dt n (x)
di
/ (Fata)
[ [ La)de
We can insert these identities into Equation (49) to get:
d dt
x (x) TH (x. u) n (x. v) de
nt (x) do
Eluglo = x0/=00
(T(
x (x) (Fo(z) + (U(x)) (T) - du
Txx)²
+U VU()) of (Fo(z)
N(1 OTa)d
xxx)²
I vo (oo;
(Fo(x) + VU(x))
(F(((()) + VU(z)) =Fo(x) VU(x)
Combining this with Equation (46), we get that:
d
L(0 p =2E|1| d
Blo a 0
(51)
d
dt
=2 [x()) =2B[]|Fb(a)
VU()))
(52)
V [1)g
(53)
= 4esssm (0; m))
(54)
19

---

Taylor approximation Finally, we can combine the above derivations to get Taylor approximation of (f, t) around t = 0, ic. for e - IR with lim 2 [e(()] = we get
2-0
d
1
Dasm (0),, Dnam (0 0) + l 1-
Dssm (01m 1) 2" (00, + e(l)
dl
12 4Tess (0; (t)
21 Tesm (0; π) + e(l)
This finishes the proof.
H
Proof of Proposition 3
Again, let's consider probability distribution π RRd R and the ODE
d at (t), "(t))=== = Fo(x(t)) dt
d
where we know allow F6 to be time-dependent. Let (zz, 01) be a solution the above with ODE with (ro, vo) = (I, v) ~ π @ (0, Lz). In addition, write [[ v, () for the distribution at time (i.e. (za, v1) ~ II(.,,, ()) and the location marginal
f Ma,,0 u. t)dv (22 t)
Finally, we write
V(x,t) E[ve|=:]] uo(oll, Vo-
for the optimal velocity predictor.
Deriving marginal ODE. We now show that the evolution of the first marginal can be replicated by an ODE that only depends on V. By the Bokker-Planci' equation, can derive for G(z, v) (v, Fo
i at
[(,, v,() Va. [TG] v,()
G(I, (,,,,(E, V.. Ve. ((),,
--Fo(z)
V
V vz,,I(I,,U,t)
=- (,ll,,,( + V
20

---

where we used in the third equation that V2, G = (i.e. that G is divergence-frees free). Therefore, we derive that:
/
8
(Z,
[1 t,
Ōt
δt
V V.ll(a, 0, 0) Fa(x" e, t)]du
/
Vall(z, () Fa(x)) / Vol,,, v,
" V.I((z, t)du 0
V. V.l((z u, t)du
V / [ell(,,, v,Eid
11(xr, v, dul
v (1(x,)))
[T(,,
V, [I(x,)) t)
UII(u/ t)du] V. (V(,,
In other words, the vector field V(x, t) satisfies the continuity equation. Therefore, if we initialize IT (") and evolve the ODE
d
d =1)) ((x(().t)
backwards until & = 0, we know that z00) TT, 0) we sample from our data distribution.
I Reflection HGFs A New Example of HGFs
We briefly show that HGFs can give us models other than Oscillation HGFs. For this, we introduce Reflection HGF's here and plot the result of training them in Figure 5 and Figure 6. The idea of the model is that particles can move freely the boundaries of the data domain making the particles bounce back (this can be made rigorous with von Neumann boundary conditions). With normally distributed velocities, the distribution of particles will converge towards a uniform distribution. Further, this model can be trained manner. We trained this HGF model on a simple toy distribution (see figure and figure 4 in the attached PDF). Such HGF model is distinct from previous models and illustrates that HGFs are not restricted only Oscillation HGFs, diffusion models, or flow matching.
box without collision with walls ("very strong forces") at
simulation- free
Data
Velocity
Figure 5: Data distribution (left) and velocity distribution (right) used for Reflection HGFs initial distribution. With the above starting conditions, reflection (="infinite force") at the boundaries of the domain is used to simulate trajectories forward (this can be computed in closed form a simulation-free manner).
21

---

Figure 6: Illustration of sampling with trained Reflection HGFs, At time t = 3.0. the distribution is uniform distribution (sampled by construction). By running the parameterized Hamiltonian ODE backwards in time, we recover the data distribution (sce Figure 5).
J
Connection between Flow Matching and HGFs
Let A Rd 4 RR + Rd be time-dependent vector field and the diffeomorphic flow defined by the ODE:
dt r() A(x(((),
x00) =I
ie. 1 - (x)))) is solution to the above ODE. With this, we get:
det - = d
d²
d
dt =()
di
di A()(t),t)
P,A(x(0),)))) + 8
t()(1)t)
0t
= ())(((()))))((((() Ot
A(x(t),tt
Therefore, if we define the force field,
F(x, Aar (xx 1) 1(x,t)
0l
we can extend the state space to (I, v) and consider the ODE:
(x(0), v(0) -(x, A(z 0))
- dt (t)) F(x(()
Then every solution (xi, v2) the above ODE is also solution the flow matching ODE and vice versa. The conditional velocity predictor loss looks as follows:
d
E|| ())
sh (x+)))
dt
This is exactly the conditional flow matching loss (see equation (14) in [31]).
K Connection between EDM and Oscillation HGFs
In this section, we discuss the relation between HGF and EDM [26]. The EDM paper assumes the perturbation kernel is isotropic Gaussian with standard deviation (t). Thus, the intermediate distribution (: (t)) = Pdata (0,, ()) If we further scale the original variable x with s(t)
and consider s()) [26] shows that the corresponding backward ODE of is as follows:
dğ |s(t)//s(t) ((t)) log p(T log
(55)
We will show that, the minimizer of the objective of the Oscillation HFG, i.e. Ey (c)s(()()) + sin(t)v, t) [- sin(t)y + cos(t)u]][}} equals the drift term in Eq. 55, when setting stt) cos(t) and (t) = tan(t).
22

---

Denote = cos(t)y + (tn(,)), then the training objective can be rewritten as Ny (lVV
]. The minimizer of the training objective is (V0.)) 11-)-(-mii -m + tan(y)
y
y
V (ū. = Byy 1
(56)
sin(t)
tan
On the other hand, we can re-express the score function in Eq. 55 as
1
log cos()); lan(()) V log p(ū/ cos(1); tan(!))
cos(t)
willyll 5/ cos(1) 1
(57)
tan cos(t)
Plug Eq. 57 into the backward ODE (Eq. 55), we have:
dø
[b(1)//s(t) -
Ev 5 [M] /// cos(t) 1
ttn(t)ÿ
tta(t)(
dt
tan (t)
cos(t)
y
y
]
By i
(v) dt
(58)
sin(t)
tan
in which the drift term matches the optimal velocity predictor in Eq. 57. Hence, when picking the proper scaling factors, the backward ODE (Eq. 55) is equivalent to dý = Vgg (ū, t)dt.
Recall that the EDM paper employs the simple scaling s(t) = 1 and (t) = in Eq. 55. Hence, to align with the time discretization { [.,] used in EDM during sampling, it suffices set the time discretization in Oscillation HFG to (arctan(t1),.... arctan(t,,) to ensure that the score functions are evaluated on the same as.
Remark. Rescaling of the EDM ODE will necessarily lead to the same endpoint if s(0) = 1 this is the case by construction. Similarly, changing the noise schedule will lead to the same ODE. However, mapping discretizations will not result in the same ODE, The reason for that is that in general
(Iata)(tat1 tn) (s(le+1) s(tn) of d'(n+1)(ttnt1 tn) (t(()+1) - (tn))
L Details for Image Generation Benchmarks
In this section, include more details about the training and sampling of Oscillation HGFs. All the experiments are run on NVIDIA A100 GPUs. We used PyTorch as library for automatic differentiation [38]. Our image processing pipeline follows [26]. We use the DDPM++ backbone 245,21 The preconditioning and 256 on FFHQ. We train for 200 million images in total, corresponding to approximately 3000 epochs and ~ 48 hours of training time for CIFAR- 10 and ~ 96 hours for FFHQ. As outlined in the experiments section, the hyperparameters and training procedure we used the Adam optimizer with learning rate 0.001, exponential moving average (EMA) with momentum 0.5, data augmentation pipeline adapted from [28], dropout probability of 0.13, and FP32 precision. For sampling,
removed. We set the reference batch size 516 on CIFAR-10
the same as [26]: namely,
the 2nd order Heun's sampler [26].
23

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and socictal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA]-
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA]" is not grounds for rejection. While the questions are phrased in binary way, acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The abstract is summary of the contributions and the scope of the method.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
It is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: While did not include the separate "Limitations" section. we discussed limitations in the "Conclusion" section as well as in the main text (e.g. Section 4.3).
Guidelines:
24

---

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers penalize honesty concerning limitations.
will be specifically instructed
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
The assumptions are fully stated in the supplementary material and referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
We provide experimental details in the supplementary material (Appendix L).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
25

---

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Code can be provided upon request.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
public/ cublic/guides. htt////e/ooeeuuumiiiiieee (CodeSummissionooo oisiinnccyy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /CodeSubmia ssi oniolicy) icy for more details.
/nips. wttpccppppcccccccecccccccc.ccccc ggwidggie////eg gewgddd/deeeeeuammmmimmmmm
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We list them in the supplementary material.
26

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Generating error bars for image generation benchmarks at this scale would be computationally infeasible.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yss]
Justification:
We provide them in the supplementary material.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: htt////eeripp..e/ee//e http//uurr..ccppuull//// wwtpublic//thicceeiiiooeeececeii www.deeeeleeeeeeeee..
Answer: [Yes]
Justification:
We follow all guidelines and the code of conduct.
27

---

Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: The core contribution of our paper is method. The extent of the societal
impact via deepfakes is not changed due to this work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [No]
Justification:
We do not release any data. We will release models upon publication.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
28

---

Answer: [Yes]
Justification:
We list the resources we used the supplementary material.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cow /ataseas has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No assets released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No research on human subjects was performed.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
29

---

Answer: [NA]
Justification:
No applicable this case.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "G78SPNFZSA", "reviewer": "Reviewer_g7vb", "summary": "The paper explores the application of Hamiltonian formalism for generative modelling.\nIn this framework, the score function is interpreted as a force field F, thus optimizing the parameters of F yields a score-matching objective.\nAuthors build the relation to Flow Matching and Diffusion models as special cases of their framework and suggest another special case, which they validate on an image generation task.", "questions": "- I am curious if it would be simple to enforce equivariance as in Equivariant Hamiltonian Flows by Rezende et al.\n- as the force field form is a choice now, did authors consider alternative forms other than osciallators? I am curious what is the intuition behind the choice.\n- authors mentioned conservation properties in the context of physical simulations.\nDo you think that they will still be beneficial in the general case, e.g. image/video generation?\nEven in physical systems, if the energy is not conserved, would those properties be a limiting factor somehow?", "limitations": "Perhaps the main limitation for me is the discussion of the framework's theoretical properties, e.g., conservation of energy and volume, in the context of generative modelling. It would be beneficial to see an analysis of how those inductive biases will play out in specific cases (e.g., physical simulations or image generation). Besides, it would be beneficial to provide computational details (runtime / memory) and compare them to SOTA methods.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- the formulation is novel and implies conservation of energy and volume.\n- the theoretical foundation and contribution of the work is very solid. \n- the authors derive diffusion models and flow matching as a special case of HGFs.\n- the experimental results are solid and indicate the impressive capabilities of the framework.", "weaknesses": "- in the general case, the framework requires joint learning of two functions: force field and velocity.\nThe authors themselves compare the optimization to GANs, which raises the question of training stability.\n- the authors do not compare the runtime and memory consumption of the framework with other SOTA methods, e.g. Flow Matching.\n- the experimental section is somewhat limited to a toy experiment and a single image-generation task.\nIt would be highly beneficial to evaluate the framework on another task (e.g. physics-related), although it is not critical given the high theoretical contribution of the work.\n- it is not clear how image generation is influenced by Hamiltonian dynamics. What is x, and what is v?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for insightful comments and for taking the time to review our work.\n\n> I am curious if it would be simple to enforce equivariance as in Equivariant Hamiltonian Flows by Rezende et al.\n\n**Enforcing equivariance:** Thank you for highlighting the work “Equivariant Hamiltonian Flows”. In this work, the authors modify an ELBO training objective as a constrained optimization such that the learnt distribution preserves symmetries. While such an idea could in principle be also applied to HGFs, it is not as straight-forward due to differences between “traditional” continuous normalizing flows (CNFs) and our approach:\n(1) Equivariant Hamiltonian Flows employ maximum likelihood estimation (via an ELBO, equation (1) in their paper) to train CNFs, while we consider velocity prediction as an objective.  \n(2) CNFs learn to transform noise to data in an arbitrary fashion, while HGFs fix the probability path (if the force field is fixed)\nOne would need to account for these differences by designing the force field in an equivariant manner. Depending on the type of equivariance considered, one might want to consider techniques as in “Equivariant flow matching” that can similarly be extended from flow matching to HGFs by introducing equivariances in the design of the force field.\n\n> As the force field form is a choice now, did authors consider alternative forms other than osciallators? I am curious what is the intuition behind the choice.\n\n**Choices of force fields other than harmonic oscillators:** As you point out, the choice of a force field is an important design parameter of HGFs. We, indeed, can consider other force fields different from Oscillation HGFs. For example, a simple force field is the “reflection force field”, i.e. the force field that just reflects a particle to stay within an interval [a,b] (usually the domain of the data). This leads to Reflection HGFs. To address your comment, we implemented these and illustrated them in figure 3 and 4 in the attached PDF. We would be happy to include these illustrations and derivations for Reflection HGFs as additional illustrations in the supplementary material of the camera-ready version. Finally, we chose Oscillation HGFs for our benchmarks because they are based on harmonic oscillators and dynamics come with beneficial properties. For example, harmonic oscillators are natural dynamics to consider under a normal approximation of our data as they correspond to Hamiltonian dynamics under this approximation. Further, they appeal due to their simplicity. We also addressed the motivation for Oscillation HGFs in more detail in our general response.\n\n> Authors mentioned conservation properties in the context of physical simulations. Do you think that they will still be beneficial in the general case, e.g. image/video generation? Even in physical systems, if the energy is not conserved, would those properties be a limiting factor somehow?\n\n**Physical simulations and conservation properties:** In physical systems, enforcing conservation properties serves as an inductive bias, similar to equivariance, and such biases have been observed to improve training accuracy. We anticipate that the same would be true of energy conservation. Conservation properties in the context of non-physical applications would also be beneficial in our opinion. As pointed out in the previous paragraph, Oscillation HGFs correspond to Hamiltonian dynamics under a normal approximation of the data. The Hamiltonian in this case is given by a sum of the squared norm of the velocity and the squared norm of the data point. This corresponds to the squared norm of the joint vector (x,v) in phase space and preserving this means that the dynamics always have constant scale. This ensures training stability. In fact, for our training we did not employ any rescaling of inputs/outputs, skip connections, or other preconditioning done to make training more stable for diffusion models (see e.g. the EDM paper). Oscillation HGFs achieved high performance with very little fine-tuning. We consider this as evidence that the conservation properties of the dynamics lead to better training dynamics.\n\n> What is x, and what is v?\n\n**What is x, and what is v?** Next, we address your question about what “x” and “v” stand for in the image generation case. For image generation, x is the image (i.e. a data point) and v is an auxiliary variable (initialized with a normal distribution). We will make sure to highlight this in a camera-ready version.\n\n**Computational details (runtime / memory):** The computational cost of Hamiltonian Generative Flows is mainly influenced by two components: (1) The cost for simulating the trajectory defined by a force field and (2) the cost of evaluating the velocity predictor. The input and output shape of the velocity predictor are the same as the denoiser or score network for diffusion. Therefore, they share the same computational cost as diffusion models and scale with the size of the network. The cost for simulating the force field depends on the choice of the force field: (A) For a trained force field (see section on Hamiltonian Score Matching), one has to backpropagate through the trajectory. This can be done with constant in memory using common techniques for Neural ODEs. In our experiments, we usually needed to simulate for ~5 time steps (so this is associated with 5 evaluations of the force field). (B) For a fixed force field, the cost is the one for simulating the ODE. In physical applications with known force fields, one could also use ODE solvers and the cost is application-specific. (C) In many cases (Oscillation HGFs, Reflection HGFs, diffusion models, flow matching), one can compute these in a simulation-free manner, so there is negligible additional cost. \n\nTogether with our general response, we hope that the above addresses all of your comments. We welcome any additional discussions. Thank you again for your helpful feedback and response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fFQ8viHHoT", "reviewer": "Reviewer_WwWf", "summary": "In this work, the authors proposed a new generative modeling approach called Hamiltonian Score Matching, which is motivated by classical Hamiltonian mechanics. This approach estimates score functions by augmenting data via Hamiltonian trajectories, and further motivates Hamiltonian Generative Flows. The authors also discuss the design space of force fields in Hamiltonian Generative Flows and connect it with harmonic oscillators. Experiments are conducted to verify the effectiveness of the proposed approach.", "questions": "See the above section.", "limitations": "The authors discussed the limitations of this work in the Conclusion section.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The connection between Hamiltonian dynamics, force fields, and score matching is interesting and provides a new perspective for the design space of generative models.\n\n2. The paper is well-written and easy to follow.\n\n3. The authors provide a clear proof section to check the correctness of propositions and theorems.", "weaknesses": "1. Regarding the methodology: The authors spend efforts to establish Hamiltonian Generative Flows from Parametrized Hamiltonian ODEs, Hamiltonian velocity predictor, and Theorem 1 for Hamiltonian Score Matching.   Although the benefits such as bringing additional freedoms for force fields and prediction objectives, the scalability of this approach is restricted as stated by the authors. The necessary trajectory simulation and min-max optimization largely limit the usage of the proposed approach. In Section 7, the authors instead seek to use a pre-defined force field instead of parametrized models, which makes the HGFs degenerate to pre-defined ODEs, further indicating the theory-practice gap of the proposed approach.\n\n2. Regarding the experiments: scales and types of experiments in this work are rather limited. In addition to the simulation experiment, the authors only conducted experiments on CIFAR-10, which even shows that the proposed HGFs cannot bring explicit benefits compared to previous approaches. It would largely improve the quality of this work if the authors could provide more diverse and strong experimental results for the proposed approach.\n\nOverall, I think the paper provides an interesting perspective, but the current issues discount the ratings of my evaluation. Currently, I vote for borderline acceptance and I will carefully read the rebuttal and other reviews to decide whether to increase or decrease my scores.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your insightful comments and for taking the time to review our work. We decided to address your questions regarding our methodology and our experimental results in the general response. We provide additional information here.\n\n**Theory vs practice:** You correctly point out that there is a trade-off between “theory and practice”. We consider elucidating this trade-off as one of the main contributions of our work. We showcase that score matching corresponds to “perfect” energy conservation but leads to a “harder” training objectives, while violating energy conservation allows us to make the algorithms scalable under the conditions we outlined. We introduced the Hamiltonian Score Matching (HSM) method first because by theorem 1, it is the “ideal” limit case one would like to learn. Despite a min-max objective, HSM provides a novel way of learning the score of a data distribution without adding noise (denoising score matching) or requiring to estimate the trace of the Hessian (implicit score matching). In Fig (2) in the attached PDF, we illustrate that energy-based models can be trained with this objective and lead to high-quality samples. However, we agree with your assessment that the proposed min-max optimization required for HSM might be an issue for training stability. For this reason, we then introduced Hamiltonian Generative Flows (HGFs) to be a scalable alternative but one that requires us to take into account that our dynamics do not necessarily conserve the energy. This, as you point out, leads us to use pre-defined force fields. The design space of pre-defined force fields is already huge and choosing the right force field is equally informed by the ideal “limit” of Hamiltonian Score Matching. For example, Oscillation HGFs correspond to Hamiltonian dynamics under a normal approximation of the data (see general response for details). There are many other interesting examples such as Reflection HGFs (see paragraph below).\n\n**Extended design space - Reflection HGFs:** To further illustrate the power of the HGF framework and that pre-defined ODEs can lead to novel interesting models, we developed “Reflection HGFs” here and plot the result of training them in figure 3 and figure 4 in the attached PDF. Let us assume that we a priori approximate our data as a uniform distribution over its support. Aiming for energy conservation, we consider the force field associated with a uniform distribution in [0,1]. This corresponds to zero force in (0,1) but a reflection (i.e. flip of the sign of the velocity) on the boundaries of the domain. A reflection can be considered an infinite force at the boundary (one can also use Neumann boundary conditions to make this precise and not an asymptotic limit). If we simulate a particle with a normally distributed velocity and let it bounce around for a long time, its location will eventually become approximately uniformly distributed. Therefore, starting with an initial distribution “p_data x N(0,I)”, we end up with a distribution of locations at time T that is uniform if T is large enough. Therefore, condition (C2) in the definition of HGFs is satisfied and condition (C1) is also satisfied as we can compute loss in a simulation-free way. Learning the Hamiltonian Velocity Predictors and running the backwards ODE allows us to generate data. We trained such a model on a simple toy distribution and illustrated it in figure 3 and figure 4 in the attached PDF. Such a model is inspired from the ideas of energy conservation developed in our work, and we hope, illustrates that HGFs are different from various previous models and not restricted to only Oscillation HGFs, diffusion models, or flow matching.\n\n**Performance comparison to EDM:** We addressed your question on this topic in our general response. \nAs a brief summary, we believe that EDM has better performance because it could build on previous works optimizing diffusion models and the compute budget used to optimize EDM was orders of magnitude larger, reducing the FID of diffusion models on CIFAR10 from >3.0 (higher than our score) to 1.97 (slightly below our score). In fact, such an extensive optimization of the design space of diffusion models was the goal of the EDM paper, while our goal was to introduce a new framework and method. We anticipate that with a similar optimization in the future, Oscillation HGFs will show a similar performance boost (see also fig 1 in attached PDF for a visual comparison of both models). For further explanation, we refer to the general response.\n\n**Experiments and choice of datasets:** We would like to draw attention to the fact that we not only conducted experiments on CIFAR-10 32x32, as suggested, but also on the FFHQ 64x64 dataset (see figure 4 in our paper). We chose CIFAR10 because of its use as a standard benchmark on image generation. To test our model on a higher resolution and more realistic dataset, we chose the FFHQ 64x64 dataset. The reason for this choice was that it provides higher resolution images at a dataset size that is possible to still benchmark on a compute budget available in academia. For example, ImageNet training would take 32 days on 8 A100 GPUs (this is the training time for the EDM model that was used). We also agree with your assessment that experiments on a physical dataset might have been advantageous. In particular, we considered molecular machine learning benchmarks such as ligand docking or protein design. Unfortunately, these modalities often require data to lie on manifolds (usually SE(3)) and require models to be designed with equivariance as an implicit bias. Designing generative models for this task are usually separate works as they require extensive design adaptations and ablations. We considered this as out of scope for this work.\n\nTogether with our general response, we hope that the above addresses all of your comments. We welcome any additional discussions. Thank you again for your helpful feedback and response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IRVDVnYGyW", "reviewer": "Reviewer_8ET8", "summary": "The authors introduce Hamiltonian velocity predictors (HVPs) as a tool for score matching and generative models. They present two innovations constructed with HVPs: Hamiltonian Score Matching (HSM), a novel generative model that encompasses diffusion models, and flow matching as HGFs with zero force fields. They showcase the extended design space of force fields by introducing Oscillation HGFs, a generative model inspired by harmonic oscillators. They show experiments validating the theoretical insights about HSM.", "questions": "See weakness.", "limitations": "The authors did not provide a clear discussion of the limitations of the proposed method. This should be included in the paper.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents an interesting generative model by leveraging  Hamiltonian velocity predictors. The authors give a theoretical analysis of the proposed method and its connections with existing methods. The authors also provide an experimental study to support the proposed method.", "weaknesses": "a. Why does it produce inferior performance on image generation compared to EDM?  The author could present the reasons for this. \n\nb. The authors also need to present the computation cost/efficiency comparison of different methods to validate the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your insightful comments and for taking the time to review our work. We are pleased to read that you consider HGFs an “interesting generative model” and address your questions and comments below.\n\n**Explanation for EDM vs Oscillation HGFs performance:** We addressed your question on this topic in our general response. As a brief summary, we believe that EDM has better performance because it could built on previous works optimizing diffusion models and the compute budget used to optimize EDM was orders of magnitude larger, reducing the FID of diffusion models on CIFAR10 from >3.0 (higher than our score) to 1.97 (slightly below our score). In fact, such an extensive optimization of the design space of diffusion models was the goal of the EDM paper, while our goal was to introduce a new framework and method. We anticipate that with a similar optimization in the future, Oscillation HGFs will show a similar performance boost (see also fig 1 in attached PDF for a visual comparison of both models). For further explanation, we refer to the general response.\n\n**Computational details (runtime / memory):** The computational cost of Hamiltonian Generative Flows is mainly influenced by two components: (1) The cost for simulating the trajectory defined by a force field and (2) evaluating the velocity predictor. The input and output shape of the velocity predictor are the same as the denoiser or score network for diffusion. Therefore, they share the same computational cost as diffusion models and scale with the size of the network. The cost for simulating the force field depends on the choice of the force field: (A) For a trained force field (see section on Hamiltonian Score Matching), one has to backpropagate through the trajectory. In our experiments, we usually needed to simulate for ~5 time steps (so this is associated with a 5x increase in cost). (B) For a fixed force field, the cost is the one for simulating the ODE. In physical applications with known force fields, one could also use ODE solvers and the cost is application-specific. (C) In many cases (Oscillation HGFs, Reflection HGFs, diffusion models, flow matching), one can compute these in a simulation-free manner, so there is negligible additional cost and the only cost consists of forward pass through the velocity predictor.\n\n**Limitations of HGFs:** We are happy to elaborate more on the limitations of our method. We will include a discussion of the above limitations in the camera-ready version. The limitations of the method proposed in our framework are - depending on the use case - as follows:\n- **Hamiltonian Score Discrepancy:** Here, limitations are the discretization error of the ODE and the simulation cost (to compute the discrepancy score, no backpropagation through the force network is needed).\n- **Hamiltonian Score Matching via joint training:** Training through a simulated ODE requires optimizing a Neural ODE. In our experiments, this was not a big problem but generally speaking led to around 5x increase of computation cost (i.e. around 5 forward passes through the network per training evaluation).\n- **Hamiltonian Generative Flows as a generative model:** To train HGFs, the final distribution might be not known (see condition (C2)). Therefore, while the learnt ODE is always “correct”, the initial distribution, which we start our sampling process with, might not approximate the actual distribution. This error might propagate forward to samples from an unrealistic distribution. However, in many cases, one knows the distribution after a certain time up to a negligible error. To illustrate this, we developed and trained Reflection HGFs. The idea of the model is that particles can move freely in a box without collision with walls (“very strong forces”) at the boundaries of the data domain making the particles bounce back (this can be made rigorous with von Neumann boundary conditions). With normally distributed velocities, the distribution of particles will converge towards a uniform distribution. Therefore, even if we cannot compute the final distribution analytically, we often might still know it approximately.\n\nTogether with our general response, we hope that the above addresses all of your comments. We welcome any additional discussions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tgC2LMuunM", "reviewer": "Reviewer_XPUP", "summary": "The paper proposes the Hamiltonian Score Matching framework, which is a new general framework of generative models. Inspired by the Hamiltonian dynamics in classical and statistical mechanics, the framework uses the Hamiltonian dynamics to generate data, which is also called Hamiltonian generative flow in the paper. The paper theoretically proves that the force field coincides with the score function if and only if its optimal velocity predictor is zero and proposes the HSM loss function. The paper also shows that the Flow Matching framework and the diffusion model are special cases of the HSM framework. Additionally, a new generative flow called Oscillation Hamiltonian generative flow is constructed. Experiments on several datasets show the experimental performance of the HSM framework.", "questions": "What is the motivation for designing Oscillation HGFs? I'm curious about how to design a force field in the HSM framework.", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Very clear writing.\n2. The method is novel and easy to understand.\n3. The paper clarifies the connection and difference with previous works.", "weaknesses": "1. The experimental result is not so good. As the paper writes, the Oscillations HGFs can surpass most baselines but still lack behind the EDM model. I think more experimental results should be provided to show that the HSM framework is a good framework for generative models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful comments and for taking the time to review our work.\n\n> What is the motivation for designing Oscillation HGFs?\n\n**Motivation for designing Oscillation HGFs and other force fields:** As you point out, the choice of a force field is an important design parameter of Oscillation HGFs. We addressed the reason to choose harmonic oscillators in the general response in detail. As a brief summary: if you assume that your data distribution is normal (as a first order approximation) and if you define the Hamiltonian as in our paper (equation (1)), then the corresponding Hamiltonian force field is F(x)=-x and the Hamiltonian dynamics are harmonic oscillators. Hence, conservation of energy and an a priori approximation of our data as a normal distribution naturally leads us to using harmonic oscillators. However, one can also consider force fields of other shapes. To illustrate this, a simple force field is the “reflection force field”, i.e. the force field that just reflects a particle to stay within an interval [a,b] (usually the domain of the data). This leads to Reflection HGFs, which are explained in more detail in the general response. To illustrate this, we trained these models and illustrated them in figure 3 and 4 in the attached PDF. We will include these illustrations and derivations for Reflection HGFs as additional illustrations for the power of HGFs in the supplementary material of the camera-ready version.\n\n> I'm curious about how to design a force field in the HSM framework.\n\n**Designing a force field in the HSM framework:** Generally speaking, we suggest the following decision tree to design a force field:\n- Trainable force field: If one decides to train the force field jointly with the velocity predictor (accepting increased computational cost), then one can parameterize the force field as a neural network and train it with the proposed procedures (we have also included experiments on this in our general rebuttal and in figure 2 in the attached PDF).\n- Fixed/designed force field: Otherwise, one should explicitly design a force field.\n\t- A. Application-specific force field: If one focuses on applications in biology/chemistry/physics, one should consider a force field relevant for this application. It is important to note that data in these fields (such as in protein folding) often lies on manifolds and future work is required to adapt HGFs to geometric data.\n\t- B. Data distribution: If an application has no a priori force field, one should aim for a force field that approximately preserves the energy of the system. If we a priori approximate our data distribution with a normal distribution (only for the purposes of designing the force field), the corresponding Hamiltonian dynamics become harmonic oscillators. This motivated us to use Oscillation HGFs (this is explained in more detail in the general response). However, other data types might have other distributions that approximate them and the respective Hamiltonian dynamics should be considered then - such as Reflection HGFs.\n\n**Lacking behind the EDM model and experimental results:** We hope to have addressed your question on this topic in our general response in detail. As a brief summary, we believe that EDM has better performance because the compute budget used to optimize EDM was extensive, reducing the FID of diffusion models on CIFAR10 from >3.0 (higher than our score) to 1.97 (slightly below our score). In fact, such an extensive optimization of the design space of diffusion models was the goal of the EDM paper, while our goal was to introduce a new framework and method. We anticipate that with a similar optimization in the future, Oscillation HGFs will show a similar performance boost (see also fig 1 in attached PDF for a visual comparison of both models). We welcome additional discussions, if needed.\n\nDo you believe our answers and additional experiments have addressed your concerns? We welcome any additional discussions. Thank you again for taking the time to review our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "liugJ3RqIf", "reviewer": "Reviewer_6n8r", "summary": "This paper proposes a new matching method based on Hamiltonian mechanics. The proposed method, algorithmically, is essentially a second-order ODE in which the acceleration (i.e., the drift of the velocity channel) is approximated by deep neural networks. The method is well grounded on the theory of classical Hamiltonian mechanics and statistical physics. As a higher-order system, the proposed method naturally absorbs prior diffusion and flow methods as special cases. Experiments were conducted on a synthetic dataset (Gaussian mixtures) and image generation.", "questions": "- Can the authors comment on the differences compared to AGM? I think AGM also constructs simulation-free (xt, vt) samples then matches the drift of the velocity.\n\n- Have the author ever tried to jointly train both $\\theta$ and $\\phi$, as mentioned in L179? I understand this may most likely be impractical but just curious from an exploratory standpoint.\n\n- Could the authors elaborate more on L175 the \"optimal\" choice of T, and the first hypothesis in L184?", "limitations": "Limitations were mentioned in Sec 10.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Presentation is overall clear and easy to follow. I like how the authors highlight important sentences for improving readability. Figs 1 & 2 are also instructive in showcasing the advantage of their method.\n\n- Theoretical contributions are solid. I think many of their results in Sec 4 may as well be of interested for readers from other domains.", "weaknesses": "- It's unfortunate that, from an algorithmic standpoint, the resulting algorithm---after all these nice theoretical results---remains somewhat similar to a few recently-proposed second-order methods, e.g., AGM (https://arxiv.org/pdf/2310.07805 ICLR'24 oral) or CLD (ref. [12]). \n\n- The use of harmonic oscillator in Sec 7 is not very motivated in my opinion. There could be other alternatives that also fulfill C1 & C2 conditions. Practically speaking, sampling (x,v) in (23) from sin/cos doesn't seem to encourage straighter generative process.  \n\n- (minor) Typo in L236, I think you meant C2 not C1.\n\nOverall, I have a mixed feeling on this paper --- on one side, I do enjoy, and acknowledge, their theoretical contributions, yet on another side, I feel like the resulting algorithm is rather weak.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful comments and for taking the time to review our work. We address your questions and comments below.\n\n> Can the authors comment on the differences compared to AGM? \n\n**Comparison with Acceleration Generative Model (AGM) model**: Thank you for pointing out the AGM paper, which was published at ICLR 2024, 1-2 months before our own submission. We will make sure to discuss it in the camera-ready version. As you point out, both AGM and our own work - as well as the mentioned critically-damped Langevin diffusion work - use constructions in phase space (joint position and velocity space). Further, they use 2nd order ODEs for their constructions, as we do. Finally, all take inspiration from concepts used in physics. However, there are important differences in the respective frameworks:\n- **Force field vs optimal velocity predictor:** AGM focuses on learning the force field (equation (10) in AGM paper). In contrast, our approach primarily focuses on learning the optimal velocity predictor. While we also consider optimizing the force field by minimizing the norm of the optimal velocity predictor, this happens in the “outer loop” of the maximization - the inner loop optimizes the optimal velocity predictor.\n- **Scope and task:** ATM focuses on bridging two desired distributions. They pose a stochastic bridge problem in phase space (equation (5) in AGM paper) that, in short, searches for the force field that optimally bridges two distributions - where optimally is framed as bridging with minimal acceleration, kinetic energy, and potential energy (weighted respectively). We do not consider the problem of bridging distributions. To illustrate the differences, we included a description of Reflection HGFs below that illustrate a model that shows the difference in the design to AGMs further.\n- **Performance:** These methodological differences are also reflected on the experimental benchmarks as our model significantly outperforms the AGM model. While AGM achieves an FID of 2.46 on unconditional CIFAR-10 generation, we achieve an FID of 2.12 with 40% less NFEs (35 NFEs vs 50 NFEs) (see table 3 of their paper).\n- **Connection to Hamiltonian physics and score metric:** The AGM framework considers dynamics changes and learns the forces that obtain desired dynamic changes. In contrast, our framework centers around energy preservation and divergence from that preservation (for optimal velocity predictors that are not zero). Specifically, we establish a connection to Hamiltonian physics and a property of the preservation of energy. This allows us to introduce a further bi-level optimization and the possibility of joint training for score matching. “Imperfect” velocity predictors allow us to build Hamiltonian Generative Flows (HGFs). We do not find such contributions to be reflected in the AGM work.\nYou also mentioned the critically-damped Langevin diffusion (CLD) work that has also been discussed in our submission. The CLD paper considers a single force field and focuses on the stochastic part (choses “optimal” amount of stochasticity to be critically damped). This is also qualitatively different from our own work. Therefore, we would make the claim that AGM and CLD are different frameworks from HGFs, while showing certain similarities.\n\n> Have the author ever tried to jointly train both 𝜃 and 𝜙, as mentioned in L179?\n\n**Joint training of 𝜃 and 𝜙:** To address your comment, we have run more experiments on this and trained an energy-based model via join training of the force field (“𝜃”) and the velocity predictor (“𝜙”). As we believe your comment was of general interest, we addressed your question on this topic in our general response and with figures in the attached PDF. We welcome additional discussions.\n\n> Could the authors elaborate more on [...] the first hypothesis in L184?\n\n**Hypothesis in L184:** In traditional score matching, the score network only ever sees points from the dataset (i.e. these are the only possible inputs). This leads to unstable training (see e.g. Kingma and LeCun (2014)). In contrast, Hamiltonian Score Matching (HSM) simulates Hamiltonian trajectories with the current estimate of the score network traversing space. Effectively, the simulation of Hamiltonian trajectories augments our data. This is similar to denoising score matching (DSM), which \"augments\" the data by adding noise. However, DSM does not learn the score of the original data distribution but a noisy version therefore and exhibits high variance. HSM does learn the original distribution.\n\n> Could the authors elaborate more on [...] the \"optimal\" choice of T?\n\n**Optimal\" choice of T**: We happily elaborate on this comment. The Hamiltonian Score Discrepancy depends on the choice of a time constant T>0. The role of T is to measure the discrepancy of energy preservation in the time interval [0,T). By theorem 1, T can be chosen arbitrarily as long as T>0. Naturally, the larger T, the more discretization error we accumulate by integrating the ODE. Therefore, without proposition 2, one might suggest that to measure the Hamiltonian score discrepancy one should choose T to be very small. However, proposition 2 establishes that there is a tradeoff between ODE integration error and the “signal-to-noise ratio”: it shows that the value we estimate is approximated by a parabola. This value goes to zero fast for t->0. Therefore, choosing T>0 to be very small will lead to very small values that we aim to estimate. Our signal-to-noise ratio will be detrimental. Hence, proposition 1 indicates that there is an optimal trade-off in choosing T to be high enough to give enough signal but low enough to lead to negligible ODE discretization error.\n\nDo you believe our answers and additional experiments have addressed your concerns? We welcome any additional discussions. Thank you again for taking the time to review our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for their constructive and positive feedback. We are pleased to see that our work is considered by reviewers as a “very solid theoretical contribution” introducing an “interesting generative model leveraging Hamiltonian velocity predictors”. Below, we’ve compiled rebuttal points asked by several reviewers while we address individual questions in the individual rebuttals. We welcome any additional discussions.

**Motivation for Oscillation HGFs**: One of the main instantiations of Hamiltonian Generative Flows (HGFs) that we study empirically are Oscillation HGFs (see section 7), an instantiation of HGFs relying on harmonic oscillators. Multiple reviewers asked us to elaborate more on the motivation to study Oscillation HGFs. We studied them for several reasons:
- **Oscillation HGFs are Hamiltonian dynamics under a normal approximation of data:** If you assume that the data distribution is Gaussian N(0,1) (as a first order approximation) and if you define the Hamiltonian as in our paper (equation (1)), then the corresponding Hamiltonian force field is F(x)=-x and the Hamiltonian dynamics are harmonic oscillators. Hence, conservation of energy and an a priori approximation of our data as a normal distribution naturally leads us to using harmonic oscillators. (The normal approximation is only used for the purposes of defining the force field and does not restrict the actual generated distribution to be normal).
- **Simplicity and analytical tractability:** The harmonic oscillator is one of the simplest linear systems with closed-form solutions, necessary to train the model “simulation-free”. The only simpler force field would be the zero force field, which, as shown, corresponds to diffusion models.
- **Training stability:** All n-order derivatives of the harmonic oscillator have constant expected norm (eq. (25)). We hypothesize that this leads to more stable training. In fact, for our training we did not employ any rescaling of inputs/outputs, skip connections, or other preconditioning. Oscillation HGFs achieved high performance with very little fine-tuning.

**Comparison to EDM paper:** Multiple reviewers asked for a comparison with the EDM model and for the reasons why EDM has slightly better performance at this point in time:
- **Scope of work:** We consider our work as introducing a novel meta-algorithm and the experiments serve as an illustration of its power. The EDM network is a diffusion model and therefore, a specific instance of HGFs. We believe that our framework offers potential to find new well-performing methods within the framework. For example, Oscillation HGFs gave a model that - almost out of the box - performs almost on par with EDM.
- **Hyperparameter tuning:** The EDM paper used an extensive compute budget to tune a variety of hyperparameters such as noise schedule, architectures, noise distributions during training, loss weightings, network preconditioning, learning hyperparameters, and many more - all optimized for CIFAR10. In fact, this was the goal of their EDM, while our paper has a methodological goal. This reduced their FID score on CIFAR-10 from >3.0 (a score higher than ours) to 1.98.  We anticipate that with similar optimization as done in EDM, Oscillation HGFs might perform even better.
- **Methodological comparison:** Another question was whether EDM paths are straighter/smoother than Oscillation HGF paths. We believe that this question was inspired by the visualization in figure 3 in the EDM paper. To address this question, we replicated this visualization for Oscillation HGFs (see figure 1 in the attached PDF). As one can see, EDM paths have stronger curvature at the beginning, while Oscillation HGFs have stronger curvature mid-way (0.4<t<0.8). Hence, it is unclear whether this provides a meaningful difference.  Please note that the results of these visualizations might not correspond to sampling performance as they do not depict the marginal “learnt” velocity predictor but only conditioned on a few data points.

**Joint training of 𝜃 and 𝜙:** Multiple reviewers asked for the possibility of training an energy-based model via joint training of the velocity predictor and the force field at the same time (we had shown how to learn a score network in figure 1 in our paper already). This corresponds to training an energy-based model via Hamiltonian score matching and sampling via Langevin dynamics or Hamiltonian Monte Carlo. To address this comment, we have trained an energy-based model on a 2d Gaussian mixture with Hamiltonian Score Matching and sampled from it. In figure 2 in the attached PDF, one can see that the generated samples closely match the desired distribution. As mentioned in the paper and as reviewer 6n8r points out, such a model is more of theoretical interest because of the required min-max optimization.

**Reflection HGFs:** Several reviewers indicated that they would be interested in whether HGFs can give us models other than Oscillation HGFs. To address this, we illustrate this possibility by introducing “Reflection HGFs” here and plot the result of training them in figure 3 and figure 4 in the attached PDF. The idea of the model is that particles can move freely in a box without collision with walls (“very strong forces”) at the boundaries of the data domain making the particles bounce back (this can be made rigorous with von Neumann boundary conditions). With normally distributed velocities, the distribution of particles will converge towards a uniform distribution. Further, this model can be trained in a simulation-free manner. We trained this HGF model on a simple toy distribution (see fig 3 and fig 4 in the attached PDF). Such a HGF model is distinct from previous models and, we hope, illustrates that HGFs are not restricted to Oscillation HGFs, diffusion models, or flow matching.

We thank the reviewers again for their helpful comments. We welcome any additional discussions.

Meta-review: This work develops methods for generative modeling based on Hamiltonian dynamics. In this setup the authors combine a learned force field (which approximate the score function of the data distribution) with an auxiliary velocity variable, for which they define a learned Hamiltonian velocity predictor (HVP). The HVP can be used to define a Hamiltonian score matching method, which then leads to the definition of Hamiltonian Generative Flows (HGFs), a class of generative models that recover diffusion models and flow matching as special cases (with zero force field).

Reviewers are in agreement that this is a clearly paper presents an interesting new Hamiltonian perspective on generative modeling. Reviewer do note some weaknesses in experiments, notably the slightly poor performance relative to EDM, though reviewers do not appear to find this a key consideration. Overall, this is a submission that is above the bar for acceptance. In the camera ready, do please make sure to discuss the connections with the AGM method as pointed out by reviewer 6n8r.
Decision: Accept (poster)