Original: Particle Dynamics for Learning EBMs
Revised: Particle Dynamics for Learning EBMs

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Energy-based modeling is a promising approach to unsupervised learning, whichyields many downstream applications from a single model.
Revised: Energy-based modeling is a promising approach to unsupervised learning, which yields many downstream applications from a single model.

Original: The main difﬁculty inlearning energy-based models with the “contrastive approaches” is the generationof samples from the current energy function at each iteration.
Revised: The main difﬁculty in learning energy-based models with the “contrastive approaches” is the generation of samples from the current energy function at each iteration.

Original: Many advances havebeen made to accomplish this subroutine cheaply.
Revised: Many advances have been made to accomplish this subroutine cheaply.

Original: Nevertheless, all such samplingparadigms run MCMC targeting the current model, which requires inﬁnitely longchains to generate samples from the true energy distribution and is problematic inpractice.
Revised: Nevertheless, all such sampling paradigms run MCMC targeting the current model, which requires inﬁnitely long chains to generate samples from the true energy distribution and is problematic in practice.

Original: In this paper, we propose an alternative approach to getting these samplesavoiding crude MCMC sampling from the current model.
Revised: This paper proposes an alternative approach to getting these samples and avoiding crude MCMC sampling from the current model.

Original: We accomplish thisby viewing the evolution of the modeling distribution as (i) the evolution of theenergy function, and (ii) the evolution of the samples from this distribution alongsome vector ﬁeld.
Revised: We accomplish this by viewing the evolution of the modeling distribution as (i) the evolution of the energy function, and (ii) the evolution of the samples from this distribution along some vector ﬁeld.

Original: We subsequently derive this time-dependent vector ﬁeld suchthat the particles following this ﬁeld are approximately distributed as the currentmodel thereby matching the evolution of the particles with the evolution of theenergy function prescribed by the learning procedure.
Revised: We subsequently derive this time-dependent vector ﬁeld such that the particles following this ﬁeld are approximately distributed as the current density model. Thereby we match the evolution of the particles with the evolution of the energy function prescribed by the learning procedure.

Original: Importantly, unlike Monte
Revised: Importantly, unlike Monte

Original: Carlo sampling, our method targets to match the current distribution in a ﬁnite time.
Revised: Carlo sampling, our method targets to match the current distribution in a ﬁnite time.

Original: Finally, we demonstrate its effectiveness empirically comparing to MCMC-basedlearning methods.
Revised: Finally, we demonstrate its effectiveness empirically comparing to MCMC-based learning methods.

Original:  Introduction
Revised:  Introduction

Original: Energy-based modeling has recommended itself as a universal approach learning a single model,which then can be applied in various scenarios: continual learning, missing data imputation, outof-distribution detection, better uncertainty of discriminative models (Grathwohl et al., 2019; Du& Mordatch, 2019; Li et al., 2020).
Revised: Energy-based modeling has recommended itself as a universal approach learning a single model, which then can be applied in various scenarios: continual learning, missing data imputation, outof-distribution detection, better uncertainty of discriminative models (Grathwohl et al., 2019; Du & Mordatch, 2019; Li et al., 2020).

Original: However, scaling this approach to real-world data such asimages encounters many complications, which the community has been approaching  trying to getbetter samples (Tieleman & Hinton, 2009; Du & Mordatch, 2019; Nijkamp et al., 2019) or targetingdifferent objectives (Grathwohl et al., 2020; Arbel et al., 2020; Gao et al., 2020).
Revised: However, scaling this approach to real-world data such as images encounters many complications, which the community has been approaching by trying to get better samples (Tieleman & Hinton, 2009; Du & Mordatch, 2019; Nijkamp et al., 2019) or by targeting different objectives (Grathwohl et al., 2020; Arbel et al., 2020; Gao et al., 2020).

Original: In this paper we approach the subroutine problem of getting samples from the current model,which arises in the learning of the energy-based models.
Revised: In this paper, we approach the subroutine problem of getting samples from the current model, which arises in the learning of the energy-based models.

Original: The conventional approach to this is torun an MCMC method targeting the current model.
Revised: The conventional approach to this is to run an MCMC method targeting the current model.

Original: Instead, we update particles deterministicallypropagating them along the derived vector ﬁeld such that after time dt the particles are distributed asthe evolved density after time dt .
Revised: Instead, we update particles deterministically propagating them along the derived vector ﬁeld such that after time dt the particles are distributed as the evolved density after time dt .

Original: This is principally different, since we don’t rely on the invarianceof the target (the current model density), and are able to match it in a ﬁnite amount of time.
Revised: This is principally different, since we don’t rely on the convergence to the target (the current model density), and are able to match it in a ﬁnite amount of time.

Original: Our maincontribution is the formula (8) for the vector ﬁeld, which matches the evolution of the model with theevolution of the particles in the space of log-densities.
Revised: Our main contribution is the formula (8) for the vector ﬁeld, which matches the evolution of the model with the evolution of the particles in the space of log-densities.

Original: Further, we discuss possible ways to simulatethis formula and demonstrate its usefulness empirically.
Revised: Further, we discuss possible ways to simulate this formula and demonstrate its usefulness empirically.

Original:  Background and Related Works
Revised:  Background and Related Works

Original: Energy-Based Models are usually learned via the maximum likelihood principle.
Revised: Energy-Based Models are usually learned via the maximum likelihood principle.

Original: That is, we startwith a model density function q ( x ) parameterized by the energy function E ( x, θ ) :
Revised: That is, we start with a model density function q ( x ) parameterized by the energy function E ( x, θ ) :

Original: [Equation]
Revised: [Equation]

Original: which is then optimized to approximate some target density p ( x ) (given empirically: as a set ofsamples).
Revised: which is then optimized to approximate some target density p ( x ) given empirically (as a set of samples).

Original: This can be done by the maximization of E p log q , or, equivalently, minimization of
Revised: This can be done by the maximization of E p log q , or, equivalently, minimization of

Original: KL( p, q ) via the gradient-descent methods:
Revised: KL( p, q ) via the gradient methods:

Original: [Equation]
Revised: [Equation]

Original: The main obstacle under this approach is the sampling from the current density q θ ∝ exp( − E ( x, θ )) .
Revised: The main obstacle under this approach is the sampling from the current density q θ ∝ exp( − E ( x, θ )) .

Original: Our work operates much in the fashion of the Persistent Contrastive Divergence (PCD)
Revised: Our work operates much in the fashion of the Persistent Contrastive Divergence (PCD)

Original: (Tieleman &
Revised: (Tieleman &

Original: Hinton, 2009): it keeps a set of samples, which are updated at every iterations to match current q θ .
Revised: Hinton, 2009). It keeps a set of samples, which are updated at every iterations to match current q θ .

Original: While PCD relies on MCMC methods targeting q θ , we propagate the particles deterministically alongthe derived vector ﬁeld.
Revised: While PCD relies on MCMC methods targeting q θ , we propagate the particles deterministically along the derived vector ﬁeld.

Original: Langevin Dynamics is a ubiquitous sampling method.
Revised: Langevin Dynamics is a ubiquitous sampling method.

Original: For energy-based models with the continuousstate-space, this method is especially attractive due to its cheap iterations (single gradient evaluationper step) and the ability to yield good samples even without Metropolis-Hastings correction (Gelfand& Mitter, 1991).
Revised: For energy-based models with the continuous state-space, this method is especially attractive due to its cheap iterations (single gradient evaluation per step) and the ability to yield good samples even without Metropolis-Hastings correction (Gelfand & Mitter, 1991).

Original: This procedure targeting the distribution p can be written as
Revised: This procedure targeting the density p can be written as

Original: [Equation]
Revised: [Equation]

Original: Its efﬁciency, however, is hindered by the random ﬂuctuations that introduce random-walk behaviour,and its deterministic analog is more efﬁcient (see, for instance, (Liu et al., 2019)).
Revised: Its efﬁciency, however, is hindered by the random ﬂuctuations that introduce random-walk behaviour, and its deterministic analog is more efﬁcient (see, for instance, (Liu et al., 2019)).

Original: This analog isderived by rewriting the Fokker-Planck equation (which describes the evolution of the density) as thecontinuity equation:
Revised: This analog is derived by rewriting the Fokker-Planck equation (which describes the evolution of the density) as the continuity equation:

Original: [Equation]
Revised: [Equation]

Original: Then the simulation of the particles can be done by propagating them along the new vector ﬁeld:
Revised: Then the simulation of the particles can be done by propagating them along the new vector ﬁeld:

Original: [Equation]
Revised: [Equation]

Original: In Monte Carlo setting, the deterministic simulation is troublesome since we don’t have an access tothe current density q t .
Revised: In Monte Carlo setting, the deterministic simulation is troublesome since we don’t have an access to the current density q t .

Original: However, we will see how the EBMs learning naturally allows for this.
Revised: However, we will see how the EBMs learning naturally allows for this.

Original:  Matching the particle dynamics with the energy evolution
Revised:  Matching the particle dynamics with the energy evolution

Original: The motivation of this section is to match two things: the update of the energy and the update ofthe particles.
Revised: In this section, we try to match two things: the update of the energy and the update of the particles.

Original: The former is deﬁned by the learning procedure maximizing the log-likelihood.
Revised: The former is deﬁned by the learning procedure maximizing the log-likelihood.

Original: Theparticles then should be moved to keep up with the updates of energy and be distributed as theup-to-date model.
Revised: The particles then should be propagated to keep up with the updates of energy and be distributed as the most recent model.

Original: We match these two dynamics by matching the updates of their log-densities in L 2 q :
Revised: We match these two dynamics by matching the updates of their log-densities in L 2 q :

Original: [Equation]
Revised: [Equation]

Original: where “ max · arg max ” denotes the scalar multiplication of the maximum and the maximizer, q t isthe prescribed evolution, and ˆ q t is the density evolution of particles deﬁned by the vector ﬁeld v , i.e.[Equation]
Revised: where “ max · arg max ” denotes the scalar multiplication of the maximum and the maximizer, q t is the prescribed evolution, and ˆ q t is the density evolution of particles deﬁned by the vector ﬁeld v , i.e.

Original: [Equation]
Revised: [Equation]

Original: Proposition 1.
Revised: Proposition 1.

Original: For the evolution of the density q t = exp( − E t ) /Z
Revised: For the evolution of the density q t = exp( − E t ) /Z

Original: t , the solution of equation (6) is
Revised: t , the solution of equation (6) is

Original: [Equation]
Revised: [Equation]

Original: (See proof in Appendix A).
Revised: (See proof in Appendix A).

Original: This formula is the main development of our work and in the next sectionwe discuss its practical implications.
Revised: This formula is the main development of our work and in the next section we discuss its practical implications.

Original: In a similar way, we can project the evolution of the density
Revised: In a similar way, we can project the evolution of the density

Original: [Equation]
Revised: [Equation]

Original: which is related to the gradient ﬂows in the Wasserstein Riemannian manifold (Otto, 2001; Benamou& Brenier, 2000).
Revised: which is related to the gradient ﬂows in the Wasserstein Riemannian manifold (Otto, 2001; Benamou & Brenier, 2000).

Original: These two vector ﬁelds are equivalent when the distribution follows the gradient ofsome functional F .
Revised: These two vector ﬁelds are equivalent when the distribution follows the gradient of some functional F .

Original: Proposition 2. Consider the functional F = (cid:82) f ( q ) , which we can optimize either w.r.t.
Revised: Proposition 2. Consider the functional F = (cid:82) f ( q ) , which we can optimize either w.r.t.

Original: q =exp( − E ) /Z
Revised: q = exp( − E ) /Z

Original: or w.r.t.
Revised: or w.r.t.

Original: E .
Revised: E .

Original: When the evolution of the density (energy) is deﬁned by the Frechetderivatve of F , we have v ∗ = v (cid:63) .(See
Revised: When the evolution of the density (energy) is deﬁned by the Frechet derivatve of F , we have v ∗ = v (cid:63) .

Original:  proof in Appendix B).
Revised: (See proof in Appendix B).

Original: This preposition gives us a reasonable result.
Revised: This preposition gives us a reasonable result.

Original: Namely, the dynamics ofthe particles is independent of the distribution parameterization when the parameterization is dense inthe corresponding spaces.
Revised: Namely, the dynamics of the particles is independent of the distribution parameterization when the parameterization is dense in the corresponding spaces.

Original: Another motivation for the derived formula is that it can be approximated by the Persistent Contrastive
Revised: Another motivation for the derived formula is that it can be approximated by the Persistent Contrastive

Original: Divergence with the Langevin dynamics.
Revised: Divergence with the Langevin dynamics.

Original: Proposition 3.
Revised: Proposition 3.

Original: The updates of the particles following v ∗ = −∇ ∂E ∂t can be approximated as
Revised: The updates of the particles following v ∗ = −∇ ∂E∂t can be approximated as

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: ) (See derivations in Appendix C).
Revised: (See derivations in Appendix C).

Original: In the following section, we will see that the derived formulav ∗ = − ∂E/∂t allows for different approximations, which avoid any stochasticity in the updates.
Revised: In the following section, we will see that the derived formula v ∗ = − ∂E/∂t allows for different approximations, which avoid any stochasticity in the updates.

Original:  Numeric approximations of the particle dynamics
Revised:  Numeric approximations of the particle dynamics

Original: First, we consider the approximations that follow straightforwardly by discretizingthe formula (8).
Revised: First, we consider the approximations that follow straightforwardly by discretizing formula (8).

Original: Discretizing the energy update, we have
Revised: Discretizing the energy update, we have

Original: [Equation]
Revised: [Equation]

Original: The update of each individual particle can be written as
Revised: The update of each individual particle can be written as

Original: [Equation]
Revised: [Equation]

Original: We denote this update rule Method α .
Revised: We denote this update rule Method α .

Original: This method is basically the simulation of the Langevindynamics, but in a deterministic way.
Revised: This method is basically the simulation of the Langevin dynamics, but in a deterministic way.

Original: Indeed, taking p ( · ) ∝ exp( − E ( · , θ ( t + dt )) in equation (5),we obtain the same formula up to the choice of the step-size.
Revised: Indeed, taking p ( · ) ∝ exp( − E ( · , θ ( t + dt )) in equation (5), we obtain the same formula up to the choice of the step-size.

Original: Intuitively, this procedure tries to matchthe new log-density following its gradient and, at the same time, unmatch the old log-density.
Revised: Intuitively, this procedure tries to match the new log-density following its gradient and, at the same time, unmatch the old log-density.

Revised: We describe the full training procedure in Algorithm 1.

Original: The second option, for the parametric models, is to discretize the time on a lower level:
Revised: The second option, for the parametric models, is to discretize the update of parameters instead:

Original: [Equation]
Revised: [Equation]

Original: This formula gives us another update rule, which we call Method β :
Revised: This formula gives us another update rule, which we call Method β :

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Unlike method α , this method is different from deterministic Langevin, and we will return to itsintuition in a bit.
Revised: Unlike method α , this method is different from deterministic Langevin, and we will return to its intuition in a bit.

Revised: For the full training procedure, see Algorithm 1.

Revised: Algorithm 1 Methods α, β

Revised: Require: samples from the target distribution p ( x ) get initial samples { x ( i ) 0 } ni =1 ∼ q θ (0) ( x ) ∝ exp( − E ( x, θ (0))) for t ∈ [0 , . .

Revised: , T ] do estimate −∇ θ KL( p, q θ ) =

Original:  The third option we consider is the non-parametric updates, which we derive approximating theenergy gradient in RKHS.
Revised: −∇ θ (cid:2) E x ∼ p E ( x, θ ) − E x ∼ q θ E ( x, θ ) (cid:3) update parameters θ ( t + dt ) = Optimizer (cid:2) θ ( t ) , −∇ θ KL( p, q θ ) (cid:3) update samples x ( i ) t + dt = x ( i ) t + dt · v α,β ( x ( i ) t ) (see the formulas for v α and v β in the text) end for return trained density model q θ ( T ) ( x ) ∝ exp( − E ( x, θ ( T ))) , ﬁnal set of samples { x ( i ) T } ni = The third option we consider is the non-parametric updates, which we derive approximating the energy gradient in RKHS H with kernel k .

Original: To minimize the KL-divergence we ﬁrst take the Frechet derivative w.r.t.the energy E along some direction h and use the fact that RKHS is actually dense in L 2 q (Duncanet al., 2019).
Revised: To minimize the KL-divergence we ﬁrst take the Frechet derivative w.r.t. the energy E along some direction h and use the fact that H is actually dense in L 2 q (Duncan et al., 2019).

Original: Then, using the reproducing property of k , we can formulate the directional derivativeas an action of a linear operator:
Revised: Then, using the reproducing property of k , we can formulate the directional derivative as an action of a linear operator:

Original: [Equation]
Revised: [Equation]

Original: D e n s i t y

Original: [-Target  
-]
Revised: [-Target  
-]

Original: P a r t i c

Original: l e s

Original: Anneal+RB

Original: PCD

Original: Alpha

Original: Beta Gamma Following (Gretton et al., 2012), we see that µ p =
Revised: Following (Gretton et al., 2012), we see that µ p =

Original: E x ∼ p k
Revised: E x ∼ p k

Original: ( x, · ) ∈ H if E
Revised: ( x, · ) ∈ H if E

Original: x ∼ p (cid:112) k
Revised: x ∼ p (cid:112) k

Original: ( x, x ) < ∞ .
Revised: ( x, x ) < ∞ .

Original: Hence,we can choose the direction h ∈ H matching the gradient.
Revised: Hence, we can choose the direction h ∈ H matching the gradient.

Original: The gradient then deﬁnes the vector ﬁeld, which we denote as Method γ :[Equation] [Figure]
Revised: The gradient then deﬁnes the vector ﬁeld, which we denote as Method γ :

Original: [Equation]
Revised: [Equation]

Original: Intuitively, the particles are attracted to the dataset and repelled from each other.
Revised: Intuitively, the particles are attracted to the dataset and repelled from each other.

Original: Also, this vectorﬁeld coincides with the MMD gradient ﬂow (Arbel et al., 2019), which is derived from a differentperspective.
Revised: Also, this vector ﬁeld coincides with the MMD gradient ﬂow (Arbel et al., 2019), which is derived from a different perspective.

Original: Proposition 4.
Revised: Proposition 4.

Original: The convergence of the dynamics (16) is described as:
Revised: The convergence of the dynamics (16) is described as:

Original: [Equation]
Revised: [Equation]

Original: (17)108 (See proof in Appendix D).
Revised: (17) (See proof in Appendix D).

Original: Hence, the KL-divergence between the target and the current approximation reduces proportionally to the squared MMD between these distribution.
Revised: Hence, the KL-divergence between the target and the current approximation reduces proportionally to the squared MMD between these distribution.

Original: The process stops when
Revised: The process stops when

Original: MMD k ( p, q t ) = 0 .
Revised: MMD k ( p, q t ) = 0 .

Original: If the kernel is expressive enough (is universal), then q converges to p .
Revised: If the kernel is expressive enough (is universal), then q converges to p .

Original: We now return to the intuition of method β .
Revised: We now return to the intuition of method β .

Original: Taking the formula (14) and assuming that the updatesof the parameters follows the gradient descent, we have
Revised: Taking the formula (14) and assuming that the updates of the parameters follows the gradient descent, we have

Original: [Equation]
Revised: [Equation]

Original: Thus, we see, that method β is essentially method γ , but with the Neural Tangent Kernelk θ ( x, y )
Revised: Thus, we see, that method β is essentially method γ , but with the Neural Tangent Kernel k

Original:  =
Revised: θ ( x, y ) =

Original: (cid:104)∇ θ E ( x, θ ) , ∇ θ E ( y, θ ) (cid:105) .
Revised: (cid:104)∇ θ E ( x, θ ) , ∇ θ E ( y, θ ) (cid:105) .

Original: Hence, it operates by targeting the distribution of data ratherthan approximating the current energy model.
Revised: Hence, it operates by targeting the distribution of data rather than approximating the current energy model.

Original: Thissurprising connection has two beneﬁts: it wouldprobably allows us to scale this method  to higher dimensions, and it allows for efﬁcient parallel batchevaluation.
Revised: This connection has two potential beneﬁts for method γ , which has the well-known downsides of the kernel methods. The ﬁrst one is the scaling to higher dimensions since NTK could be more expressive than conventional kernels like RBF. The second beneﬁt is the scaling in terms of batch size since the scalar product kernel allows for efﬁcient parallel computation of the vector ﬁeld. We describe the full procedure in Algorithm 2.

Original: We hope that this connection could help us to overcome two main obstacles hindering theefﬁciency of kernel methods.
Revised: Algorithm 2 Method γ Require: samples from the target distribution p ( x ) get initial samples { x ( i ) 0 } ni =1 ∼ q θ (0) ( x ) ∝ exp( − E ( x, θ (0))) for t ∈ [0 , . . , T ] do estimate ∇ θ KL [Equation]

Revised: update samples x (

Revised: [Equation]

Revised: end for return ﬁnal set of samples { x ( i ) T } ni =

Original:  Empirical evaluation
Revised:  Empirical evaluation

Original: We empirically test the proposed methods α, β, γ and compare them against conventional approaches:
Revised: We empirically test the proposed methods α, β, γ and compare them against conventional approaches:

Original: PCD (Tieleman & Hinton, 2009) and sampling withthe Replay Buffer (Du & Mordatch, 2019). {+We D e n s i t y Target  
+}
Revised: PCD (Tieleman & Hinton, 2009) and sampling with Replay Buffer (Du & Mordatch, 2019). {+We D e n s i t y Target  
+}

Revised: P a r t i c

Original: Wefound that, for the Replay Buffer, it is important to reduce the noise magnitude (as also proposedin (Du & Mordatch, 2019)), which yields sampling from an annealed target.
Revised: l e s Anneal+RB PCD Alpha

Revised: Beta

Revised: Gamma   found that for the stability of Replay Buffer it is important to reduce the noise magnitude (as also proposed in (Du & Mordatch, 2019)).

Revised: That, however, yields sampling from an annealed target.

Original: For both “PCD” and“Anneal + RB”, we make 20 steps of stochastic Langevin on every iteration.
Revised: For both “PCD” and “Anneal + RB”, we make 20 steps of stochastic Langevin on every iteration.

Original: For our methodswe propagate the particles along the corresponding vector ﬁelds and make additional 10 steps of stochastic Langevin alleviating possible numerical errors.
Revised: For our methods we propagate the particles along the corresponding vector ﬁelds and make additional 10 steps of stochastic Langevin to alleviate possible numerical errors.

Original: For the target distribution we take toy2-d distribution, and try to match it with 2-layer fully-connected neural network ( 300 hidden units, Swish activations (Ramachandran et al., 2017)).
Revised: For the target distribution we take toy 2-d distribution, and try to match it with 2-layer fully-connected neural network (hidden units, Swish activations (Ramachandran et al., 2017)).

Original: For method gamma, we found that using the sameparameters θ throughout the learning leads toa degenerate solution.
Revised: For method gamma, we found that using the same parameters θ throughout the learning leads to degenerate solutions.

Original: Thus, we sample using the kernelk θ ( x, y ) =
Revised: Therefore, we sample using the kernel k θ ( x, y ) =

Original: E θ ∼ π 0 (cid:104)∇ θ E ( x, θ ) , ∇ θ E ( y, θ ) (cid:105) , where parameters θ are sampled from the initializationdistribution π 0 at each iteration, i.e. we use unlearned random networks to propagate particles.
Revised: E θ ∼ π 0 (cid:104)∇ θ E ( x, θ ) , ∇ θ E ( y, θ ) (cid:105) , where parameters θ are sampled from the initialization distribution π 0 at each iteration, i.e. we use unlearned random networks to propagate particles.

Original: [Figure]
Revised: [Figure]

Original: In Fig.
Revised: In Fig.

Original: 1, we demonstrate the learned densitiesand the particles.
Revised: 1, we demonstrate the learned densities and the particles.

Original: In Fig.
Revised: In Fig.

Original: 2, we report the metricsfor models and samples averaging over 10 independent runs.
Revised: 2, we report the metrics for models and samples averaging over 10 independent runs.

Original: We don’t report standard errors tokeep the plots readable.
Revised: We don’t report standard errors to keep the plots readable.

Original: As we see, only α andβ nicely capture all of the modes.
Revised: As we see, only α and β nicely capture all of the modes.

Original: We found thelearning via PCD to be the most unstable andunable to capture all of the modes in most cases.
Revised: We found the learning via PCD to be the most unstable and unable to capture all of the modes in most cases.

Original: Annealing with the Replay Buffer is the moststable method in our experiments, however, ityields the density with scaled temperature due tothe incorrect noise magnitude.
Revised: Annealing with the Replay Buffer is the most stable method in our experiments. However, it yields the density with scaled temperature due to the incorrect noise magnitude.

Original: Finally, we conclude, that the proposed methods demonstratebetter performance with a lower computationalbudget.
Revised: Finally, we conclude that the proposed methods demonstrate better performance with a lower computational budget.

Original: Interestingly, all of the methods manageto match the ﬁnal set of particles with the targetdistribution regardless of the learned energy.
Revised: Interestingly, all of the methods manage to match the ﬁnal set of particles with the target distribution regardless of the learned energy.

Original:  Conclusion
Revised:  Conclusion

Original: We approach the problem of sampling from a distribution, which evolves in time, which is especiallyimportant in the context of energy-based learning.
Revised: We approach the problem of sampling from a distribution evolving in time, which is especially important in the context of energy-based learning.

Original: Our main contribution is the approximate formulafor the vector ﬁeld that propagates the particles matching the evolution of the distribution.
Revised: Our main contribution is the approximate formula for the vector ﬁeld that propagates the particles matching the evolution of the distribution.

Original: We demonstrate that this formula yields several reasonable algorithms connected to deterministic Langevin and MMD gradient ﬂows.
Revised: We demonstrate that this formula yields several reasonable algorithms connected to deterministic Langevin and MMD gradient ﬂows.

Original: Intuitively, method α moves the particles matching the new energy and, atthe same time, unmatching the old energy.
Revised: Intuitively, method α moves the particles matching the new energy and, at the same time, unmatching the old energy.

Original: In contrast, methods β and γ propel particles aiming thetarget data distribution and repelling particles from each other to cover the space.
Revised: In contrast, methods β and γ propel particles aiming the target data distribution and repelling particles from each other to cover the state-space.

Original: Finally, we showthat our deterministic approach can be favorable in practice for learning energy-based models.Checklist
Revised: Finally, we show that our deterministic approach can be favorable in practice for learning energy-based models.

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [No]

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [No]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] will be includedin the ﬁnal version

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,

Original: how they

Original: were chosen)?

Original: [No] will be included in the ﬁnal version

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [No]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?

Original: [No]

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]

Original: (b) Did you mention the license of the assets?

Original: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?

Original: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A](b)

Original: Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A](c)

Original: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A]

Original: A Proof of proposition1
Revised: A Proof of proposition

Original: Proposition.
Revised: Proposition.

Original: The solution of
Revised: The solution of

Original: [Equation]
Revised: [Equation]

Original: is v ∗ =
Revised: is v ∗ =

Original: −∇ ∂E∂t .
Revised: −∇ ∂E∂t .

Original: Proof.
Revised: Proof.

Original: The evolution of ˆ q t is evolution deﬁned by the vector ﬁeld v , i.e.
Revised: The evolution of ˆ q t is evolution deﬁned by the vector ﬁeld v , i

Original: [Equation]
Revised: [Equation]

Revised: e.

Original: [Equation]
Revised: [Equation]

Original: and the ﬁrst argument of the scalar product is deﬁned by the updates of the energy
Revised: and the ﬁrst argument of the scalar product is deﬁned by the updates of the energy

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We rewrite the scalar product in (20) as
Revised: We rewrite the scalar product in (20) as

Original: [Equation]
Revised: [Equation]

Original: Integrating by parts, we have
Revised: Integrating by parts, we have

Original: [Equation]
Revised: [Equation]

Original: B Proof of proposition
Revised: B Proof of proposition

Original: Proposition.
Revised: Proposition.

Original: Consider the functional F = (cid:82) f ( q ) , which we can optimize either w.r [Equation]
Revised: Consider the functional F = (cid:82) f ( q ) , which we can optimize either w.r.t.

Original: t. q =exp( − E ) /Z or w.r.t.
Revised: q = exp( − E ) /Z or w.r.t.

Original: E .
Revised: E .

Original: Vector ﬁelds v ∗ = v (cid:63) when the evolution of the density (energy) isdeﬁned by the Frechet derivatve of F .
Revised: Vector ﬁelds v ∗ = v (cid:63) when the evolution of the density (energy) is deﬁned by the Frechet derivatve of F .

Original: Proof.
Revised: Proof.

Original: The directional derivative (along the direction h ∈ L 2 ) is
Revised: The directional derivative (along the direction h ∈ L 2 ) is

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We can think of δF ( q ) δq as of the formal symbolic application of differention rules.
Revised: We can think of δF ( q ) δq as of the formal symbolic application of differention rules.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: (29) which coincides with the vector ﬁeld given by the Otto Calculus (Otto, 2001).
Revised: 29) which coincides with the vector ﬁeld given by the Otto Calculus (Otto, 2001).

Original: For the energy, weconsider the direction h ∈
Revised: For the energy, we consider the direction h ∈

Original: L 2 q , then we have
Revised: L 2 q , then we have

Original: [Equation]
Revised: [Equation]

Original: Finally, we see that the two derivatives yield the same vector-ﬁeld.
Revised: Finally, we see that the two derivatives yield the same vector

Original: [Equation]
Revised: [Equation]

Original: C Proof of proposition3
Revised: C Proof of proposition

Original: Proposition.
Revised: Proposition.

Original: The vector ﬁeld v ∗ =−∇
Revised: The vector ﬁeld v ∗ =

Original: ∂E ∂t may be approximated by the “conventional update rule”of the particles following the Langevin dynamics targeting the updated density
Revised: −∇ ∂E∂t may be approximated by the “conventional update rule” of the particles following the Langevin dynamics targeting the updated density

Original: q t + dt .
Revised: q t + dt .

Original: Proof.
Revised: Proof.

Original: Let’s approximate the vector ﬁeld v ∗ as
Revised: Let’s approximate the vector ﬁeld v ∗ as

Original: [Equation]
Revised: [Equation]

Original: Then the evolution of the density is described by the FP equation:
Revised: Then the evolution of the density is described by the FP equation:

Original: [Equation]
Revised: [Equation]

Original: Hence the evolution of particles can be described by the Ito equation
Revised: Hence the evolution of particles can be described by the Ito equation

Original: [Equation]
Revised: [Equation]

Original: (34) where dW t is the Wiener process, which can be simulated by the normal random variable N (0 , dt (cid:48) ) .
Revised: 34) where dW t is the Wiener process, which can be simulated by the normal random variable N (0 , dt (cid:48) ) .

Original: Taking dt (cid:48)
Revised: Taking dt (cid:48)

Original: =
Revised: =

Original: dt , we have the conventional update rule (up to the step size choice)
Revised: dt , we have the conventional update rule (up to the step size choice)

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: D Proof of Proposition4
Revised: D Proof of Proposition

Original: Proposition.
Revised: Proposition.

Original: The convergence of (16) is described by the equation:
Revised: The convergence of (16) is described by the equation:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: [Equation][Equation] [Equation] [Equation]
Revised: [Equation]