Hitchhiker's guide on the relation of Energy-Based Models with other generative models, sampling and statistical physics: a comprehensive review
Authors: Davide Carbone
Venue: Accepted by TMLR ()

Abstract: Energy-Based Models (EBMs) have emerged as a powerful framework in the realm of generative modeling, offering a unique perspective that aligns closely with principles of statistical mechanics. This review aims to provide physicists with a comprehensive understanding of EBMs, delineating their connection to other generative models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Normalizing Flows. We explore the sampling techniques crucial for EBMs, including Markov Chain Monte Carlo (MCMC) methods, and draw parallels between EBM concepts and statistical mechanics, highlighting the significance of energy functions and partition functions. Furthermore, we delve into state-of-the-art training methodologies for EBMs, covering recent advancements and their implications for enhanced model performance and efficiency. This review is designed to clarify the often complex interconnections between these models, which can be challenging due to the diverse communities working on the topic.

Published in Transactions on Machine Learning Research (MM/YYYY)
Hitchhiker's guide on the relation of Energy-Based Mod- els with other generative models, sampling and statistical physics: a comprehensive review
Davide Carbone
davide, carbone Sphys. ens.fr
Laboratoire de Physique de l'Ecole Normale Supérieure, ENS Université PSLL CNRS, Sorbonne Université, Université de Paris, Paris
Reviewed on OpenReview: http:///penmriieene//fmumuuuun Pid=
Abstract
Energy-Based Models have emerged as a powerful framework in the realm of generative modeling, offering unique perspective that aligns closely with principles of statistical mechanics. This review aims to provide physicists with a comprehensive understanding of EBMs, delineating their connection to other generative models such as Generative Adversarial Networks, Variational Autoencoders, and Normalixing Flows. We explore the sampling techniques crucial for EBMs, including Markov Chain Monte Carlo (MCMC) mothods, and draw parallels between EBM concepts and statistical mechanics, highlighting the significance of energy functions and partition functions. Furthermore, we delve into recent. training methodologies for FBMs, covering recent advancements and their implications for enhanced model performance and efficiency. This review is designed to clarify the often complex interconnections between these models, which can be challenging due to the diverse communities working on the topic.
1 Introduction
Several surveys Energy-Based Models (EBMs) have been published in recent years, each offering valuable insights into specific aspects of this modeling framework. Some works focus on their historical development and original formulation prior to the deep learning era (LeCun et al. 2007). while othors examine thoir conceptual ties with statistical physics and the theory of Boltziann Machines (Huembeli et al., 2022), or their evolving role the modern generative modeling landscape (Bond-Taylor et al.] 2021 Song & Kingma, 2021). However, these reviews often lean towards either algorithmic and empirical advances from machine learning perspective or foundational insights rooted in statistical mechanies, rarely addressing both com- prehensively. Moreover, crucial notions such as sampling, energy normalization, and the interpretation of the Boltzmann- distribution are frequently treated as technical details or relegated to side discussions, despite being central to both the theory and practical implementation of EBMs.
In contrast, the present review aims to provide a unified and pedagogical account of Energy-Based Models that is both self-contained and accessible to broad interdisciplinary audience. Specifically, we take the viewpoint that EBMs should not be understood solely as machine learning model class defined by an un- normalized density function, but rather and modern generative modeling, Our goal is to elevate the role of sampling -froffoommm peripheral computa- tional tool to contral component of the model's thcoretical foundation and training Synamics-emphasizing dynamics,
natural interface between statistical physics, sampling theory,
how concepts from nonequilibrium statistical physics, such as Langevin and directly shape the behavior and feasibility EBMs.
This work not just a tutorial on existing training algorithms; rather, seeks to clarify the fundamental principles and limitations that stem from the very definition of EBMs as energy-parametrized distributions. In doing so, aim to help readers understand why tasks such as likelihood evaluation, generation,

---

Published in Transactions on Machine Learning Research (MM/YYYY)
inference remain non-trivial within this framework. This perspective also allows us to critically assess the normalizing flows, GANs, VAEs, and
relationships between EBMss and other generative diffusion models highlighting both structural differences and shared algorithmic tools (e.g., score-based learning, Langevin sampling).
We believe that such an integrative approach is currently lacking in the literature. While many excellent surveys focus on specific algorithmic developments or empirical applications, few attempt to articulate an auto-consistent conceptual framework that situates EBMs the intersection of physies, sampling, and gener- ative modeling. This review aspires to fill that gap, acting "Hitchhiker's Guide" for physicists venturing into generative modeling, and for machine learning researchers interested in the deep physical analogies un- derlying their models, trying to balance soundness for newcomers and technical details.
To provide historical depth and context, we also include in Appendix Al brief chronology of the development of EBMs and their connections to key ideas in both physics and AL. While not necessary for understanding the main technical content, we strongly encourage readers to consult for a broader perspective.
Contribution. The structure of the review will be the following:
in Section 2 we define Energy-Basse Based Models and we highlight the main difficulty related to its training through cross-cntropy minimization;
in Section 3 we present a review of the principal generative models as opposed to EBMs. Then, we conclude the section with a comparative scheme between all the presented models, with an accent on the unique features of EBMs;
in Section 4 we review the main MCMC methods used to sample from a Boltzmann- Gibbss ensemble, hence used the context of EBM training to generate the necessary sample used to perform gradient descent cross-entropy;
in Section 5 we summarize the derivation of Boltzmann- Gibbs equilibrium ensemble, motivating the importance of concepts free energy in the context of statistical learning;
in Section D we present recent progress in EBM training and we highlight their limitation, in relation with relatively old methods as Constrastive Divergence.
2 Definition of EBM
In this Section, we provide the basic formal definition of Energy-Basod Model. We will adopt the notation and the presented assumption throughout the present work. First of all, the problem we consider can be formulated as follows: assume that we given
N data points {x [n Tizı in F drawn from unknown probability distribution that is absolutely continuous with respect to the Lebesgue mcasure on Rd with a positive probability density function (PDF) (.(x) > 0 (also unknown). This is a standard problem in statistical learning, where learning from data here refers to the ability to fit the data distribution and to generate new examples. More precisely, our aim is to estimate (z() via an energy- based model (EBM), i.e. to find a suitable energy function in parametric class, Ug : Rdd - [0, oo) with parameters O, such that the associated Gilbsmann-Gibbs PDF
po(x) = -ull))
Zg
/ e Uo(ox) dr JRC
(1)
is an approximation of the target density (x). Actually, any probability density function can be written as a Boltzmann Gibbs ensemble for a particular choice of Uz). The normalization factor Zy is known as the
partition function in statistical physics (Lifshitz & Pitaevskii 2013). see also Section and as the evidence in Bayesian Statistics(Fero) & Hobson 2008).
Remark 2.1. Even if Ua known, an erplicit analytical computation of the partition function is generally unfeasable. If the dimension d big enough, the integral defining Zg cannot be computed using standard quadrature methods. The only possibility is Morle-Carlo sampling Lia 2007). To employ such method, one
2

---

Published in Transactions on Machine Learning Research (MM/YYYY)
express the partition function expectation Eg with respect to chosen probobility density function Po, i.e.
e Up
Ze = Eo
(2)
PD
The selected density must be known pointwise in RR,, including the normalization constant, and it should be easy lo somple from. If these conditions are met, one can compute the partition function by simply replacing the expectation in equation P with the corresponding empirical overage computed using samples drown from po- Unfortunately, finding a probability density that satisfies these properties is challenging. For a general choice that not tailored e e-V,, the estimator likely very poor, characterized by a very large, or even infinile, coefficient of varialion.
One advantage of EBMs is that they provide generative models that do not require the explicit knowledge of Zg. In Section 4 we will review some routines that can in principle be used to sample po knowing only Up the design of such methods is an integral part of the problem of building an EBM.
To proceed we need some assumptions on the parametric class of energy: Assumption 2.1. For all 0 E e:
Uo E CRRR)); AL E R : [VVG(x)) I Vr RR::
2. Ea E R2 and compact set C Rd: x (Uo(2) 2 a|r|? Vr E RRd \
The need for the first assumption will be discussed in Section 4 it is related to wellposedness and convergence properties of the dynamics used for sampling, i.c. Langevin dynamics and its specifications. The second assumption guarantees that Zo < xo (i.e. we can associate a PDF Po to Up via equation for any 0 E θ).
We provide now two important definitions:
Definition 2.1 (Convexity). A function vp : 12 - (-00, +ool is convex if given 0 <<A A < 1 and 21, E gd such that Iı F 22, the following is true
(1x1 + (1-1)a2) S l0p (x1) (1 1)sp (x2)
(3)
Definition 2.2 -ogg-Conaavity). A density function p with respect to Lebesgue measure (R4, Bd) is log-concave if - where is conver.
A non-convex function could have more than one local but not global minima; conversely, a non-log-coneave probability density could have more than local maxima, which are called modes. Tt. is important to stress that Assumption equation 2.1 does not imply that Ug is convex (i.e. that Pe is log-concave): in fact, we will be most interested in situations where Ug has multiple local minima so that Pa is multimodal. We will elaborate on the topic in Section ! It is well known as for optimization problems, non-convex cases are the most complicated. Similarly, sampling from a non-log-concave probability density function (PDF) can be extremely challenging. Another assumption we will adopt is:
Assumption 2.2. Without loss of generality 30, θ Pa. = Pa. that is pa the parametric class of Pg.
The aims of EBMs are primarily identify 0, and to sample po.; the process, we will also show how estimate Ze_.
Example 2.1. Let us present simple example to visualize the relation between converity and bog-concavity. In Figure 7 we plot side by side the PDF of Gaussian mixture in ID and the associated potential
(x 11)2 (- of
+ (L-p) p) exp (-60- H2)
(x (x-
Up(x) log p exp
(4)
where 6 = M1.2, 012.. The specific values are p = 0.7, H1 O, p2 5, σ1 and 02 0.5. It is clear the correspondence between minima of (x) and masima, that is modes, of p.
3

---

Published in Transactions on Machine Learning Research (MM/YYN
Gaussian Mixture and Sampled Data
Energy
0.30
Dall
0.25
0.15
6.05
Figure 1: Gaussian Mixture. Plot of PDF with sampled histogram and associated energy Ug.
2.1 Cross-entropy minimization
Once we defined an EBM, we need to measure its quality with respect to the data distribution. Possibly, this would provide way train its parameters. Hence, we define some important quantities: Definition 2.3. Consider two probabilily densities Rd and absolutely continuous with respect to Lebesgue measure, namely P1 and P2. We define
1. Cross Entropy
H P2) -- log pa (1) (x)dx
(5)
2. Kullback-Leibler divergence
KivergenceKKulbackkk Kullback & Leibler 1951
DKL(M ll p2) / (1()) log P1 (2:)
p2(x)) dx
(6)
Entropy
H(p)) = - log (1) (x)dz
(7)
The KL divergence widely used estimator for the dissimilarity between probability measures. It satisfies the non-negativity condition
DKL(P1 12) 2 0, (KL |l P2) = = P2 a.e.
(8)
However, it is not a proper distance since it not symmetric and it does not satisfy triangular inequality. The following trivial lemma. relates the three quantities introduced in Definition 2.3
Lemma 2.1. The following equality holds for any choice of PDFs Pr and P2
H(P1.P2) H(2)) + DKu(p2 ll P1)
(9)
One can also the cross-entropy of the model density pe relative to the target density P+ estimate of diversity between the two PDFs; in such case, 5 simplifics becoming
H(p., Po) = log Zo + / (z)))).(2)dd Rd
(10)
4

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Because of 9 the difference between the cross-entropy and the KL divergence is Hpx), term that depends just on the data distribution. Hence, the optimal parameters 0* are solution of an optimization problem on θ namely
0* = argmin min (p-(p=)) = arg min H (ps, lEe
(11)
AEe
meaning that the entropy of p. plays active role in solving such minimization problem. There is subtle issuc in this reasoning: unlike KL divergence, the cross-cntropy is not bounded from below, and in particular (p, = (p) + 0. That is, we should compute (p-) to estimate the minimum value of cross-entropy. Unfortunately, most of the empirical estimators to be used when ps is known through samples suffer high
dimension(Ao & Li 2023). Solving equation 11 equivalent to maximum likelihood method, a widely used practice in paramctric statistics 2007):
The use of cross-entropy avoids the very problematic computation of H(.)) but in 10 the estimation of Zo is also needed. However, the most common routines for cross-entropy minimization are gradient- gradient-based: they rely on the gradient of 0g Hg po) and not on the cross-entropy itsclf. The former can be computed using obtaining
the identity 0g log Zo = - SRR
(p = [ (2))
/ Jac (gU)(())po()ddz
(12)
= [0g - R.g (0gU
This is a crucial expression for the present work, and the consequence immediate: Remark 2.2 (Fundamental problem for EBM training). Estimating 0g (p requires calculating the expectation Eg [O,U)) In contrast I., (00U5] can be readily estimated on the data.
Typical training methods, e.g. based on the so-called Constrastive Divergence (Hinton 2002) and its speci- fications (sce Subsection 0 resort to various approximations to calculate the expectation Eg (O,U)). While these approaches have proven successful in many situations, they are prone to training instabilities that limit their applicability. The cross-entropy is more stringent, and therefore better, than objectives like the Fisher divergence used to train other generative models: for example, unlike the latter, it is sensitive to the relative probability weights of modes on p. separated by low-density regions (Song & Kingma 2021
3 EBMs among generative models
In this section, we briefly outline the main generative models, aiming to provide a general framework without focusing on technical dotails. As introduced carlier, a generative model is a computational tool that produces new instances representative of a dataset for example, generating new images of dogs after training on a dataset of dog images. Evaluating the quality of generated data is often non-trivial, and designing good generative model typically involves a trade-off between ease of training and ease of generation.
Historically, generating data meant collecting measurements. The advent of computer simulations, such as the pioneering Dormi-Pasta-Ulante Ulam- Ulam-Tsiinou experiment (Fermi et al., 1955), marked a shift toward generating synthetic data from models, In statistics, this process is referred to as "sampling" (see Section 4 To- day's generative AI, enabled by vast computational resources and data availability, continues this shift by generating data from data. As with the early interne what began in research now shapes everyday life. Generative Pre-Trae Trained Transformers like Chat exemplify this trend. These tools are prompting broader discussions about artificial general intelligence and its societal implications (Morozov) 2023 Fjelland 2020 Federspiel et al. 2023)
We now turn to the core families of generative models, and ultimately, we will highlight their connections to Energy-Bass -asee Models.
3.1 Variational Autoencoders
As the name suggests, to introduce variational autoencoders (VAEs) GGrin et al. 2020), we first recall the structure of a standard autoencoder (AE) (Hinton & Salakhutdinoy 2006 illustrated in Figure An
https://www.1ivvininnteet wttp://eeeeooo//oe.eeeeeeaa htmm////iarraaet.httt
5

---

www.:aatfoooollattttomdrrreeeeee Published in Transactions on Machine Learning Research (MM/YYN)
neural network encoder
neural network decoder
x
2 z=
* = d(z)
loss = || x I|2
d(z)) 112
x d(e(x)) I|2
Figure 2:
Representation of an autoencoder, taken from
http:://owarddddttasieer...om
AE is a deep neural network composed of two parts: an encoder e(x), which maps the input x Ied to latent reprosentation E R with L < d, and decoder d(2), which attempts to reconstruct ir from z. Training minimizes the reconstruction error between & and d(e(x)). Once trained, AEs can be used as compression tools, and additional layers can applied in latent space for downstream tasks (Bank et al. 2023 This process entirely deterministic. Variational Autoencoders (Kingma Welling 2014) extend A by introducing probabilistic encoders and decoders, respectively referred to inference and generative models. Let pa =) po(x)=)p(=) be joint distribution on Rd x TR4,, with prior p(=) (2; Oz, Ir.) and decoder defined by a conditional Gaussian:
(0(x|2) N(;; (o()), diag(
(13)
where the mcan and variance neural network outputs. Alternatives exist for other data types, e.g., audio Girin et al. 2
The marginal model distribution is given by
pa(x)= J po(x(z)p(=)dzz
(14)
and training aims to minimize the KL divergence from the true data distribution p=(x):
0" arg max E. E.(log po(x)),
(15)
d=
analogously to EBMs. However, the marginal likelihood equation 14 is generally intractable, preventing direct evaluation. To address this, VAES rely on variational approximation, which we introduce next:
Definition 3.1 (ELBO). Let F denole variational family defoned as set of PDFs over the lalent variables 2. For any ((2) E F, the Evidence Lower Bound (ELBO) (also known variational free energy)
C. x Rd + R is defined as
C(O, <0){());<) = pog llg {())]
(16)
Lemma 3.1. The following properties hold true:
1. Decomposition of marginal log likeihoo Neal & Hinton 1998).
log Pa L(0,9(=);;)) D Dxa(g(=) (=(=)=))
(17)

---

Published in Transactions on Machine Learning Research (MM/YYY)
2. Bound marginal log-likelihood.
(()))(2); 1 log po(x))
(18)
((0,g(2);0) = log po(x)) 7 ((2) (=(==a))
Proof. The proof of (1) is trivial:
[(0, <());z) + (xl( |l po(2|z)) = BB(z) (o log [()]
+ (los) (=2) log po = By(s)) log po(2z;2) 2) (=( )
(19)
log (x(
where used the definition of conditional probability and the fact that the expectation is computed in the latent space. (2) is direct consequence of equation 17 since the KL divergence is non-negative and identically zero just when ((2) = po(2
Thanks to these results, the log- liglliieihood can be estimated via the Iopectation-Maxxiiiaan (EM) algo- rithm (Dempster et al. 1977 the (F) step solves the unconstrained variational problem at fixed
(2) arg max L(0, {()):))) OEF
(20)
while the (M) step maximizes the ELBO w.r.t. fixed g(z). Since ((2) depends on æ, is more precisely written (2((z). Under suitable conditions, this EM procedure converges maximum and saturates the bound in equation 18
However, solving this functional optimization can be intractable. A practical alternative is fired-form vari- ational inference (Honkela et al. 2010), where the variational family F is restricted to parametric densities (x(=(z) with parameters À E A. For instance, a Gaussian family with (x(x(=) = N(:; μt, Σ) corresponds to = {H. E}. Then, the E-step reduces to
X arg max L(0, A;
(21)
For a dataset X' { the total ELBO becomes
((0, Σ A(0,),
(22)
but optimizing N local parameters A quickly becomes unmanageable, especially in high dimensions. To overcome this, amortized variational inference assumes parametric map fo such that X, fz(z), making the ELBO
C(0, o;; X)
E (al [log po( 2;) log (o(2)
(23)
Thus, training the decoder p=(x)2) reduces to maximizing this objective jointly over and d, where to (2(x))
approximates the intractable posterior po(2[z). The Variational Autoencoder (VAE) is special case of this framework, where (=(2))) is parameterized by neural network (encoder). A common choice is Gaussian:
as (x(2)) ((2; (x),,, diag(o)
(24)
with ss and a learned via backpropagation. Joint training of encoder and decoder can be suboptimal He et al. 2018), yet it remains common.
Using KL divergence and Bayes' rule, we rewrite equation 2 as
N
N DKL (2<(2((2)))l((3)))
L(8 x) Σ Ens(edaa) [log (2)
Σ
(25)

---

Published in Transactions on Machine Learning Research (MM/YYYY)
where the first term promotes reconstruction fidelity and the second regularizes the posterior to align with prior (typically standard Gaussian), aiming for disentangled latent codes. The last important point is that in computing gradients, we must avoid the intractability of the marginal likelihood. While the KL term is analytically tractable for Gaussians, the expectation term requires Monte Carlo sampling 2, this:
(r) ~ (o(=))m;,, but this sampling is non-differentiable. The reparametrization trick resolves
ps(x;) + diag( ( e ~N(,1)
(26)
enabling gradient flow through stochastic nodes. The resulting estimator is
R
N
[(0, d; X)
log pe (1: (x:l<()))
Σ
DKL. [=((</=)))l)
(27)
VAEs thus optimize surrogate for the log-likelihood while enabling efficient sampling from p(z).
For completeness, we just. mention some references showing main limitations of VAFs (Wei et al. 2020:
Oussidi & Elhassouny 2018 Sengupta et al. 2020): performance is sensitive to hyperparameters (c.g., prior choice, weightings), the latent structure may fail to disentangle data factors, and expressiveness may be limited due to simple priors. Generated samples may appear blurry or suffer from mode collapse, and VAEs often underperform on complex, high-dimensional datasets compared to alternative generative models.
3.2 Generative Adversarial Networks
Generative adversarial networks detworss(Goodfelloor Goodfellow et al. 2014) (GANs) are class of generative models which take inspiration from game theory. They consist of two neural nelworks (see Figure 3), namely a generalor G and a discriminator D, trained simultaneously through the so-called adversarial training. Given a dataset X' sampled from the unknown data distribution p.; the generator devoted to generate synthetic data that ideally resembles the training data. On the other hand, the discriminator has to discern between fake and true samples. In this sense, G and D are adversary: the generator aims to produce realistic data to fool the discriminator, while the discriminator strives to correctly classify real and fake data. Thus, the training ends when the discriminator becomes unable to effectively distinguish between real and generated samples. Let us present the mathematical formulation: firstly we define a prior P=(z), which is a PDF easy to sample
Training set
Discriminator
Real
Random noise
Fake
8
Fake image
Generator
Figure 3: Scheme of the structure of GANs, taken from
from that serve to inject noise into the generator. The latter is a function Go, (3) that fed with noise and generate "fake" samples that should be similar to samples from px. The discriminator function Do, (x) that gives the probability that sample comes from the training set rather than have been generated by G. Both 0, and 0g are parameters of NN. The optimal weights are solution of the following two-player minimax problem:
parametric
arg min arg max E. [log Do, (x) + Ep, [log(1 Do, (2))) arg min arg max V(G, 0g
(28)
0g
8

---

Published in Transactions on Machine Learning Research (MM/YYYY)
We refer in the following to pz(x) the distribution of 'fake' samples induced by the generator, that is such that
Pp. log(( - (o,(Go,(=))) = Ep log(1 - (o,
(29)
The empirical idea to solve the minimax game is via an alternating algorithm: Proposition 3.1. The oplimizalion algorillum for a GAN is made by two allernating sleps:
Update of the discriminator
(data) from p.. 2. Compute Vo, V(G, D) and perform gradient ascent to update 0d-
1. Sample {2(1) M==1 (noise) from Pz and {2!0)
Update of the generator
1. Sample (2 Fi=1 (noise) from Pz.
2. Compute Vo, V(G, D) and perform gradient descent to update 0g-
This proposal is driven by common sense, but more careful analysis of the minimax game is necessary to eusure convergence of such algorithm. In order to characterize the solutions of this adversarial game, it is necessary to search for the optima. The method of proof is: (1) classify solutions of optimization of D at fixed G and viceversa and then (2) present a convergence result of the alternating game. Let us start from the update the discriminator:
Theorem 3.1 (Existence of optimal discriminator or Goodfellow el al. 2014 ). For G fixed, the optimal dis- criminator is
P. (x(x)
D2 (x(x) + Po(x)
(30)
Proof. Using equation 28 and equation 29 we have
/ (p. log (x) (1) Do (x))
V(G, D)
(31)
The function y - log((y) + log(1 y) achieve its maximum in (0,1) at a/(a b) for (a, + (0,0). Applied to the case in study, the discriminator can be defined just in Supp(p. U Supp(e, (x))) hence concluding the proof.
This lemma ensure that the gradient ascending will eventually reach maximum, that is
C(G) arg max V(G, = =E P. (x)
Po(x)
E
(32)
D
(x) + Pg
(.(x) Pg (x(x)
Now we need to characterize the solutions of the minimization problem arg ming C(G) Theorem 3.2 (Existence of optimal generator G* induce Py such that Py P.- At that point, C(G*) log
GGoodfelow et. al.) 2014)). At fixed D D', the oplimal
Proof. Regarding the last point, for Py P. we obtain D = 1/2, that inserted in C(G) gives exactly log 4. We need to test whether this is global optimum:
and subtract log 4 to C(G) obtaining
C(G) log(4) DKL. P+
P. P.tPs + DKL. Pg
p. Pg
2
(33)
log (4) + 2 pv)
where JSD is the Jensen-Shannon divergence Manniig & Schutze 1999: Such quantity has the same non- negativity property of KL divergence, i.c. JSD(p, Pg) 2 and JSD(p- || pg) iff Pg p.. This proves that Pg P., more precisely the corresponding generator G", is the global minimum for C(G). L

---

Published in Transactions on Machine Learning Research (MM/YYYY)
To summarize, we have showed separate theoretical guarantees about convergence of gradient ascent and descent. However, we need to show that alternating those two steps would eventually converge to the global Nash equilibrium of the minimax game, i.e. P. = Pg- The result is summarized in the following Theorem Goodfellow et al., 2014) of which omit the proof for the sake of brevity.
Theorem 3.3. If G and D have enough capacily, and at each slep of the allernating algorithm, the dis- criminator is allowed to reach its optimum given G, and Pg is updated so as to improve the criterion
[og Dī(()) + Ep, Dog(1 - -D(())
(34)
then Py converges to P..
Ideally, the theoretical treatment of Generative Adversarial Networks concludes with the proof that the pro- posed minimax game has a unique Nash equilibrium. This equilibrium corresponds to a generator capable of sampling from Pe, making it indistinguishable from true samples by the discriminator, performing no better than a random classifier with a probability of 1/2.
GANs have several drawbacks (Radford al.) 2016
Salimans et al, 2016 Arjovsky & Bottou 2016),
Optimization in parameter space on 0g rather than functional space on Pg introduces challenges, making what should be a convex problem non-convex. Ad- ditionally, training the generator is difficult due to gradients being near zero early on when the generator performs poorly. GANs are also known for training instability, requiring careful balancing between the gencrator and discriminator. Hyperparamctor tuning is often needed, and GANs require large datasets for generalization. Sample generation can suffer from mode collapse, where outputs lack diversity, and computa- tionally, GANs can be demanding, especially with high-resolution images. Evaluating GAN performance is difficult, as metrics like Inception Score and Frechet Inception Distance have limitations, complicating model comparison. Despite these issues, GANs remain important for unsupervised learning and model robustness.
highlighted by practical applications of Theorem 3.3
3.3 Diffusion Models
Diffusion generative models (Sohl-Dickstein et al. 2015 Yang et al. 2023 Croitoru et al. 2023 are class of models that use diffusion processes to transform simple distribution into a more complex one over time. The process starts with a basic distribution, such as Gaussian noise, and iteratively transforms it into a target distribution that approximates real data, such as images. Recently, diffusion models have become stale-of-the-art several domains, partly replacing GANs (Dhariwal & Nichol] 202T). In this section, we summarize the common features of diffusion models without diving too deeply into specific variations.
As with other generative models, the core ingredient is dataset X' a;i where x; are drawn from an unknown target density (z). For simplicity, we assume X C Rd. In VAEs and GANs, the goal is to generate new samples from noise: VAFs decode from a Gaussian in latent space, and GANs generate from noise through the generator G. Similarly, in diffussin models, the aim is to push samples extracted from a simple distribution, like a Gaussian, toward the data distribution.
Since the following discussion is closely related to stochastic calculus (Rogers & Williams 2000), we introduce the notation. We denote X, E Rd as stochastic process, a sequence of random variables where t e R represents continuous time. Unlike deterministic processes, the focus here on the distribution in law of Xt, denoted p(x, t), rather than on the individual trajectory. Just as deterministic trajectories are solutions to ordinary differential equations (ODEs), stochastic processes are solutions to stochastic differential equations (SDEs).
Proposition 3.2 (SDE and Fokker-Plank PDE). Given the drift u Rd x R + Rdd and the diffassio malria Rd x + Rddd let us consider the stochastic process Xe solution for t E [0,T] c [0, +20] of the SDE
dX. = p(X,, t) + a(x,, t)dWe,
Xo Po
(35)
where Wg is a Wiener process. Using Ito convention, the law of Xt, namely p(a, t), satisfies the Fokker- Fokker-Plance partial differential equation (PDE)
o oz t)²
((x, 1)
5y (xr, 1) = -v. Vla, ())(z, 0) + A
((x;0) = Px(x))
(36)
2
10

---

Published in Transactions on Machine Learning Research (MM/YYYY)
This proposition is important to understand the relation between the single random process X and its distribution in law. Let present simple example to clarify such connection.
Example 3.1 (Wiener process). Let consider the case p(x,t) 0 and oxr in 1, that corresponds to the SDE
=X,=W;
(37)
The solution of the associated Fokker-Planck equation
0p(z,t) t)
10 p(r,t)
(38)
Ōt
dx²
for a delta initial datum p(z,0) 0) = 6(x) is precisely
p(a,t)
x² 122
(39)
V2Tt
This is a gaussian density with voriance proportional t. That is, the initial concentrated density spreads the real line.
This brief summary about SDEs is sufficient provide a consistent definition of generative diffusion model:
Definition 3.2 (Generative diffusion model). Let us consider the data distribution pa R + R+ and a base
distribution p(x) Rd Rt. Given a time interval [0. 7] E [0, ool, generative diffusion model an SDE with fized lerminal condilion
dx, ((x,))) + (Xx,
Xa ~ P, NT ~
(40)
where W. is Wiener process.
This definition resembles concepts from stochastic optimal control Fleming & Rishel 2012) in fact, the terminal condition is not sufficient to uniquely fix p(X,,t) t) and t),,t). Under this point of view, the
specification particular class of diffusion models reduces to a prescription on how to determine the drift and the diffussin matria. the following we will summarize two highlighted methods present literature.
Score- based diffusion(Sonna et al., 2020b). To explain what is score-based diffusion we need the fol- lowing preliminaries:
Definition 3.3. Given a PDF p(z), the score is the vector field
s(x) Vlog p(z)
(41)
Proposition 3.3 (Naive score-based diffusion). For any O and Po(z), the choice p(x,t) =Sx(x) <V log (x) and a /28 in equation 40 salisfies the endpoint condition for =
Proof. If we consider the we have
PDE associated to equation 40 with the selected drift and variance,
((xx (ele.1)
O,plx,t) t) = V V- (x) Vp(x,t)] V. p(z, ((zx,t)Vlog
(42)
p=(x))
By direct substitution, the stationary probability density (x( is a solution. For uniqueness, we need to prove that any solution of the PDE would converge to this solution. A formal argument is based on Jordan- Kinderlehrer- Otto (JKO) variational formulation Fokker-Planck equation gradient flow in probability space with respect to Wasserstein-2 distance. An alternative way is the following: for any solution p(x,t), we compute the time derivative of the KL divergence between such solution and p. (x). If we define R = p/p.:
al.) 1998). interpreted
d DKL(p 11 pa) dt
d /f p log R dx / Osplog Rdz + / P Lo dt
(43)
gg R
11

---

Published in Transactions on Machine Learning Research (MM/YYYY)
We can use Fokker-Planck equation to substitute dep and integrate by parts: d dt pl) dt / plog Rd - f plV R| da / P.O, dx
d
(44)
We notice that p.R.R- = V (plog R), hence that the second addend is zero by integration by parts. The conclusion that
dt Dkl(ell / p V
Vog log Ri dx I 10,
(45)
concluding the proof.
The result seems to say that able to build diffusion generative models estimating the score of the larget. In data driven context, P. is known just through data points and one has to face the problem of estimating 8., A possible Sarroach(Hyvärinee & Dayan 2005) is score matching. Definition 3.4 (Fisher divergence). Given two PDF's p(z) and π(²), the Fisher divergence is defined as
DF(e [ T) / p(x) V Vlogp(a) V log (z)](>))l dr
p(z)
(46)
Even if in some sense Dr seems to measure some distance between two PDFs, it is very different from the KL divergence, see following Remark.
Remark 3.1. By defimition, both KL and Fisher divergence between two PDFs satisfy the non-negativity properly, i.e. they are striclly positive, and zero only when the densities are the same. DF does not depend on normalization constants of the PDFs because of the gradients. This is a double-edged weapon: it is apparently useful in high dimension, where the computation of normalization of a density is impractical (as for instance the partition function for EBMs). But if the distribution is multimodal, the local nature of Dr is very insensible to global characteristics of the densities, as for instance the relative mass in each mode. Let us consider key exomple: the distributions we would like to compare are:
(1 = 0.5.( -5, 1)(x) (x, P2(1) = oo) (2,N(a,,5),))) 5) - (1-0
(47)
where = + is sigmoid function. The two densities are bimodal gaussian mixture in 1D with same means and variances; the second mizture is balanced with relative mass equal to 1/2. We would like to compare Dr(p) ll p2) and DKI(PI || p2) as functions of z. In Figure A we plot the two divergences in function of z. We estimate the expectations that define the two divergences using a Monte Carlo estimate, namely
N
Dr(p P2) ~
V log V log P2(:))1
(48)
[ (2 (2(2))
DRi(pi p2)
log
where I; ~ -1()). The minimum value is O and corresponds to z = O, that is P1 = P2- The first difference in general, this could be a problem in
is that the values of Dy are smaller of several order of magmitude practical implementations. Most importantly, the shape of the curve is very different. In this one dimensional example we need N 10000 to appreciate a similar growth, even if Dr curve more steep. For smaller N, Dr is basically flat. for + 0. This is related to the absence of points in low densily regions, that is where the integrand in DF gives a non-zero contribution.
In score matching, one propose a parametric score so, for instance neural network, and train such model to match the true score s.. The loss on which the model is trained, using for instance gradient routines, is
( pol1s0 Vog 1.l 2 dr = E.[lssol2 + tr(J=so)] + Cp gd
C(so, p.)
(49)
12

---

Published in Transactions on Machine Learning Research (MM/YYY)
Picls Ip)))
0g)
10
N=10000
N-10000
N=1000
N-1000
Nm100
1-100
10 45
10a
Fither Diverganee
KL K.Dioeegence
10
-0 22
10.2
10
10 14
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Figure 4: Comparison between KL divergence and Fisher divergence for the two bimodal gaussian mixtures in equation 47 The variable z (0, oo) is related to the relative mass of the two modes via a sigmoid function (2) E 1); the plots for 2 < 0 are analogous by symmetry. Notice the different scales of the y axes. The Monte Carlo estimation performed using N 100, 1000, 10000 samples.
where we integrated by parts, Cp = const does not depend on and Ju is the Jacobian with respect to x. Denoting with Po one (n.b. not. unique) PDF associated to 80, the loss is evidently Dr(p. po). The right hand side reformulation in equation 49 crucial: the expectation E. can be estimated using data points our disposal, bypassing the problematic term v log P..
Unfortunately, the naive score-based approach is plagued by two fundamental issues that make it impractical. The first regards the score estimation itself: usually, data at our disposal comes from high density region of Pe, that is the estimation of E., hence of the score, will be inaccurate outside such areas. The problem is that the initial condition (e.g. noise) of the SDE is usually located far from data. An imprecise drift will critically affect the generation process, leading to unpredictable outcomes. The second regards the difference between the PDE and the practical implementation through equation 40 The generation problem convex in probability space, i.e. P. is the unique asymptotic stationary solution, but the rale of convergence of the law of X, is critically related to the particular pt in study, in particular in relation with multimodality and slow mixing. We will discuss in details about this issuc in Section 4
The next step towards state-of-the-art score-based diffusion is the following lemma Anderson Lemma 3.2. Any SDE in the form
1982
dX, = ((Xr,)))L g()dW,,
Xo ~ Pi, Xx
(50)
with solution X, p(x,1) 1) admils an associated reversed SDE
dX, - - /())2 log p(a, s))ds g(s)dW,,
XT p2, Xo p
(51)
where ds is a negolive infinilesimal lime slep and flows backward from T to 0. By convention, equation 50 is also called forward SDE and equation 51 backward one.
Exploiting this result, we can define a score- based diffusion model: Definition 3.5. A score-based generative model is the backward SDE equation 51 where P1 = P. and
P2 p
Apparently, the situation is even worse with respect score matching: the score in equation 51 related the law of X,, i.e. it is time dependent and generally not analytically known score estimation already
13

---

Published in Transactions on Machine Learning Research (MM/YYYY)
issue for s. (2). The core idea in score-based diffusion is to extract information about V Voog p(:; from the forward process since the solutions of equation 50 and equation 51 have the same law, see Figure 5 By
Forward SDE (data noise)
x(0)
dx f(x, + g(t)dw
x(T)
score function
x(0)
dx [(xx t) gg2 Vx lo dt + g(t)dw
x(T)
Reverse SDE (noise data)
Figure 5: Schematic representation of forward and backward process in score-based diffussio. Image taken from Song et al. 2020b).
Definition 3.5 the forward process brings data to noise and the model to be learned is time dependent parametric vector field so(x, t). The loss used during the forward process to learn the score is:
V log p(a, 1)
(52)
where A: [0, R4 is a positive scalar weight function and T), is the uniform distribution in (0, T). After the same integration by parts used in equation 52 there is still the problem that computing the hessian is expensive in high dimension, especially if sy is a neural network. Several proposal to solve this issue have been proposed and successfully exploited, such as denoising score matching Vincent 2011) or sliced score matching(Song et al. 2020a). Another subtle issue is that generally the forward process would generate pure noise for 7 oo one could be worried that the truncation at finite time would provide an imprecise estimate of the score at that time scale, that is close to noise, and induce errors during the generative phase. This problem is attacked by practitioners via several tricks but the theoretical results in this sense are not complete.
Let us provide brief interpretation of why score- based diffusion works better than naive score matching (Proposition 3.3 Let us consider the simple case f(x, 1) = 0 and g(() = e"; the resulting forward process is perturbing data with gaussian noise at increasing variance scale(Song Kingma 2021). That is, time scale corresponds to amount of noise in this setup. We recall that the problem of naive score matching was the lack of data in low density region for the larget density. In score-based diffusion one perturbed data to populate those region and compute the score each time scale that serves bridge from ザ and the target p². A more recent class of diffusion-based generative models is the framework of stochastic interpolants (Albergo & Vanden-Eijnden 20022.. offering unified view through continuous stochastic processes, This approach is related to flow-based methods, such as rectified flow and flow matching [Liu et al. 2022 Lipman ct al. 2022), which can be seen as different parametorizations of similar principles. These methods transform a simple prior distribution (e.g., Gaussian noise) into a complex target distribution by modeling probability flow dynamics. With strong theoretical foundations and empirical success, they form the basis of state-of- the-art generative models like Stable Diffusion $ We present in Appendix B brief review of stochastic interpolation, even if not stricly related to EBMs. Moreover, it is worth mentioning flow-based generative models are similar to the Schrödinger Bridge method (De Bortoli et al.) 2021), which solves an entropy- regularized optimal transport problem. Unlike flow-based models, Schrödinger Bridges match distributions while minimizing transport cost under stochastic constraints, making them more complex to solve as they require iteratively solving coupled forward and backward processes bridging larget and initial densities.
14

---

Published in Transactions on Machine Learning Research (MM/YYY)
3.4 Normalizing Flows
The fundamental idea underlying Normalizing Flows Tabak & Vanden-Eijuden 2010 Tabak & Turner, 2013) (NF) is very close to the usual in generative modelling: to transform samples from a straightforward base distribution, often a Gaussian, to data distribution. The main feature of NF is that the transformation is performed through a series of invertible and differentiable transformations.
The core concept revolves around constructing a model capable of learning sequence of invertible operations that can map samples from a simple distribution to the target distribution. In particular, we recall the well- known lemma:
Lemma 3.3. Let us consider a random variable Z E Id and its associated probability density function (z). Given an invertible function Y = =(Z on R the probability density function in the variable Y is defined through
(y = P (v)) det Jop = pz(())))>------)-::::: (w))) det J,0(6-
(53)
where Φ-1 is the inverse of & and J, the Jacobian w.r.t. y. The density py is also called pushforward of pz by the function o and denoted by @=PZ.
In generative modelling, Pz is identified with the base distribution and its pushforward as the target, i.c. data, distribution. The direction from noise to data is called generative direction, while the other way is called normalizing direction- - data are normalized, gaussianized, by the inverse of Φ. The name Normalizing Flow originates from the latter. In fact, the mathematical foundation of NF is reduced to Lemima 3.3
The whole problem reduces to design the pushforward in a data driven setup, that is where we just have a datasct X' of samples from the target and no access to the analytic form of p.. In order to link NF with other generative models, let us denote with de with A E θ the parametric map that characterizes the pushforward Po PP. In practice, this map is usually a neural network and Po will implicitly depend on it. The optimal parameters 0 are chosen to be solution of the following optimization problem:
0r = arg min (p.(() || po(x)) = arg max E. [logg po(x)) dEe
(54)
OEN
As already stressed, this formulation in term of maximum log- lig-liielhhodd is equivalent to cross-entropy min- imization for EBMs. As for VAEs, the analytical form of pe is not known: in NF is implicitly defined through the pushforward. This issue is attacked using Lemma 3.3 to rewrite the right hand side in equation 54
3.3
as
arg arg max E. [log po (x()) = arg max E. Pz (0, (v)) + log del ( dEe
(55)
The likelihood of a sample under the base measure is represented as the first term, and the second term, often referred to as the log-determinant or volume correction, accommodates the alteration in volume resulting from the transformation introduced by the normalizing flows. After this manipulation every addend inside the expectation is calculable the map & and the noise dist.ribution pz are given (e.g. a gaussian). As usual, the expectation can estimated via Monte Carlo using the finite dataset X' at our disposal. Any gradient based optimization routine can be then exploited to optimize 0. During training, the model adjusts the parameters 8 to bring the transformed distribution in close alignment with the true data distribution.
The main limitation in NF is that the pushforward map must be bijective for any O. Not only that: both forward and inverse operations are required to be computationally feasible to perform generation and normalization. Furthermore, the Jacobian determinant must be tractable to facilitate efficient computation. These requests constrain the possible neural architectures that one can use to model Φa. The following lemma provides a docisive tool in this sense.
Lemma 3.4. Let us consider set of M bijective functions (f If we denote with f = fM ff their composition, one can prove that f is bijective and its inverse is
f- f- ff ....o fM11 o fnl
(56)
Moreover, is we denote with x4 Sio.... f1(2)= = =si o... S and y = IM, we have
M
det Jof = II
det (x:)
(57)
15

---

Published in Transactions on Machine Learning Research (MM/YYN
fi(za)
fe(sa-1)
2a po(
24 ~M (2
2k pb(*k)
Figure 6: Schematic reprosentation of Normalizing Flows, image taken from httt:t/////otttoc...aall.
Implementation of Maximum Log- Likelihodd
Cross Entropy Minimization
EBM
atg mingge [. + log Ze
Latent Space
VAE
arg maxg,0 E 1 Ex
iirr log Po(=:)2;
> go ll ((=))
Minimax Came
GAN
arg min, maxg E, [log (2) + Ep, [log(1 - (Ca((())))
Implicit via Trausport-Diffussee 0,p(e, V. ((ba(a, () + ese (a. ())(())) (Vpxz))
Equation
SBD or S1
Volume Correction Factor
NF
arg maxage E. [log (ó, (w) log J,0 (v)l
Table 1: Comparison of implementation of maximum log-likelihood for different generative models.
Exploiting this factorization result, the strategy is to compose invertible building blocks (de); to construct function do that is sufficiently expressive. In general, the architecture of Normalizing Flows encompasses various transformations (see Figure 6), including simple operations like alfine transformations and permu- tations, as well as more complex functions such as coupling layers. Common flow architectures include RealNVP, Glow, and Planar Flows, each introducing unique ways to parameterize and structure transfor- mations[[obyzev et al., 2020).
Regarding baawbacks of NF, one significant limitation lies the computational cost associated with training NF, particularly model complexity increases. The requirement for invertibility and the computation of determinants of Jacobian matrices contributes to the time-consuming nature of training, especially in deep architectures.
The architectural complexity of NF poses another challenge. Designing an optimal structure and tuning parameters may prove challenging, necessitating experimentation and careful consideration. Moreover, they may face challenges in scaling to extremely high-dimeensional spaces, limiting their applicability certain scenarios. Despite their expressiveness, NF may struggle to capture extremely complex distributions, requir- ing an impractical number of transformations to model certain intricate data distributions effectively. Another degree of freedom is the choice of the base distribution Pz, which can impact NF performance. Using base distribution that does not align well with the true data distribution may hinder the model's ability to accurately capture underlying patterns. Training NF is observed to be less stable compared to other genera- live models, requiring careful luning of hyperparameters and training strategies to achieve convergence and avoid issues like mode collapse. Lastly, interpreting the learned representations and transformations within NF can be challenging, which an obstacle for straightforward comprehension of how the model captures and represents information.
3.5 Comparison with EBMs
In this soction, we present a summarized comparison of EBMs and the other generative models. First of all, the main similarity is the objective: maximize the log-likelihood is the general aim. In Table D we present how this task is instantiated case by case. In generative models, there exists inherent trade-
16

---

Published in Transactions on Machine Learning Research (MM/YYYY)
off between the model's ability to generate data and its alignment with real-world data. Essentially, the paradigm followed in each optimization step involves two key stages: (1) the generation of data using fixed model, and (2) the evaluation of the model's performance by comparing the generated ("fake") data with the actual dataset. This dual-step process is universally applicable, albeit with variations in implementation. It represents an interpretation of generalive models as a balance bet their discriminative and sampling capabiliiiis. The key difference among generative models lies in how they learn. Normalizing Flows and Generative Adversarial Networks directly learn a deterministic mapping from data noise, while VAEs, diffusion models, and EBMs rely stochastic sampling process. Deterministic methods can be more efficient but are prone to amplifying errors from finite datasets, whereas stochastic approaches, like Score- Based Diffusion, benefit from noise as a regularizer. The choice of noise level in diffusion models and latent structure in VAEs remains an open challenge. Most generalive models are bidirectional, except for GANs, where invertibility is absent. Recent models, such as Score-Based Diffusion, improve fidelity by leveraging information from the noising process. This suggests that tools from Mathematical Physics, particularly stochastic processes, could provide deeper insights into generalive modeling and drive future advancements.
Now, let's delve into more detailed mathematical comparison, with focus on Energy-Based Models Specifically, we demonstrate how, in certain cases, other generative models can be interpreted as EBMs:
For GANs, if the discriminator is (x)) x e- Ugr (x), we immediately recover the term E.[U)]. The training of the generator correspond to learn a perfect sampler, and resemble the use of machine learning to improve MCMC in computational (Song et al. 2017
For SBD and SI, if the score is modelled by (2, x - -U (2:) the law of the process solution of the SDE is a Boltzmann-Gibbs ensemble by construction. Thus, the strong analogy is related to the constrained structure imposed to the law of the bridging process between the noise and the data, forced to be a BG ensemble. Regarding the loss, since the model is trained on Fisher divergence or on the interpolants, there no direct analogy between the losses.
For NFs, if de is the map associated to flow that brings Xo px((0) x e Uo((m) to X1 Pa: then the term E. [Uo] present in EBMs is analogous to E, [log Pz (2 (v)) for NFs. In practice, if the composition of Pz with the normalizing flow can be written as an EBM, there is no difference between the models. This is of course not true in general it not given that for any 0, a composition of the inverse map oy and pz can be always associated to an EBM parameterized via Ug.
For VAES the situation is a bit more intricate because of the ELBO reformulation. A possible interpretation towards EBMs is to think about encoder and decoder as a forward and backward processes from data to a latent space (possibly independent features, similarly to gaussian noise).
One could imagine po (2z)(2/(") and qor (2/1z;) as EBMs that have to match with some constraint the z space. In fact, the original EM steps represent an alternating optimization, where 0 is not related to 0', In this sense, VAËs tries to match the forward and backward processes, similarly to SBD where they are the same by construction of the score.
When comparing the theoretical properties of generative models with EBMs, several aspects are worth to be stressed, such as approximation properties, sample efficiency, and the ability to handle manifold data.
Approximation Properties: One of the key distinctions between models is their ability to ap- proximate complex distributions. Normalizing Flows offer excellent approximation properties due to their ability to exactly model any distribution through a sequence of invertible transformations (Pa- pamakarios et al.) 2021). However, these transformations can become computationally expensive as the complexity of the data grows. On the other hand, EBMs also have strong approximation abilities but rely on the flexibility the energy function. EBMs can approximate complex distributions well, but the necessity of calculating the partition function Z and performing sampling (e.g.. MCMC) makes this process more complex, especially for high-dimensional spaces.
17

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Sample Efficiency: The efficiency of generating high-quality samples differs among models. GANs are typically faster in generating samples compared to EBMs, which require iterative sampling meth- ods such as MCMC. Diffusion models and flow-based methods, while offering impressive generation quality, can also be computationally expensive in practice, although recent advancements in cffi- cient. sampling techniques have improved this almost beyond GANs performance (Ma et al., 2022; Liu et al., 2022; Tong et al. 2023). NFs can generate samples quickly once the transformation is learned, but their scalability and sample efficiency degrade with the dimensionality of the data.
Ability to Handle Manifold Data: Models like VAFs (Zhng et al. 2022 and NFs (Köhler et al., 2021) perform well on manifold data due to their explicit modcling of latent spaces and invertible mappings. EBMs, due to their flexible energy function, can also handle manifold data but require careful design of the energy function to ensure that the model appropriately reflects the underlying structure of the data. (Arbel et al. 2021). Diffusion models naturally handle continuous data spaces and can be adapted to deal with manifold structures by carefully designing the noising and denoising processes Pidstrigach) 2022).
In conclusion, generative models can be seen as bridges linking a simple souree, like noise, to real data. Just as constructing a physical bridge requires stable loundations, building a generative model involves statistical analysis of the dataset and selecting an appropriate noise source. Data preprocessing, akin to adjusting docking configurations, is crucial for proper alignment. Different generative models suit different data structures, much like roads adapt to terrain, with the goal of maximizing oog-likeihhod to ensure an effective connection between source and target distributions.
3.6 Why (and why not) EBMs in practice
First of all, key advantage of EBMs over other generative models is their ability to define an explicit probability density function in terms of an energy function. Unlike models that indirectly parameterize a distribution (o.g., VAEs, GANs, or diffusion models) or impose structural constraints to ensure tractability (e.g. normalizing flows), EBMs offer a more flexible framework in which the probability of a sample is directly related to its energy through the Boltzmann distribution. This explicit formulation, assuming a computable partition function Z along training (cfr. (Carbone et al. 2024ab): ) allows EBMs to provide nor- malized probabilities for arbitrary data points, making them particularly suitable for tasks such as anomaly detection (Zhai et al., 2016: Yoon et al. 2024). likelihood-basel evaluation (Du & Mordatch) 2019), and prob- abilistic rcasoning (LeCun et al., 2007). ). Unlike normalizing flows, which rely on invertible transformations, EBMs impose no such constraints, making them highly expressive. Moreover, they naturally generalize many probabilistic models used in physics and machine learning, providing strong theoretical foundation that connects them to statistical mechanics and variational inference (Huembeli et al. 2022 Chong & Courville, 2024). Because energy functions directly shape the probability landscape, practitioners can analyze how different regions of input space contribute to model confidence, uncovering structured relationships in the data, like manifold identification (Yang & Jil 2023 Arbel et al., (202)) this fact, together with the di- reet probabilistic content, makes Energy-Based Models more interpretable with respect to other generative models. This property is particularly valuable in scientific applications and settings where understanding model decisions is as important as predictive performance, for instance in molecular graph generation {Liu et al. 2021), molecular reactions (Sun et al.) 2021), text modelling (Yu et al. 2022). agent policy regulation (Sharma et al. 2021), predictions of human behaviours (Pang et al.] 2021), protein modelling (Frey et al.
2023), high-energy physics (Cheng & Courville| 2024), statistics (Zhang et al., 2022), drug discovery
et al. 2023), ; out-of-distribution detection (Liu et al. (020),, and many others.
While EBMs offer significant advantages in terms of explicit probabilistic modeling and interpretability, they may not be the ideal choice when efficient and fast sample generation is the primary goal. The need for methods like MCMC for sampling mcans EBMs can be computationally expensive and slow, particularly in high-dimensional spaces, see Section 4 In lasks where real-time generation is crucial such as image synthesis, text generation, or interactive applications- mooells like GANs, diffusion models, or based models are often preferred. These models are capable of producing high-quality samples in a much more efficient and faster manner, making them better suited for scenarios where speed and scalability are essen- tial. Moreover, other generative models tend to outperform EBMs in terms of sample quality. For instance,
18

---

Published in Transactions on Machine Learning Research (MM/YYYY)
GANs are well-suited for applications that prioritize high-quality generative performance without the need for in-depth probabilistic insight.
There have been tentative hybrid approaches attempting to combine the strengths of EBMs with other gen- crative models, such as diffusion-baseh models and flow-matching techniques (Du et al.] 2023 Chao et al. 2024). These hybrid models aim to leverage the sampling efficiency and generation quality of diffusion models or normalizing flows, while also incorporating the probabilistic grounding and interpretability of EBMs. By combining the advantages of both paradigms, these approaches seek obtain the best of both worlds -offerinf more efficient generation without sacrificing the explicit probabilistic framework provided by F.MMs.
Take Home Message: EBMs are very useful and preferable for tasks that require detailed probabilistic un- derstanding and interpretability. On the contrary, they are generally less practical for rapid, large-scale generation or when interpretability is not a major concern.
4 EBMs and sampling
The challenge of sampling from the Boltzmann-Gible (BG) ensemble arises in statistical mechanics, par- ticularly when dealing with complex systems at equilibrium. This ensemble encapsulates the probability distribution of states for a system with numerous interacting particles at given temperature. The primary obstacle in that context lies in the exponential number of possible states and intricate dependencies among particles, rendering brute-force methods impractical for large systems. A similar difficully is encountered during EBM training, since the computation of the expectation Eg requires the ability to sample from a Boltzmann-Gibbs density.
Let us restrict to the case in which the energy (x) is defined on R4, that corresponds to continuous states in Statistical Physics. Any proposed techniques to efficiently sample from PBc(z) = ex=(-U(<=)// can rely just on U(x) or on its derivatives, even if the computation of many iterated derivatives can be expensive in high dimension. The estimation of the partition function or the shape of the energy landscape are in general unknown on the contrary, they are the unknowns. Methods as rejecting sampling(Casel et al., 2004) cannot be used since one has usually access to (zx) and not to the normalized density (BG( Since the advent of computational science, sampling has been attacked with many methods a complete and exhaustive review of the existent methods would lead us off off-topic. In this Section, we will highlight three common routines for sampling from BG enseble: and lastly Metropolis Adjusted Langevin Algorithm, a sort of fusion of the first two.
and Unadjusted Langevin Algorithm.
Let us better define the mathematical setting. We consider a space Ω S Rd and a discrete sequence (tx)x20 C N. Then, we consider Xt, == Xx to be a stochastic process in and discrete time. For the sake of simplicity we will always consider absolutely continuous densities with respect to Lebesgue measure.
Definition 4.1 (Informal). Sampling from a BG ensemble consists in defining the process Xş such that ET > 0, not necessarily unique, for which XT ~ exp(-U()))/
Once we manage to define such a process, and implement it in practice, we have solved the problem of sampling from a BG ensemble. A possible implicit way to define such stochastic process is via a transition kernel. Suppose we are interested in the law of the process X at time k + 1, that we denote p(Xx+1) with an abuse of notation (n.b. analogous of p(z,t) in the context of SDEs and Fokker-Planck equation). By definition of conditional probability, there exists function T D - R4 such that
p(Xe+1) = /
Xo) II dx. 1=0
(58)
This equation asserts that any properly, uniquely defined by the law p(Xx+1) of the system at time k 1, depends on the system's state at any 2 O. Generally, this strict constraint is relaxed by imposing Marko- 2013), which is the property of the transition kernel to depend solely on the present state X3 and not on previous states, i.c.
X ..,, Xo) T(Xe+1 | X(+1)
(59)
19

---

Published in Transactions on Machine Learning Research (MM/YYYY)
The sequence (Xx)n 0 is called Markov chain if the associated transition kernel is Markovian. The question now is how to design such chain to solve the sampling problem. Traditionally, it is simpler to identify a transition kernel for which (B)(x) is the unique stationary distribution, i.e,, p(Xx) = PBC(I) for any n > T in Definition 4.1 Moreover, the integral definition equation 58 is not suitable for applications since one usually evolves X and not its law. Typically, time evolution for the process, namely an explicit mapping X F())).
required that. is associated with an explicit
For historical reasons, let present the most famous procedure to build the required sampling stochastic process, namely the Metropolis- Hastings algorithm algorithm(Metropolle (Merrooolis et al. 1953 Hastings
1970). Such techniques
stand out as foundational Markov chain Monte Carlo (nnrriiAnndiiu et al. 2003 method. Here, we provide its definition and a sketch of the proof of its properties.
Definition 4.2 MMetropolis-aastings (MH) algorithm). Let us consider an initial condition Xo ~ Po(x), where Po(x) simple to sample from (e.g. Gaussian or uniform). Let us consider a conditional probability distribution g(Xx+1|Xxx, also called proposal distribution, defined on the state space Ω and let pBa (x) exp(-U(x)))Z the BG ensemble we would like to sumple from. Starting at n = O, we define a Markow chain Xs via the following repeated steps:
1. Given Xk, generate a proposal X using the time evolution prescribed by T.
2. Compute the acceptance ratio
A(X (p) R+1 Xx) = arg min { Pno
k+t10 b+t1)
(p) )g ((()
(60)
PBG (B (P) [xx)
Sample real reumber u U[0 1]. If u A(Xi, X accept the proposal and set X otherwise, refuse the move and set X Xt. Then, increment to n n+? 1.
k+11
Proposition 4.1. The Markov chain X2 defined via MH algorithm has PBG(x) as unique stationary distri- bution, i.e.
PBC(x) = S (pp((r)))))c(()))))) Vx, E Ω
(61)
where (ala)) the transilion kernel of MH algoritho..
Proof. We have show that (1) (Bc(x) is a stationary distribution and (2) it unique. Regarding (2) we advocate to geometric ergodicity (Mengersen & Tweedie 1996). We present the proof of property (1): firstly, it is equivalent to detailed balance conditionRRobert al. 1999)
pno(z)TMm(Z,d) x') = pno(x (anocse)Tunn''ee Ve, En
(62)
The transition kernel is by definition
Tun x') = -000)==(((((((())))))) 5(x x') (1-1/ A(x, s)g(s)))ds))
(63)
where the first addend takes into account the case of accepted move, while the second of the rejected one. Actually, for z' the detailed balance condition is trivially true. Then, for x # x' compute the left hand side of equation 62
Mun(,)) = (1=
PPc(()g((' (x) arg min { (Bc()')g(a())
(64)
pno (x)g(>
arg min (PEC (/ (x), Peo(d (eo(2")g(aaa))
The right hand side is symmetric with respect to swap of with x', hence concluding the proof.
20

---

Published in Transactions on Machine Learning Research (MM/YYYY)
In practice, convergence is considered achieved when the acceptance ratio is consistently close to 1. In such cases, every newly generated proposal can be regarded as an independent sample obtained from PBG-
Despite its popularity, the Metropolis- Hastopolis-Hasttngss algorithm has some limitations. It is sensitive to the choice of the proposal distribution g and its parameters, and improper tuning may result in inefficient exploration. For instance, in the so-called random walk setting, g is chosen to be Gaussian transition kernel, and its variance is a critical hyperparameter in this case, Moreover, the algorithm generates correlated samples, impacting the independence of successive samples and hindering accurate estimation even after convergence. Convergence may be slow in tigh-dimensional spaces, requiring numerous iterations. In such setups, the algorithm's 's performance is influenced by the initial state, and initial points far from the basin of the target may impede efficient exploration, leading to an acceptance rate close to zero. Another issue pertains to multimodal distributions, especially those with widely separated modes. They pose a significant challenge for the Metropolis-Hassee Hastings algorithm because, depending on the choice of g, jumps between modes can be very rare and may necessitate a very long chain practically observe convergence.
The second class of Markov chain we would like to review are the Langevin-based algorithms. The basic idea is very close to the definition of naive score-based diffusion in Proposition 3.3 For the sake of simplicity let us fix the state space f2 = Rdd.
Proposition 4.2. Let denote with dW, Wiener process. Under Assumplion
2. namely
3a E IR, and a compact set C E IR x. --UUx) 2 a|x?? Va E Rd C,
(65)
the Longevin SDE
= -VU + V2dW Xo Po
(66)
have global solution in law and is ergodic [Oksendol 2003 Mattingly et al. 2002 Talay & Tubaro 1990)
For any inilial condition po(x)) such solution is pBG(x).
Given this result, one can define Markov chain based on the time discretization of such SDE and use it for sampling Parisi 1981). Such procedure is commonly named Unadjusted Langevin Algorithm (ULA) (Roberts)
& Tweedie 1996).
Definition 4.3 (ULA). Given a time step h > O and a set of i.i.d. gaussian variables {Ex} ~ No Ia), the Unadjusted Langevin Algorithm (ULA) is the Markov chain defined as
X = Xx NVV,, (Xx) + 22h
Xo ~ Poo
(67)
for l E N.
Under Assumption 2.1. the Unadjusted Langevin Algorithm (ULA) is crgodic and possesses a unique global solution. An advantage over the Hasttigs (MH) algorithm is that the chain is uniquely defined via (x), and no proposal distribution is necessary. However, it is well-known that ULA represents biased implementation of Langevin dynamics Synaaics(Wibisonog 2018h. For a nonzero time step, the global solution is Pbius + PBG. Let us illustrate this point with simple example.
Example 4.1. Let Uxx) = (x =n)=))=>-(=== 1)/2 (og]det(2+E)))
that is BG ensemble is gaussian with meon μ and covoriance matriz E. The associated Langevin SDE is also known Ornstein- Uhlenbeck (OU) process(Uhlenbeeck Uhlenbeck & Ornstein 1930), having a linear drift as peculiarity:
-x- - wddt + V2dW
(68)
It is possible to write on explicit solution using Ito integral, namely
X.- p - (Xo--)++ + (Is-e- 1)8 Z
(69)
for any 2 0 and where Z ~ (O,, Ia) indipendently from Xo. It means that the law of the process converges eaponentially fast to N(, E). The associaled ULA is
X p (Ia 11-11) (Nx 14) / 2h
(70)
21

---

Published in Transactions on Machine Learning Research (MM/YYN
and the corresponding solution in law is
X* 1 A (Xo (1) + 2h (Ia AB) (Ia ) Z
(71)
where An -- Namning Amin (E) > 0 the minimumn eigenvalue of the covariance matriz, for O < h < Amin (E) we have limg- A = O. Thus, for k + 0c
X* 4 + v 2h Ia Ai)
(72)
This means that the limiting measure for ULA is not PBG, but
Phins (x) =N 11, la I-1
(x)
(73)
This example illustrates that the Unadjusted Langevin Algorithm (ULA) exhibits bias even for a very simple target density, This phenomenon has been recently analyzed mathematically (aaitticcallyiiisoonooo (Wibisono. 2018). The physical interpretation is that detailed balance is broken by construction. Let elaborate on this point: in Proposition 3.2 we demonstrated how Stochastic Differential Equation (SDE) can be associated with Partial Different Equation (PDF). The specific case studied in this section was previously analyzed in Proposition 3.3 Specifically, the Boltzmann-Gibbs (BG) density is the unique minimizer of the Kullback- Leibler (KL) divergence functional DKI. (p || PGR). Moreover, the Fokker-Plank PDE corresponds to the gradient flow in P(RR) with respect to the 2- Wasserstein distance W2 (Jordan et al. 1998). we split equation two substeps
X X++ Xx - (VV((x))
(74)
Xx+1 Xx+1 + 22Ex
associate each of them to a precise operation in probability space. In particular, denoting with p the law of X;, obtain
Pk+ (Va-AVU)
(75)
Pk+1 /N,0,, * Pk+
We recall the decomposition of the Kullback-Leibler (KL. divergence as (KL || PCB) = = PC( H(p) In equation 75 the first step involves the forward discretization of gradient descent on (pB)) = Ep while the second step represents the exact gradient flow for negative entropy in probability space. Therefore, the Forward-Flow method in probability space. The bias arises because the forward
CLA also referred gradient descent docs not correspond, in probability space, the adjoint of the flow at iteration 1+ One possible solution is to use forward-backwarr combinations, referring to proximal algorithms Parikh el al. 2014). In particular, the Forward-Backward (FB) implementation for Langevin dynamics would be
Pk+ = (Ia hVU) #Pk
{
1 W2 (p: (btttt) }}
(76)
Pk+1 arg min
-H(() +
PEP
2
Similarly, the Backward-Forward (BF) version
Pk+ = ((Ia + -VU))1)) Pk:
(77)
Pk+1 = cXPpart (-hVlog
where exp is the exponential map. Unfortunately, both FB and BF are not implementable in practice, except for the trivial case of gaussian initial data and target PBG. The heat flow (the step + 1/2) is the most problematic since it concerns steps in probability space. Neither forward (n.b. beyond one iteration) nor
22

---

Published in Transactions on Machine Learning Research (MM/YYYY)
backward are usable. As side note, one could imagine to directly perform single forward or backward step on the KL. divergence. Unfortunately, the encountered issues are the same one has for the heat flow, i.e. the hard task appears to be the actual implementation of forward or backward routines in probability space. In conclusion, ULA appears to be the simplest time discrctization of Langevin dynamics, since it is practically implementable in general, hence very used for sampling from a BG ensemble. However, is known to be biased and other methods are studied to climinate, or at least reduce, such bias.
One possibility we would like to review is Metropolis Adjusted Langevin Algorithm Soforithm/Grenanin Grrnander & Miller 1994h (MALA), which represents a sort of hybrid between MH and ULA.
Definition 4.4 (MALA). Metropolis Adjusted Langevin Algorithm is a particular case of MH algorithm 4.2 where the proposal distribution is the transition kernel associated to ULA equation 67 nomely (for x Rd)
1 ||'z - -x hx()11 g(x'l x) (2Th) 4h
(-1
exp
(78)
On the other hand, one can interpret MALA as a corrective measure for the breakdown of detailed balance in CLA. While the Metropolis- Hastoppllss-Hasiinss algorithm inherently respects detailed balance, implying that MALA becomes asymptotically unbiased for a large number of iterations as k + co, certain challenges persist. A primary concern is the sensitivity to the choice of the step size h during the discretization of Langevin dynamics, significantly influencing the efficiency of sampling. When h is too small, it can lead to poor explo- ration and potentially a very low acceptance rate, while an excessively large h can lead to instability of the chain. Determining an optimal h lacks a general rule, contributing to MALA introducing bias in samples, particularly evident when the target distribution features sharp peaks multimodal structures. This bias introduces potential inaceuracics in statistical estimates.
In practical applications, MALA may exhibit random walk behavior, especially when step sizes are inade- quately tuned, resulting in inefficient exploration and sluggish convergence. The algorithm's performance is further contingent on the choice of initial conditions, and beginning far from high-probability regions may necessitate a considerable number of iterations for meaningful exploration. Additionally, MALA may struggle to adapt to changes the geometry of the target distribution, particularly when facing varying curvatures or strong anisotropy.
While various more advanced algorithms exist, they often build upon the foundational concepts discussed in this section. Notable among them is Hamiltonian (or Hybrid) Monte Carlo[Duane et al. 1987 (HMC), an ad- vanced MCMC method inspired by Hamiltonian mechanics. HMC utilizes fictitious Hamiltonian dynamics to propose new states, enhancing exploration, especially in high-dimenniooal systems. Gibbs sampling (Geman) & Gemaan 1984), another MCMC approach, iteratively samples from conditional distributions given current variable va values on a single dimension, proving effective, particularly in high-dimensional spaces. Parallel Tempering, or Replica. Exchange(Swendder & Wang] 1986). involves rumning multiple chains different lemperatures concurrently, with periodic swaps between neighboring chains to facilitate improved explo- ration.
In general, most methods aim to find chain that produces independent samples from Boltzmann- ensemble, particularly when run for extended periods. A critical issue is measuring the effective bias due to the truncation at finite time of the chain, posing challenges for convergence towards the asymptotic PBG. Unfortunately, few general results are available, and they are often limited to specific BG ensembles, such as Gaussian or log-concave densities. This becomes particularly problematic in the context of Energy-Basce Models (EBM), as outlined in Remark 2.2 where sampling from BG distribution is required al each step of parameter optimization.
5 EBMs and physics
In this section, we review the Boltzmann-Gibbs ensemble in the context of equilibrium Statistical Physics, deriving it through the maximum entropy principle. This approach provides a mathematically precise yet concise justification for the Boltzmann distribution, which underpins the probabilistic structure of EBMs. Our goal is to offer compact introduction to key statistical physics concepts, such as free energy and partition functions, that recur in the discussion of EBMs. While this derivation is not directly related to the
23

---

Published in Transactions on Machine Learning Research (MM/YYY)
practical training of an EBM, we believe it is important to equip the reader of this review with the necessary physical and mathematical tools to understand recurring concepts in EBMs, such as the Boltzmann-Gibles distribution, entropy, and free energy.
The first step involves the derivation of the Boltzmann-Gible ensemble. Here, we present a derivation basod on information
1957a b), offering a posteriori physical interprelation of the quantities we will manipulate. Alternative methods of proof are also (aalavbl:[alllaootti 1999). Consider a physical system whose state is uniquely determined by a variable I E Q Rd, where S2 often referred to as phase space. The connection with information theory is linked to the fundamental problem of Statistical Physics: describing system as statistical ensemble, i.e., identifying an observation of x as sample from an underlying PDF p. Like classical statistics, P contains a wealth of information about the system, particularly its global properties.
In Physics, this dichotomy translates into the microscopic versus macroscopic realms. Let's envision a simple thought experiment: picture large city where each of the N inhabitants is given fair coin, with the coin's 's state represented by our variable x {-1,13A, Twice a day, everyone has to flip their coin. If were omniscient, there would be way to predict the state with no error (i.e., the microscopic state) and derive any global (macroscopic) property, such the sum product of the state values at each flipping event, with no error. However, in reality, nobody could achieve this; we rely on statistics, the central limit theorem, and so forth. In other words, we know the probability density p(x) from which the process is a sampled event. For instance, if is large enough, we expect the average of the state vector to be 0 for any flipping event, and we can deduce so directly from p.
In Statistical Physics, each coin represents a component of system, such a particle in gas, for which direct mcasurement of x is unattainable. The goal is to determinc p that standard statistical tools can be used to analyze global properties. The challenge that makes Statistical Physics more complex than the simple example above is that the dynamics of individual components can be unknown and inaccessible. Additionally, interactions between components can make the identification of p challenging, even if the underlying microscopic dynamics are known.
To address this issue, we recognize that, before formulating any physical model, we need some motivated assumptions constraints or information regarding how the system should behave, at least on macroscopic level. This is the bare minimum; without any information about system, it is impossible to provide any meaningful analysis. Thus, adopting claim of epistemic modesly, one can state that we aim to select the model compatible with such constraints that maximizes our ignorance about the system. The mathematical translation of such idea is the Principle of Mazimum Entropy-
Assumption 5.1 (Principle of Maximum Entropy). Let consider the unknown p: Ω - R4 that describes the probabilily distribution of the states. We assume p to be absolutely continuous w.r.l. Lebesgue measure without loss of generality. Given vector field F ; Ω + Rd, A E Rd and a PDF a set of construints is any component-wise (in)equaliyy
T [x| = J Fa(e))(o)de Su A
k= 1......,d
(79)
where the symbol So be equality or incquality. The Principle of Maximum Entropy is
p = arg max ((T)
(80)
REP(M) A
where (p) the usual differential entropy, efr. equation
This variational formulation identifies the "ignorance' about the system with the entropy associated to It has been proven that the entropy can be characterized in an axiomatic waydAczál et al. 1974), so that the definition of differential entropy is unique with respect to certain properties. For the sake of the present. treatment, let us motivate the Maximum Principle with simple example.
Example 5.1 (Maximum principle on an interval). Let us consider an interval Ω = [a C R, with Vol(?) _ dx. Moreover, the only constraint is that p can be normalized and is positive. Thus,
24

---

Published in Transactions on Machine Learning Research (MM/YYY)
have Io|=]] Sa p(x)drr and
TEP( p arg max ((T)
(81)
1a #>0
We can use Lagrange multipliers to solve constrained optimization problem, solving the unconstrained oplimization of the Lagrangian
J(T) H(T) + No (f
(82)
To find stationary points we can compute the first variational derivative with respect to a and finding its roots, namely solutions of
(J(T) log -1++1110 7
(83)
that is p(r,, exp(>o 1). To find do we substitute p(ar, Aa) into the constraind, yielding Ao - log((b (0-a] a). In conclusion, p(z) 1/(6-1100))) a), which also satisfies the positivity request. We have just to check that such stationary point is mazimum. The second variation of J(x) evoluated in the stationary point is
((T) 8T
1 <0 p(x)
(84)
m=p
Hence, conclude that p(z) is mazimum. If Q is discrete such derivation be easily generalized. The interpretation is straightforward: imagine that Q is the event space for some random process. Without any knowledge, the simplest possible model is the one that associales the same probability to all the possible outcomes.
At this point we have all the ingredients to present the derivation of Boltzmann-Gibbs ensemble.
Proposition 5.1 (Boltzmann-ibbee ensemble from Maximum Entropy principle). Given U(x) that satisfies Assumption 2.1 and a constant U*, the Boltzmann- Gibbs PBG = (a))(a) /Z, where A1 > O, is the unique sobulion of the varialional mazimizzaion problem equalion 80 where the set of constraints
Iu[T]] = / (x))dx=1
(85)
[m) = ((x)x(r)dr== U
Proof. The proof proceeds similarly to Example 5.1 The constrained optimization problem equation associated to the unconstrained one
80 is
arg max ( = arg max H(T) + No (((o))d--1) + A (( TEP(R)
-
(86)
#EP(O)
where No A are Lagrange multiplier. We compute the first variational derivative and find its roots
(T) log No (x))==
(87)
That is, p(z) exp(Ao + (z) 1). This means that Åo can be incorporated in the normalization factor, namely the partition function Z-1 = exp(Ae 1), and determined via the constraint Tolp] = 1. While Y1 is implicitly determined by J To check that such solution is a maximum, we compute the second variation obtaining a result analogous to equation 84
The remaining issue is the identification of A with 3 1/kgT, related to temperature T and Boltzmann dimensional constant hg 1.23 10 10-28 J. K-1 The reason that if we interpret the Boltzmann-Gibb ensemble with an equilibrium ensemble, the derivation via Maximum Entropy principle must be coherent with Thermodyynamics(Adkin 1983 A complete survey such field would lead the present treatment
25

---

Published in Transactions on Machine Learning Research (MM/YYYY)
off-topic. The take home message is related to different interpretation of the unconstrained optimization problem equation 86 namely
8
[(T)+ + 20 I x(z)dr + An I ((z)m())dz)
(88)
In particular, the following lemma holds true:
Lemma 5.1. Marimam Entropy principle and its variational formulation are equivalent to
Minimization of energy functional U = (x)) under the constraints
[o[=] /. x(x)dr
(89)
I 1 H(T)(( H >
Minimization of Helmhollz Free Energy functional F = Ū (HH(",,, where T > O is the usual
thermodynamic temperature, under the constraint
To To( = x(x)dr=1
(90)
Proof. The proof is just related to a redefinition of the Lagrange multipliers, leading to the same eq. the stationarity condition. For the minimization of energy, one define Xi = -1/A1 and X₂ = -oo/Aa A1, where the sign is just convention, obtaining
87 for
5 (()(())+ Xo m<))dx+ / ((z))())dz)) =
(91)
While for the Helmholtz Free Energy, we have just to impose the thermodynamicn constraint Fermi 201 2
OF/0S- = -T that is d1 -1/T.
Remark 5.1 (Free Energy in EBM training). Section 2. we presented the training procedure for on EBM as the KT. divergence ménimization. (g is the same class of Pe; the global minirrum in probability space corresponds to Pe Pa, t.e. KL divergence equal to zero since by definition DEI(Po I| po) O. However, if we expand the such identily, we have
log Zo + B Ja (o(r)po(r)do - H(pp) =
(92)
where we used equalion 10 and restored 3 in front of Un (n.b. pul. Kn and T in Section If identify Fa log Ze, inmedditeel notice that equation 92 is the definition of Helmholta Free Energy. In fact, the convergence of the training corresponds to have reached the equilibrium. The equality
2.
F = - - (H(T) is not true out of equilibrium - the KL divergence DKI(po || px) is zero iff Pe = Px. Moreover, it is even more clear the statement of Lemma 5.1 since
[8 f JE
log 7. +
min TEP(1)
Uo(z) H(x))
(93)
1o|m==1 P6200
and F=U-TH(T at equilibrium F is necessarily minimized in correspondence of F(po) = log Zo.
The requested compatibility between Thermodynamies and the Maximum Entropy principle in Lemma 5.1 represents the final ingredient needed to define the Boltzmann-Gibbs probability density associated with a system at equilibrium with a thermal reservoir at temperature T. For the purpose this review, would be beneficial claborate on the physical significance of EBM. Assuming we are dealing with an equilibrium ensemble, we presume that the parameters 6 in the energy Up have already been determined. Similar to physical gas where particles move within energy landscape, different datasets
individual
26

---

Published in Transactions on Machine Learning Research (MM/YYYY)
data points can be envisioned snapshots of evolving physical system. The crucial aspect is that from statistical perspective, the average energy Ü associated with the ERM must remain constant, with fluctuations suppressed as the number of components increases. An example of dynamics consistent with such a constraint is Langevin dynamics. The connection with sampling and Physics becomes evident: sampling is the process of relaxation(Evans et al., 2009) towards equilibrium. Utilizing our understanding of nature entails designing sampling routines capable of facilitating such relaxation.
We introduced the concept of free energy the Boltzmann-Gibil ensemble. Generally, computing free energy is extensively studied problem in Chemistry(Jorgensee 1989). spanning from organic Chemistry to protein folding(Dinner et al.) (000). However, the concept of free energy appears ubiquitous, extending into seemingly disparate contexts far from computational chemistry, such as autoencoders Hinton & Zemel 1993), lattice field (heory(Nico)) et al. 2021), and neuroscience(Fristnn often directly linkod to the use of a generalization of the Boltzmann- ensemble.
thermodynamic quantity minimized at equilibrium by
2009). Invariably, it is associated with some equilibrium principle,
The importance of free energy can be readily understood: the expected value of any observable al equilib- ensemble,
rium can be computed if we have access to the normalization constant of the Boltzmann- which is the partition function Z = c= However, as demonstrated in Section 2.1, computing the partition function, and consequently the free energy, is exceedingly complex using standard Monte Carlo methods for systems with many degrees of freedom, ronghly corresponding to dimension d for EBM training. Among the (etooodsSStotzz
various proposed advanced methods (Stoltz et al. 2010), the utilization the Jarzynski identity (Jarzynsi)k 1997) stands out very notable tool. Recently, an application of such result for improving the training of EBMs has been proposed, efr. (Carbone et al. 2024a and next Section.
6 Recent Developments in EBM training
In this section we summarize the most common algorithms used for EBM training and some recent developments in this area. In the first part, will discuss the most common approach to directly minimize the KL divergence loss, which is Contrastive Divergence. In the second part, we will briefly present the so-called sampling-frec mothods, namely Scoro-Matching and Noise Constrastive Estimation.
6.1 Sampling-based methods
For readers convenience, we fix the notation: in the following, pa(x)) (x(c)(x)) = (xp(-Uno((:)//4 is the EBM we aim to train. As we showed in Section 2 training an EBM reduces to perform gradient- based optimization on cross-cntropy. After some manipulation, the gradient of H(p:Po) reduces to
(p. Pe) E. [0,U]] Eg[0pUa] = -D
(94)
As we stressed in Remark 2.2 the main issue is the estimation of Ee(O,Uo)- An analytical computation is outreach for generic Up. as well as the use of numerical spline methods which are impractical in high dimension. The only possibility is to generate a set of N samples {X*), distributed as Po(s) and exploit a Monte Carlo integration, namoly
1 N = N Σ OxU)(X')
Eg(00Ub]
X po
(95)
We stress that such generation is required at each optimization step of 0. The basic idea is to couple a gradient-based routine with a Markov Chain [Liu 2001) devoted to the generation of the needed samples (see Section i for a detailed description of sampling). Without loss of generality, we present the state-of-the- art algorithm using Unadjusted Langevin algorithm (ULA) Parisi 1981) as the sampler.
As mentioned, problem encountered by standard sampling routines (such as ULA) is related to multi- modality; that is, for fixed and a general initial condition X म for the Markov Chain, there are no general results the convergence rate towards the desired equilibrium X po. However, if one were to
27

---

Published in Transactions on Machine Learning Research (MM/YYYY)
choose smart initial condition, such issue is alleviated. For instance, in the ideal case where could sample from initial distribution very close to po- In this sense, the naive approach in which the sampling routine restarts from the same "simple" distribution, like a Gaussian, for every optimization step, is not well adapted to EBM training. The question then arises: how to select an appropriate initial condition?
The idea of Contrastive Divergence Hinnon 2002) (CD) and Persistent Contrastive Divergence Tieleman 2008) (PCD) in their original formulation is use the unknown data distribution p. as the initial condition for Markov Chain sampler. This is feasible since have the dataset; that is, could simply extract some data points from it and use them the initial condition of the sampler at every optimization step. To better analyze the two routines, we present CD and PCD in Algorithms and 2 where ULA is chosen the sampling routine.
A Algorithm Contrastive divergence (CD) algorithm
1: Inputs: data points X' {af in IR;; energy model Uo; optimizer step opt D) using 0 and the empirical gradient D; initial parameters do; number of walkers N E No with N < n; total duration
K E N; ULA time step h; P E N.
2: for h: = 1,...,
K ,
do
for = 1,..,, N do
Xj Rondom (tandomSample(X))
for O.... P - do
6:
X p+1 X; NVU, (xx) + v 22
$ ~~~(00 N0a, Ta)
P ULA
7:
end for end for
8:
9:
D2 = N N Ni= OgUio (Xp) - n EE (xU))(())
empirical gradient optimization step
10:
8x+1 opt D2)
11: end for
12: Outputs: Optimized energy Uox set of walkers {
Algorithm Persistent contrastive divergence (PCD) algorithm
Inputs: data points X' {=; (=) in RR;; energy model Ug; optimizer step opt D) using 0 and the empirical CE gradient D; initial paramcters du; number of walkers N No with N < n; total duration K E N; ULA lime step h.
2: X3 = Randoo.Sample(A
for i = L....,N.
3: for k: = 1,..., K -1 do
4: D = N 1
doUse (x+)
( - -n 'I (x.)
empirical gradient optimization step
0x+1 = (fk, Dx)
for = l,,.. N do
X = hVU,, (Xt) + V2h
~N(aaa
ULA
end for end for 10: Outputs: Optimized energy Uog; set of walkers [
Let us clarify the notation. Each X used for the estimation of the gradient of cross-entropy is named "walker". Fach walker indexed by a superscript, and the function Random.Sanple((2) performs a random extraction of N points from X. In CD, the chain for sampling is reinitialized data at every cycle; in PCD, for the name, the chain is 'persistent', meaning it starts from the data just at the first iteration after each optimization step, the sampling routine restarts from the samples found at the previous itcration. Traditionally, x" is referred to as 'positive' samples, while the samples from are termed "negalive", especially in the community of Boltzmann Machines. The adjective 'Contrastive" originates from the minus sign between expectations in equation 94 the contribution of negative and positive samples the variation of cross-entropy is indeed opposite. In fact, the ODE associated to gradient descent on cross-cntropy minimization is
i = Eo( F. (QgUo)
(96)
28

---

Published in Transactions on Machine Learning Research (MM/YYYY)
This equation can be interpreted gradient descent the energy per positive sample and gradient ascent for the energy per negative sample. It corresponds to increasing the probability of data points in the dataset and decreasing it for the samples obtained from the chain. Stationarity is reached when p. = Pe, so that generated points belong to the same distribution as true data points.
The natural question that arises concerns the convergence of the algorithms. To simplify the treatment, we do not analyze the algorithms for a finite set of walkers, but we study the time evolution of the probability distribution of the walkers Ø(), Ideally, this should remove any possible spurious bias from the analysis and permit an easier analytical study. We can write down an equation that mimics the evolution of the PDF of the walkers in the CD algorithm in the contimuous-tine limit. This equation reads:
o. = aV (VUnto(z) + v - v16 -(p-p-), p-),
0) = P+
(97)
with fixed > and where the parameter at the data points: the last term in equation 97 is birth-death term that captures the effect of these reinitializations. The solution to this equation is not available in closed from (and p(t, * Pa(e) (x) in general), but in the limit of large (i.e. with very frequent reinitializations), we can show Soww<DDmingo-ain Enrich
0 controls the rate at which the walkers are reinitialized
ctt al. 2021 that
p(1,z) = (.( + aV
aV (+)) + (()))))))))))))))))))))))))))))) + -Ov (p)(z))+O(v)
(98)
As result, the gradient of cross-entropy equation 94 is
1 (x) p(t,
= =-v [ (oU)( (Ue())())) + Vp.(x)) dx + 2
(99)
= ( (8)VU VU (z) 0g ( P (x) + 0
The leading order term at the right hand side is preciscly v-1 times the gradient with respect to 0 of the Fisher divergence
1 [Uo(1) V log 1o (x)) (3))dr
2 g=
(100)
1
/
[[VU)(>)] 2AU + p. pa (x)do
2 Rdd
where A denotes the Laplacian and we used
[Uo(x) v log (z)) Rd
VUo(z) (x)()) = (1)r)
(101)
This confirms the known fact that the CD algorithm effectively performs GD on the Fisher divergence rather than the cross-cntropy (Hyvarinen 2007), similarly to score matching.
Regarding PCD, the associated PDE is equation 97 with
0. Again, the solution ((1, + (a)) in gen- eral, thus for any finite a, we have [p] * E [gU)- In other words, one cannot be sure to perform true if we were able to estimate the loss, we could observe non-monotonic
gradient descent on cross-entropy behavior. Extensions of standard PCD exploit an initial condition different from P. for the persistent chain, but such approach is plagued by the same issue regarding the convergence rate towards equilibrium. Neither Contrastive Divergence nor Persistent Contrastive Divergence perform true gradient-based optimiza- tion of cross-entropy. The bias in both methods is inherently tied to time scales: in PCD, to the length of the Markov chain used for sampling, and in CD, to the reinitialization frequency. Despite their widespread use, this fundamental limitation affects the applicability of Energy-Based Models, making their performance highly problem--ependent. A key lakeaway is that while generaling individual samples is not an issue, CD and PCD may fail to capture the global mass distribution due to the properties of Fisher divergence, partic- ularly multimodal distributions. Morcover, both methods do not provide a 'full" EBM since they do not provide any estimate of the partition function, critically affecting the advantages of EBMs that have been discussed in previous sections.
29

---

Published in Transactions on Machine Learning Research (MM/YYY)
Alternative training methods leveraging nonequilibrium statistical physics have been recently explored in CCarbone et al. 2024a]by by, offering a detailed comparison with CD/PCD and extensive numerical experiments. In particular, these approaches employ fluctuation theorems, such as the discrete-time Jarzynski equality, to correct for the bias introduced by short Markov chains and finite discretization steps in ULA. The key idea is that the paramelers 0 are evolved along a protocol {0x keeNo while the samples X and associated nonequilibrium weights Aş are iteratively generated by:
Xx+1 =x, - AVU,, (Xx) + Ex
Xo ~ Pao
(102)
ARt1 An 0041 Xx) + ax Xk-1),
Ao = 0,
where v) is defined by:
(x(2,)) Uo (x) 2 x) VU. (x) + VUo, (x:)
(103)
Despite the fact that the sampling dynamics is based on the unadjusted Langevin algorithm (ULA), which introduces discretization error and mixing inefficiency, the inclusion of the weights As allows for an exacl
expression of the model gradient:
Es. [0Ue = BldaUa (x))eA
Zo. Ze [e
(104)
EleAu]
This reweighting strategy ensures that sampling errors are corrected and enables unbiased learning of EBMs via the update:
D& - (p... p-) ElÜgUo, (Xx)e
0k+1 = 0% + DD
(105)
E Bledu]
where Nh 0 is the learning rate. This update can also be implemented with adaptive optimizers such as AdaGrad ADAM using the unbiased gradient DR. The interpretation is that reweighted samples are always in equilibrium with the time-evolving Boltzmann- Gibbs density during training, allowing to compute exactly the problematic expectation in the gradient of KI; the presence of the weights corrects the bias present in CD and PCD, More importantly, this framework allows to recursively tracking the cross-entropy via
H(pb P=) log l[e + log Zeo + [UoA).
(106)
and the normalization constant Zo. The former is principled, tractable metric of training progress, while the latter is important for interpretability features of the trained EBM, as previously discussed. These developments open new directions for scalable, statistically consistent training of EBMs, with applications evaluation and calibrated uncertainty quantification are essential. Similar
in fields where likelihood- ideas were developed later in the context of sampling, where the target density is known analytically, but no data samples are available, cfr. Albergo & Eijnden (2024): Vargas et al. (202]).
6.2 Sampling-fre free methods
From the previous treatmentm, it is evident that sampling- methods such as CD or PCD rely on approximate sampling from the model distribution po(x)) x -Uo()) and introduce several issues: (i) they require careful tuning of hyperparameters like the step size and number of iterations; (ii) the resulting gradients be biased due to finite-length chains; and (iii) the computational cost can be prohibitive in high-dimensioonll or multi-modal settings.
In response to these challenges, a class of methods known as sampling-free approaches has been developed. These methods avoid the need to generate samples from the model distribution altogether. Instead of attempting to approximate the intractable expectation over pa, they formulate surrogate objectives that are tractable and differentiable without nceding the partition function Zg. The two most prominent techniques in this family are Score Matching and Noise Contrastive Estimation, although several generalizations and related techniques exist.
30

---

Published in Transactions on Machine Learning Research (MM/YYY)
Score Matching. Introduced by Hyvärinen & Dayan (2005) Score Matching provides a principled al- ternative to maximum likelihood by directly matching the scores of the model and the data. Specifically, Score Matching minimizes the squared difference between the model score v log (x) and the unknown data score, under the empirical data distribution (r(x). For EBMs where the model density expressod as po(x) x e-Vo(z) we have Vg (o()) = -V ~vz (zt), so the objective becomes:
I
VaUo(=)) V. V.Uo(a)). 2
(0) = Eusup
This formulation is noteworthy for its independence from the partition function Zg, since the score function does not involve log Zg. As a result, the training objective is tractable and differentiable. However, this advantage comes with a major drawback: the model does not learn a normalized densily. In particular, because the loss function does not involve the log-likelihood, there no mechanism to recover or estimate Zg. Consequently, the learned model suitable for tasks such as mode-seeking or denoising, but not for tasks requiring calibrated probability density.
Moreover, the divergence term V VaUb(()) (i.e., the Laplacian of the energy function) can be computation- ally expensive and numerically unstable, especially in high dimensions when using deep neural network parameterizations. To alleviate this, [Vincett (2011) proposed Denoising Score Matching, variant where the score is estimated on noisy versions of the data:
T x o²
(D) = Exmp, (a)g((lx)
Here, (((xz) typically taken as a Gaussian corruption kernel, and is a fixed noise scale. DSM is more stable numerically and can be implemented via automatic differentiation. This formulation also plays foundational role in the development of modern score-based generative models and diffusion models, where generative sampling is achieved via Langevin dynamics or stochastic differential equations guided by learned score functions (Song & Kingma 2021
Noise Contrastive Estimation. Proposed by Gutmann & Hyvärinen 2010 NCE formulates the train- ing of unnormalized models as a binary classification task. The idea to train a discriminator that distin- guishes between real data samples æ ~ (x( and noise samples x ~ Pnoise (z). The EBM learns an energy function (z) such that the log-ratio fo(z) (z) -log Puoise (z) approximates the log odds of the sample being real versus noise. The objective function is standard logistic loss:
JNCE(() = -nmpp (Jo((())))) + Ear-paoine [log(1 (fo()))))
where a(2) = 1te-= is the sigmoid function. The resemblance with the minimax formulation of GANs is clear: the energy function plays the role of a discriminator, while the noise distribution is analogous to a fixed generator.
1
One key advantage of NCE is that it allows for consistent estimation of the partition function Zg. at least under certain assumptions. If the noise distribution (a) is known and overlaps well with pr(x), then Zo can be estimated jointly during training, as shown in Chehab et al. (2024). This contrasts with score matching methods, where the normalization constant is fundamentally unidentifiable
However, NCE has its own limitations. Its effectiveness depends critically on the choice of the noise dis- tribution: if the noise is too dissimilar from the data, the model receives little signal and may fail to learn meaningful structure. Conversely, if the noise is too similar, the classification task becomes too hard, leading to vanishing gradients. Moreover, in high-dimensional spaces, designing an informative noise distribution that supports the full data manifold is nontrivial, often requiring domain knowledge or adaptive methods.
Extensions and Other Approaches. Beyond SM and NCE, a variety of other sampling- free techniques have been explored. These include:
31

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Sliced Score Matching (SSM) (Song et al., 2020a]:: variant of SM that reduces the compu- lational cost of the trace term by projecting the gradient along random directions. This enables scaling to high-dimensional data while retaining stability.
Cow-Contrastive Objectives Gao et al. 2020 hybrid methods that use normalizing flows to approximate the model distribution or its gradients, blending the benefits of tractable likelihoods with contrastive learning. This also permits approximate estimation of the partition function via flow-based auxiliary models.
Denoising Score Matching with Langevin Dynamics (Song et al. 2021 used as building block for diffusion probabilistic models, this technique combines DSM with an iterative generative procedure based on Langevin steps or reverse-time SDEs.
Sampling-free training methods have become central to the modern landscape of Energy-Baser Models, offering scalable alternatives to traditional MCMC-based maximum likelihood estimation. These approaches bypass the ncod sample from the model distribution during training-a key computational bottleneck for likelihood-based learning. Despite their practical success, these methods come with fundamental limitations that constrain their applicability:
1. Lack of normalization: Perhaps the most critical drawback is that these methods do not yicld an estimate of the partition function Ze, except in particular cases. As a result, the learned energy function defines only an unnormalized density. This prechudes the use of these models for tasks that require calibrated likelihoods or probabilistic reasoning, such as Bayesian inference, marginalization, or likelihood-basee evaluation. Even approximate likelihoods are often not computable without additiona assumptions ad hoc sampling.
Many contrastive methods, such as NCE and generalizations, require the specification of noise or negative sample distribution. The success of training depends heavily on how well this distribution matches the model's inductive biases. Poor choices can lead to degenerate or mode-collapsing solutions, and there is often principled way to select adapt this distribution during training.
Scnsitivity design choices:
3. Computational challenges: Although they avoid sampling, methods like Score Matching or its regularized and denoising variants involve computing gradients and sometimes second-order deriva- lives (Hessians) of the energy function. This can become prohibitively expensive in deep architec- tures unless approximations or tricks (e.g., Hutchinson's estimator) are used, which may further affect stability or performance.
As of today, sampling free methods constitute a significant portion of state-of-the-art approaches training EBMs, particularly in high-dimensional generative contexts where MCMC is infeasible. However, because of the issues mentioned above, these methods are rarely sufficient on their own. They are often combined with auxiliary objectives, learned samplers (e.g., amortized Langevin dynamics), or score-based generative modeling frameworks (e.g., diffusson models), which reinterpret EBMs as noise-conditioned score networks. While these hybrid approaches have led to impressive results in practice, they do not resolve the fundamental limitations concerning normalization and likelihood evaluation.
Further advancements in the field have focused on improving the underlying architectures and leveraging additional tricks to alleviate some of the computational challenges and increase model flexibility. For exam- ple, the use of energy- based deep networks with specialized architectures like residual networks (ResNets) transformers and convolutional layers has proven to be effective in learning complex energy functions Hoover et al. (2023). Additionally, techniques like lemperature annealing have been used to improve convergence and prevent mode-collapse in generative settings.
In summary, while sampling-free methods are indispensable for scaling EBMs to realistic data regimes and complex architectures, they fundamentally trade away full probabilistic interpretability in favor of compu- fational tractability. This tradeoff must be carefully considered depending on the downstream application.
32

---

Published in Transactions on Machine Learning Research (MM/YYY)
7 Conclusion and Open Perspectives
In conclusion, Energy-Based Models represent a versatile and conceptually rich framework within the land- scape of generative modeling. They provide not only powerful lools for learning data distributions but also offer deep insights grounded in the formalism of statistical physics. This review has provided a comprehensive exploration of the foundational principles and practical methodologies underlying EBMs, with a particular emphasis on their synergy with the language and techniques of statistical mechanics.
By elucidating the connections between EBMs and other generative paradigms such Generative Adver- sarial Networks, Variational Autoencoders, and Normalizing F Flows, we have identified the specific strengths and limitations of cach approach. In particular, the contrality of energy functions and partition functions in EBM formulations mirrors the core structures of equilibrium statistical mechanics, making them uniquely suited to modeling complex systems with rich internal structure. We have also discussed the crucial role of sampling particularly via Markov Chain Monte Carlo methods in enabling EBMs to generate samples, approximate gradients, and estimate expectations, despite the intractability of the normalization constant in most realistic scenarios. Moreover, we have reviewed state-of-the-art training techniques for FBMs, includ- ing contrastive learning, score-based objectives, and recent sampling-fres frreeinnnggeeeeeeee methods, along with architectural innovations and training tricks that improve stability and efficiency. These advances have helped alleviate long-standing issues such as mode collapse, vanishing gradients, and poor mixing, A central motivation of this review has been to foster deeper interdisciplinary dialoguc-por patloguc-patticularly between the communitics of statistical physics and machine learning. EBMs are inherently connected to equilibrium and nonequilibrium thermodynamies through their reliance on energy- based formulations and sampling processes that parallel physical dynamics. In this light, we have argued that the integration of EBMs with Monte Carlo methods and non-equilbriium statistical mechanics is not only natural but necessary. Techniques from Langevin dy- namics, thermostatted ted systems, and fluctuation theorems can all play role in advancing the training and understanding of EBMs, both theoretically and computationally.
Looking forward, we identify several promising directions for future research. First, the systematic incorpo- ration of physically inspired Monte Carlo algorithms particularly those rooted in non-equilibrumm statistical mechanics could lead to more efficient and interpretable EBM training protocols. Second, the design of new energy functions informed by physical intuition (c.g., symmetrics, conservation laws, locality) could offer inductive biases beneficial for both scientific and engineering tasks. Finally, the exploration of EBMs physically grounded models of computation and learning opens conceptual pathways toward an interpretable and principled generative artificial intelligence. In summary, EBMs provide a robust and unifying perspective on generative modeling, with strong theoretical underpinnings and broad applicability. We hope this review serves as a valuable resource for physicists, computer scientists, and machine learning practitioners alike, helping to clarify the intricate landscape of energy-basod approaches and inspiring new interdisciplinary collaborations. By emphasizing the deep connections between physics and generative modeling, we aim to contribute to a more coherent and physically motivated understanding of learning in complex systems.
Acknowledgments
D.C. worked under the auspices of Italian National Group of Mathematical Physics (GNFM) of INdAM. D.C. expresses their gratitude to Prof. Lamberto Rondoni and prof. Eric Vanden-Eijnden for their support and the valuable discussions. D.C. gratefully acknowledges support from the Italian Ministry of Universily and Research (MUR) through the grant "Ecosistemi dell innovazione", costruzione di "leader territoriali di R&S" with grant agreement no. ECS00000036. D.C. was supported in part through the NYU IT High Performance Computing resources, services, and staff expertise.
References
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. A learning algorithm for boltzmann machines. Cognitive science, 91):1147-16)) 1985.
János Aczél, Bruno Forte, and Che Tat Ng. Why the shannon and hartley entropies are 'naturall Advances in applied probability, 6(1):131-146, 1974.
33

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Clement John Adkins. Equilibrium thermodynamics. Cambridge University Press, 1983.
Jim Al-Khalili. The birth of the electric machines: a commentary on faraday (1832) experimental rescarches in electricity'. Philosophical Transactions of the Royal Sociely A: Malhemalical, Physical and Engineering Sciences, 873(2033):2014022) 2015.
Michael S Albergo and Eric Vanden Eijnden. Nets: non-equilibrium transport sampler. arXiv preprint arXiv:2410.02771 02711, 2024.
Michael Samuel Albergo and Eric Vanden- Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In The Fleventh International Conference on Learning Representations, 2022.
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313 326, 1982.
Christophe Andrieu, Nando De Freitas, Arnaud Doucet, and Michael Jordan. An introduction to meme for machine learning. Machine learning, 50:5 43, 2003.
Ziqiao Ao and Jinglai Li. Entropy estimation via uniformization. Artificial Intelligence, pp. 103954, 2023.
M Arbel, Zhou, and A Gretton. Generalized energy based models. In ICLR 2021-9th International Conference Learning Representations. ICLR, 2021.
Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In Neural Information Processing Systems Conference Workshop: Adversarial Training, 2016.
Dor Bank, Noam Koenigstein, and Raja Giryes. Autoencoders. Machine learning for data science handbook: data mining and knowledge discovery handbook, pp. 353 374, 2023.
Ludwig Boltzmann. Studien über das gleichgewicht der lebendigen kraft zwischen bewegten materiellen punkten [studics on the balance of living force between moving matcrial points]. Wiener Berichte, 58: 517-560, 1868.
Sam Bond-Tayloo Adam Leach, Yang Long, and Chris G Willcocks. Deep generative modelling: A compar- alive review of vaes, gans, normalizing flows, energy-based and autoregressive models. IEEE transactions on pattern analysis and machine intelligence, 44(11):7327-73 7347, 2021.
Davide Carbone, Mengjian Hua, Simon Coste, and Eric Vanden-Eijned Efficient training of energy-based models using jarzynski equality. Advances in Newrol Information Processing Systems, 36, 2024a.
Davide Carbone, Mengjian Hua, Simon Coste, and Eric Vanden- Eijuden. Generative models out-of- cquilibrium particle systems: Training of energy-based models using non-equilibrium thermodynamics. In International Conference on Nonlinear Dynamics and Applications, pp. 287 311. Springer, 2024b.
Miguel A Carreira-Perpinan and Geoffrey Hinton. On contrastive divergence learning. In International workshop on artificial intelligence and slalislics, pp. 33 40. PMLR, 2005.
George Casella, Christian P Robert, and Martin T Wells. Generalized accept-reject sampling schemes.
Lecture Notes- Wotes-Monograph Series, pp. 342-34 2004.
Chen-Hao Chao, Wei-Fang Sun, Yen-Chang Hsu, Zsolt Kira, and Chun-Yi Loc. Training energy-basod normalizing îlow with score-matching objectives. Advances in Neural Information Processing Systems, 36, 2024.
Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, and Yoshua Bengio. Your gan secretly an energy-based model and you should use discriminator driven latent sampling. Advances in Neural Information Processing Systems, 33:12275-12287, 2020.
Omar Chehab, Aapo Hyvarinen, and Andrej Risteski. Provable benefits of anncaling for estimating nor- malizing constants: Importance sampling, noise-contrastive estimation, and beyond. Advances in Neural Information Processing Systems, 36, 2024.
34

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Taoli Cheng and Aaron C Courville. Versatile energy-based probabilistic models for high energy physics. Advances in Newral Information Processing Systems, 36, 2024.
Barry A Cipra. An introduction to the ising model. The Americun Mathematical Monthly, 94(10)9337 959, 1987.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor lonescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications to score-based generative modeling. Advances in Neural Information Processing Systems, 34: 17695-17709, 2021.
Arthur Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological)) 39(1):1 122, 1977.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8799 2021.
Aaron R. Dinner, Andrej Sali, Lorna J Smith, Christopher M Dobson, and Martin Karplus. Understanding protein folding via free-energy surfaces from theory and experiment. Trends in biochemical sciences, 25 (7):331-333) 2000.
Carles Kaningo--Enrihh Alberto Bietti, Marylou Gabrié, Joan Bruna, and Eric Vanden-Eijnden. Dual training of energy-based models with overparametrized shallow neural networks. ar preprint arXi::210.005194, 2021.
Yilun Du and Tgor Mordatch. Implicit generation and modeling with energy based models. Advances in Newral Information Processing Systems, 2019.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. In International Conference on Machine Learn- ing, pp. -510. PMLR, 2023.
Simon Duane, Anthony D Kennedy, Brian Pendleton, and Duncan Roweth. Hybrid monte carlo. Physics letters B, 195(2):216 222, 1987.
Freeman Dyson et al. meeting with enrico fermi. Nature, 427(6972):297 297, 2004.
Denis J Evans, Debra J Searles, and Stephen R Williams. Dissipation and the relaxation to equilibrium. Journal of Statistical Mechanics: Theory and Experiment, 2009(07):P07029, 2009.
Frederik Federspicl, Ruth Mitchell, Asha Asokan, Carlos Umana, and David McCoy. Threats by artificial intelligence to human health and human existence. BMJ global health, 8(5):e010435, 2023.
Enrico Fermi. Thermodynamics. Courier Corporation, 2012.
Enrico Fermi, Pasta, Stanislaw Ulam, and Mary Tsingou. Studies of the nonlinear problems. Technical report, Los Alamos National TTb.(T.ANT)), Los Alamos, NM (United States), 1955.
Farhan Feroz and Mike Hobson. Multimodal nested sampling: an efficient and robust alternative to markov chain monte carlo methods for astronomical data analyses. Monthly Notices of the Royal Astronomical Society, 384(():449-463 2008.
Ragnar Fjelland. Why general artificial intelligence will not be realized. Humanities and Social Sciences Communications, 7(1):1-9, 2020.
Wendell H Fleming and Raymond W Rishel. Deterministic and stochastic optimal control, volume 1. Springer Science & Business Media, 2012.
35

---

Published in Transactions on Machine Learning Research (MM/YYY)
Alexander L Fradkov. Early history of machine learning. IFAC-PapersOnin 53(2):1385-1399) 2020.
Nathan C Frey, Dan Berenberg, Joseph Kleinhenz, Isidro Hotzel, Julien Lafranco-Vanasse, Ryan Lewis Kelly, Yan Wu, Arvind Rajpal, Stephen Ra, Richard Bonneau, et al. Learning protein family manifolds with smoothed energy-based models. In ICLR 2023 Workshop on Physics for Machine Learning, 2023.
Karl Friston. The free-energy principle: rough guide to the brain? Trends in cognâtive sciences, 13(7): 293-301, 2009.
Giovanni Gallavotti. Statistical mechanics: A short treatise. Springer Science & Business Media, 1999.
Ruiqi Gao, Erik Nijkamp, Diederik P Kingma, Zhen Xu, Andrew M Dai, and Ying Nian Wu. Flow contrastive ostimation of encrgy-based models. Proceedings of the IEEE/CVE Conference on Computer Vision and Pullern Recognition, pp. 7518 7528, 2020.
Stuart Geman and Donald Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on pattern analysis and machine intelligence, (6):721-741) 1984.
Josiah Willard Gibbs. Elementory principles in statistical mechanics: developed with espectal reference to the rational foundations of thermodynamics. C. Scribner's sons, 1902.
Laurent Girin, Fanny Roche, Thomas Hueber, and Simon Leglaive. Notes on the use variational au- toencoders for speech and audio spectrogram modeling. In DAFr 2019-22nd International Conference on Digital Audio Effects, pp. 8, 2019.
Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier Alameda-Pinedde Dynamical variational autoencoders: A comprehensive review. arXiv preprint arXiv:2008. 12595, 2020.
Ian Goodfellow, Jean Pouget-Abadis Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Ulf Grenander and Michael Miller. Representations of knowledge in complex systems. Journal of the Royal Statistical Sociely: Series B (Methodological), 56(4):549 581, 1994.
Michael Gutmann and Aapo Hyvärinen. Noiss-conttrative estimation: A new estimation principle for unnor- malized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 297-304. JMLR Workshop and Conference Proceedings, 2010.
W Keith Hastings. Monte carlo sampling methods using markov chains and their applications. Biometríka. 57(1):97 109, 1970.
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg- Kirkpairka....c... Lagging inference networks and posterior collapse in variational autoencoders. In International Conference on Learning Representations, 2018.
Donald O Hebb. Organization of behavior. new york: Wiley. J. Clin. Psychol, 6(3):335-300 1949.
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Newral computation, 14(8):1771-1800, 2002.
Geoffrey E. Hinton and Ruslan Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):5044 507, 2006.
Geoffrey E Hinton and Richard Zemel. Autoencoders, minimum description length and helmholtz free energy. Advances in neural information processing systems, 6, 1993.
Antti Honkela, Tapani Raiko, Mikael Kuuscla, Matti Tornio, and Juha Karhunen. Approximate ricmannian conjugate gradient learning for fixed-form variational bayes. The Journal of Machine Learning Research, 11:3235 3268, 2010.
36

---

Published in Transactions on Machine Learning Research (MM/YYY)
Benjamin Hoover, Yuchen Liang, Bao Pham, Rameswar Panda, Hendrik Strobelt, Duen Horng Chau, Mo- hammed Zaki, and Dmitry Krotov. Energy transformer. Advances in neural informalion processing systems, 36:27532 27559, 2023.
John Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national acaderry of sciences, 79(8):2554 2558, 1982.
Patrick Huembeli, Juan Miguel Arrazola, Nathan Killoran, Masoud Mohseni, and Peter Wittek. The physics of energy-based models. Quantum Machine Intelligence, 4(1):1, 2022.
Aapo Hyvarinen. Connections between score matching, contrastive divergence, and pseudolikelihood for vontinuous-valued variables. IEEE Transactions on neural networks, 18(5):1529 1531, 2007.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
C Jarzynski. Nonequilibrium equality for free energy differences. Physical Review Letters, (8)1))2669,, 1997.
Edwin T Jaynes. Information theory and statistical mechanics. Physical review, 106(4))620, 1957a.
Edwin T Jaynes. Information theory and statistical mechanics. ii Physical review, 108(2):171, 1957b.
Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker planck equation. SIAM journal on mathemalical analysis, 29(1):1 17, 1998.
William L Jorgensen. Free energy calculations: breakthrough for modeling organic chemistry in solution. Accounts of Chemical Research, 22(5):184- 1899 1989.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Ivan Kobyzev, Simon JD Prince, and Marcus A Brubaker. Normalizing flows: An introduction and review of current mothods. IEEE transactions on pattern analysis and machine intelligence, 43(11):396--3979, 2020.
Jonas Köhler, Andreas Krämer, and Frank Noé. Smooth normalizing flows. Advances in Neural Information Processing Systems, 34:2796-2841 2021.
Solomon Kullback and Richard Leibler. On information and sufficiency. The annals of mathematical statistics, 22(1):79 86, 1951.
Pierre-Simon Laplace. A philosophical essay on probabilities. Courier Corporation, 2012.
Yann LeCun, Sumit Chopra, and Raia Hadsell. A tutorial on energy-based learning. In Gökhan Baklr, Thomas Hofmann, Alexander Smola, Bernhard Schölkopf, and Ben Taskar (eds.), Predicting structured data, chapter 10. MIT press, 2007.
Junde Li, Collin Beaudoin, and Swaroop Ghosh. Energy-based generative models for larget-specific drug discovery. Frontiers in Molecular Medicine, 3:1160877, 2023.
Evgenii Mikhailovich Lifshitz and Lev Petrovich Pitaevskii. Statistical physics: theory of the condensed state, volume 9, Elsevier, 2013.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Lc. Flow matching for generative modeling. In The Eleventh International Conference Learning Representations, 2022.
Jun S Liu. Monte Carlo stralegies in scientific compuling, volume 75. Springer, 2001.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. Graphebm: Molecular graph generation with energy-based models. In Energy Based Models Workshop-ICI.R 2021, 2021.
37

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Aduances in neural informalion processing systems, 33:21464-21475, 2020.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint ArXiv:2209.088: 03003, 2022.
Siegrid Löwel and Wolf Singer. Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. Science, 255(5041):209-22 1992.
Hengyuan Ma, Li Zhang, Xiatian Zhu, and Jianfeng Feng. Accelerating score-based generative models with proconditioned diffusion sampling. In European Conference on Computer Vision, pp. 1-16. Springer, 2022.
Christopher Manning and Hinrich Schutze. Foundations of statistical natural language processing. MIT press, 1999.
J. C. Mattingly, A. M. Stuart, and D. J. Higham. Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise. Stochastic Processes and their Applications, (2):21185-233. 2002.
Warren S McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysicss 5:115-133, 133, 1943.
Kerrie Mengersen and Richard L Tweedie. Rates of convergence of the hastings and metropolis algorithms.
The annals of Statistics, 24(1):101-22 121, 1996.
Nicholas Metropolis, Arianna W Rosenbluth, Marshall Rosenbluth, Augusta H Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):0087-
1092, 1953.
Evgeny Morozov. The true threat of artificial intelligence. International New York Times, pp. NA-NA, 2023.
Radford M Neal and Geoffrey E Hinton. A view of the algorithm that justifies incremental, sparse, and other variants. In Learning in graphical models, pp. 355-368. Springer, 1998.
Kim A Nicoli, Christopher J Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Pan Kessel, Shinichi Nakajima, and Paolo Stornati. Estimation of thermodynamic observables in lattice field thcories with deep generative models. Physical review letters, 126(3))03200,, 2021.
Bernt Oksendal. Stochastic Differential Equations. Springer-Verie Verlag Berlin Heidelberg, 6 edition, 2003.
Achraf Oussidi and Azeddine Elhassouny. Deep generative models: Survey. In 2018 International conference on intelligent systems and computer vision (ISCV), pp. 1 IEEE, 2018.
Bo Pang, Tianyang Zhao, Xu Xie, and Ying Nian Wu. Trajectory prediction with latent belief energy-based model. In Proceedings of the IEEE/CVE Conference Computer Vision and Pottern Recognition, pp. 11814-11824, 2021.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshmi- narayanan. Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research, (11)::26772266) 2021.
Neal Parikh, Stephen Boyd, et al. Proximal algorithms. Foundations and trends in Optimization, 1(3): 127-239, 2014.
Giorgio Parisi. Correlation functions and computer simulations. Nuclear Physics B, 180 1981.
Jakiw Pidstrigach. Scoro-based generative models detect manifolds. Advances in Neural Information Pro- cessing Systems, 35:35852 2555555--3565,, 2022.
38

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convo- lutional generative adversarial networks. In Proceedings of the 5th International Conference on Learning Representations Workshop Track, 2016.
Christian P Robert, George Casella, and George Casella, Monte Corlo statistical methods, volume 2. Springer, 1999.
Gareth O Roberts and Richard Tweedie. Exponential convergence of langevin distributions and their discrete approximations. Bernoulli, pp. 341-363, 1996.
L Chris G Rogers and David Williams. Diffusions, Markou processes and martingales: Volume 2, Itō calculus, volume 2. Cambridge university press, 2000.
Tim Salimans, lan Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pp. 2226 2234, 2016.
Saptarshi Sengupta, Sanchita Basak, Pallabi Saikia, Sayak Paul, Vasilios Tsalavoutis, Frederick Atiah, Vadla- mani Ravi, and Alan Peters. review of deep learning with special emphasis on architectures, applications and recent. trends. Knowledge- Based Systems, 194:1 10596,, 2020.
K Sharma, Bhopendra Singh, Edwin Herman, R Regine, S Suman Rajest, and Ved P Mishra. Maximum in- formation measure policies in reinforcement learning with deep energy-based model. In 2021 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE), pp. 19 24. LEEE, 2021.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp. 2256-2265. PMLR, 2015.
Jiaming Song, Shengjia Zhao, and Stefano Ermon. A-nice-me: Adversarial training for meme. Advances in neural informalion processing systems, 30, 2017.
Yang Song and Diederik P Kingma. How to train your energy-based models. arXiv:2101.03288 2021.
arXie preprinl
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pp. 374-584. PMLR, 2020a.
Yang Song, Jascha Sohk-Dicks,ein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations, 2020b.
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34:1415 1428, 2021.
Stephen M Stigler. The epic story maximum likelihood. Statistical Science, pp. 598 620, 2007.
Gabriel Stoltz, Mathias Rousset, et al. Free energy computations: A mathematical perspective. World Scientific, 2010.
Amos Storkey Increasing the capacity of hopfield network without sacrificing functionality. In Artificial Newral Networks-Tor ICANN'00: 7th International Conference Lausanne, Switzerland, October 8-10, 1997 Proceeedings 7, pp. 451 456. Springer, 1997.
Daniel W Stroock. An introduction to Markow processes, volume 230. Springer Science & Business Media, 2013.
Ruoxi Sun, Hanjun Dai, Li Li, Steven Kearnes, and Bo Dai. Towards understanding retrosynthesis by energy-based models. Advances in Neural Information Processing Systems, 34:10186-10194, 2021.
Robert H Swendsen and Jian-Sheng Wang. Replica monte carlo simulation of spin-glasses. Physical review letters 26627) 1986.
39

---

Published in Transactions on Machine Learning Research (MM/YYYY)
E. G. Tabak and Cristina V. Turner. A family of nonparametric density estimation algorithms. Communi- calions on Pure and Applied Mathematics, 66(2):114-106 164, 2013.
Esteban G. Tabak and Eric Vanden- Eijnden. Density estimation by dual ascent of the log-likelihood. Com- mamicalions in Mathemalical Sciences, 8(1):217 233, 2010.
Denis Talay and Luciano Tubaro. Expansion of the global error for numerical schermes solving stochastic differential equations. Stochastic Analysis and Applications, 8(4): 509, 1990.
Yee Whye Tch, Max Welling, Simon Osindcro, and Geoffrey E Hinton. Energy-based models for sparse overcomplete representations. Journal of Machine Learning Research, 4(Dec):1235 1260, 2003.
Tijmen Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In International conference on Machine learning, pp. 1064 1071, 2008.
Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Xiv preprinl ar Xr::22300 00442, 2023.
George E Uhlenbeck and Leonard S Ornstein. On the theory of the brownian motion. Physical review, 36 (5):823, 1930.
Francisco Vargas, Shreyas Padhy, Denis Blessing, and Nikolas Nüsken. Transport meets variational inference: Controlled monte carlo diffusions. In The Twelfth International Conference on Learning Represenlations, 2024.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23 (7):1661-1674, 2011.
Ruoqi Wei, Cesar Garcia, Ahmed El-Sayed, Viyaleta Peterson, and Ausif Mahmood. Variations in variational autoencoders-a comparative evaluation. Jeee Access, 8:153651 153670, 2020.
Andre Wibisono. Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem. In Conference on Learning Theory, pp. 2093-3027. PMLR, 2018.
Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In International Conference on Machine Learning, pp. 2635-2644. PMIR, 2016.
Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 1644):133, 2023.
Xiulong Yang and Shihao Ji. M-ebm: Towards understanding the manifolds of energy-based models. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 291 302. Springer, 2023.
Sangwoong Yoon, Young-Uk Jin, Yung-Kyun Noh, and Frank Park. Energy- models for anomaly detection: manifold diffusion recovery approach. Advances in Neural Information Processing Systems, 36, 2024.
Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, and Ying Nian Wu. Latent diffusion energy-based model for interpretable text modelling. In International PMIRR 2022.
Conference Machine Learning, pp.
Shuangfei Zhai, Yu Cheng, Weining Lu, and Zhongfei Zhang. Deep structured energy based models for anomaly detection. In International conference on machine learning, pp.
PMLR, 2016.
Yao Zhang, Jeroen Berrevoets, and Mihaela Van Der Schaar. Identifiable energy-based representations: an application to estimating heterogencous causal effects. In International Conference on Artificial Intelli- gence and Statisties, pp. 4158 4177. PMLR, 2022.
Yijia Zheng, Tong He, Yixuan Qiu, and David Wipf. Learning manifold dimensions with conditional variational autoencoders. Advances in Neural Information Processing Systems, 1434779-347221 2022.
40

---

Published in Transactions on Machine Learning Research (MM/YYY)
A Historical Perspective on Generative models
The problem of description of data through a mathematical model is very old, being the basis of scientific method. The set of measurements use to fulfill the role that contemporary data scientists now refer to as a datasel. Presently, the model can take the form of an exceedingly complex neural net but the un- derlying extrapolation remains akin to P.S. Laplace's famous deterministic statement [Laplace (Laplace 2012): "An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed [data], if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula. the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past could be present before ils eyes.". One can easily extend this reasoning, asserting that the more data one possesses, the more robust and detailed the model that can be constructed atop them. This leads to en- hanced predictions and greater stability concerning unforeseen behaviors.
This line of thought gained momentum in the previous century with the advent of automatic calculators, leading to an astounding pace of development. For instance, consider the remarkable difference in computa- tional power between modern personal computers and the computer used by NASA for the Apollo program in the 1960s-the latter had only fraction of the data processing capability available today F Data has always been a fundamental part of science, but today they sort of gained the leading role as opposed to pure theory. For instance, consider the history of Kepler's laws, The genesis of such results is deeply rooted in a vast collection of obsorvational data amassod by T. Brahe D Kepler's formulation was, in fact, motivated by the necessity to explain these astronomical measurements. In a simplified analogy, we observe the dichotomy between the "model,' ¹ embodied by Kepler, and the 'dataset,' represented in this narrative by Brahe. Since the 17th century, these two actors have played equally fundamental roles in the advancement of science, taking turns on the stage with the same importance. Consider the pivotal role played by Faraday's experi- ments in understanding electromagnetism (AI-Khalili 2015), long before Maxwell's 's laws. Or, conversely, the impact of theory of Relativity way before its experimental confirmation.
Thanks to the aforementioned technological advancements in computer science, the volume of generated scientific (and not) data has dramatically increased, resulting from advancements in simulation and storage capabilities. Thus, particularly during the 2000s, we have witnessed a profound paradigm shift represented by the Big Data Era a growing collection of data on human activities, including images, text, sounds, and more are available nowadays. This shift posed a methodological problem in what we now refer to as data science. In an historical analogy, it is akin to Brahe suddenly providing Kepler with a thousand times the amount of data that the latter was accustomed to. And this is where the machine learning approach came into play (aydFrakoov] 2020). As a side remark, the models required to process Big Data had already been theoretically studied since the invention of the perceptron(MeCulle constrained by computational power in the last century, but, as peculiar example of convergent develop- ment, they became the primary tools in the toolbox of data scientists in the 2000s, simultaneonsly to the appearance of Big Data on the stage.
& Pitts 1943). Their application was
There is indeed a discontinuity that deserves more attention: the increasing collection of data generated by humans. The term Big Dala is sometimes limited to images, sounds, videos, lext, and metadata resulting from human activities Unlike scientific measurements, having access to an extensive quantity of information produced by humans opened Pandora's box, prompting the natural question: can we build artificial intel- ligence by leveraging Big Data? In other words, can we construct a machine capable of generaling data as humans do, by training it in some smart way? Data here is to be understood in a broad sense, encompassing new theorems, art pieces, images, videos, and even novels.
Generative models represent, in this sense, the most recent breakthrough in technological advancement to- wards intelligent-like machines. It is complicated to provide a general definition, and there are already many available from different sources However, if we informally focus on those already known to the general public, such as Generative Pre- Prainod Transformers ( a possible simple definition is: generative
Caliled
https://nediun.. httpp///ledtinnnoo//ssl/b/dddd/p
https://www.yttiii http::/wwwwaaattttee///////////0 botttbbcthhhhtthtmm
41

---

Published in Transactions on Machine Learning Research (MM/YYYY)
models model the observed data through a probabilistic program, able to output samples from the distribu- lions. Returning to the historical analogy: nowadays, we are able to build 'BraheGPT', which can generate and gather new plausible measurements about the orbits of planets in unobserved planetary systems after training on obscrved data from the solar system. However, it is not Kepler - the probabílistic model is generally not informative about the datasel; deductive reasoning is not necessary to generate new data in- stances, although it remains fundamental to understanding the world. Von Neumann would certainly adapt his famous statement (Dyson al.) (204)) about overfitting to modern data science, cautioning against the ability to generate examples without general picture.
Prominent data scientists, such as Yann LeCun, have recently emphasized that the use of interpretable generative models is crucial for achieving a "unified world model for Al capable of planning ! This thesis becomes imperative in the realm of computational sciences, where qualitative generation alone is insufficient as a benchmark to evaluate model performance. In sectors like Molecular Dynamics, Biochemistry, and similar fields, the model must convey substantial information about the dataset. The generative models that excel in terms of interpretability, which form the main foeus of the present work, are precisely the Eneryy- Based Models (EBMs). These models offer unique advantage in their ability to provide insights into the underlying mechanisms of the data they generate. The reason is that a trained EBM provides a normalized probability model - having such object is equivalent to interpretability since you can give probabilistic meaning to each sample, generated and not. One can exploit this fact for instance in anomaly detection (Zhai et al. 2016). On the contrary the original formulation of state-of-the-ar methods as flow-based ones do not output a normalized probability, but usually an efficient sampling method. For this reason building a link between EBMs and other generative models is a very active line of research (Che et al. 2020 (Chao et al.) 2024). In areas such as Molecular Dynamics and Biochemistry, where understanding the intricate relationships within the dataset is crucial, the interpretability of EBMs stands out. As the pursuit of a unified world model for AI continues, the emphasis on interpretable generative models, particularly EBMs, plays a pivotal role in bridging the gap between data generation and comprehensive understanding.
A.1 A long story: from Boltzmann- Gibbs ensemble to the advent of EBMs
After outlining the historical evolution of generative models, this section explores the origins and develop- ment of Energy-Based Models. The theoretical foundation of EBMs spans multiple disciplines, including statistical physics, probability theory, and computer science, often appearing under different names. Here, we adopt historical perspective, while deeper theoretical discussions are deferred to later chapters. This review aims to provide a broad audience with coherent underslanding of EBMs by contextualizing their emergence.
Since an EBM is a probabilistic model, it is natural to ask for its definition. To answer, we begin with the Boltzmann-Gibbs measure, a fundamental concept in statistical mechanics originating from the works of Ludwig Boltzmann and Josiah Willard Gibbs in the late 19th century. Boltzmann introduced the statistical interpretation of entropy and the Boltzmann distribution (Boltzmann [168),, linking microscopic configura- tions to macroscopic thermodynamics. In parallel, Gibbs formalized these ideas into the canonical ensemble and the Gibbs measure (Gibbs 1902), providing a mathematical framework for thermodynamie properties. The resulting Boltzmann- Gibbs measure describes the probability distribution over energy states at thermal equilibrium at uemperature T, forming the foundation of statistical mechanics across diverse physical sys- tems. We informally recall its definition: given the state of the system x E 2, where Q is the so-called phase space, and an energy function U U - RR,, we can express the associated probability density function
p(x) x e (BU(x)
(107)
where 3 = 1/kgT kg being the Boltzmann constant. A detailed mathematical description will be provided in the next chapters.
The next chapter of the story happens in 1924, when E. Ising presented his PhD thesis The so called Ising model is fundamental mathematical model in statistical mechanics. Tt serves as a simplified yet powerful representation of magnetic systems, which the Ising Model represents as a graph with N nodes. In the Ising
http:///www.......o.....oo.ooone frontier/
https:://ww.hht www.lssuuuuuuuuggg.. de/bburr..dd//deee hrge
html
42

---

Published in Transactions on Machine Learning Research (MM/YYYY)
model, each lattice site is associated with magnetic spin, which can take two possible values: +1 ("up") and -1 ('down"),, yielding f = {-1,1] as set of possible lattice configurations. The energy of the system is modeled as
U1a(x)= - - EJ
Σ hear
(108)
(K5)
where i,j E A are indexes of sites in the lattice; (1,5) the standard notation (Cipral 1987) to denote that the sum is restricted to first neighbours and Jiy is the strength of the interaction. The field h; instead individually acts on each site and y is just constant that traditionally corresponds to magnetic moment. In laymen terms, each magnetic spin interacts with its first neighbours and with an external field. The alignment of spins is encouraged.
In considering equation 107 as associated to Use, the primary focus is often on the behavior of the system as a function of temperature. In a nutshell, at high temperatures, thermal fiuctuations dominate, and the system exhibits no long-range order, iee. spin assumes up and down state with equal probability. As the temperature decreases, there is a critical point at which the system undergoes a phase transition, leading to spontaneous "ordering"; i.c. magnetizaion, of the system, that is either most of the spin are up or down respectively.
For some decades the interest for Ising model and its extensions was confined to physics. The motivation for invoking such model in the present work is the following: in the 80s a fundamental connection be- tween Ising model and data science manifested through Hopfield networks (Hopfiill 1982) and Boltzmann machines(Ackley el al. 1985). Both can be viewed as an Ising lattice where interactions are not confined to first neighbors. Apart from the initial summation, which, for the former, extends Vi, E rather than just (ij), the energy function bears resemblance to equation [08] From a statistical physics standpoint, the distinction between a Hopfield network and Boltzmann machines lies solely in the temperature value. The purpose of the former is pattern recognition and associative memory tasks. A distinctive feature of Hop- field networks is their proficiency in storing and retrieving patterns through symmetric connections between neurons, that is Ising siles, in the network. In practice, when provided with a set of net configurations y E 52 representing patterns, denoted by x= ==1,, -,...,,hh n, one constructs the coupling as follows:
1 n
(109)
'neurons wire together if they fire together' (Löwel & Singer 1992), but further specifications (Storkey 1997) are available. This phase is commonly referred to the training of the network. Subsequently, one can define a retrieval iterative dynamics starting from any
This involves employing the Hebbian rule(Hebb,
1949
configuration k= E Ω. as exemplisied by the equation:
===l=su([J++ + h)
KeN
(110)
Here, J represents the coupling matrix defined element-wise in equation 109 and h is a bias vector that influences the preferences for 'up' or 'down'. It is noteworthy that in Hopfield network, there is no use of the Boltzmann-Gibbs ensemble; the objective is to construct a dynamical system with prescribed attractors, which are the minima of (xx by design.
Boltzmann Machines share the same structure and energy function but the goal extends beyond the mere retrieval of patterns; it is to model their overall distribution. To illustrate this concept, consider a finite set of n natural images of cats and dogs. A meticulously designed Hopfield Network could perfectly retrieve any of these examples. On the contrary, a trained Boltzmann Machine aspires to generate new instances of cats and dogs, capturing, in sense, the distribution of such images. The objective appears to be a different level of difficulty: although possibly big, the cardinality of the set of patterns is finite; the number of possible variations of cats and dogs is not. Thus, one can immediately guess why the training and generation phases (n.b. it is no more just a retrieval) are completely different w.r.t. Hopfield Networks. The take home message is the hypothesis that the distribution of the given patterns can be described by a Boltzmann- ensemble associated to the energy of the Hopfield Network at temperature T.
It is convenient to consider Boltzmann Machines as a specific instance of Energy- Based Models, a term introduced by Hinton et al. (Teh et. al. 2003), to describe both training and generation phases. EBMs
43

---

Published in Transactions on Machine Learning Research (MM/YYYY)
differ from Boltzmann Machines in the use of generic parametric energy Ua(2) instead of the usual choice made for the latter. Here, 0 E @ needs to be selected and trained so that the Boltzmann-Gibbs ensemble associated with Up(r) 'fits well' the distribution of the given patterns, which we refer to as p.. After training the EBM, the generative phase involves sampling equilibrium configurations from pa. Specifically, Boltzmann Machine corresponds to an EBM with the choice of ((r) as the energy Hopfield Network and / =
Despite their conceptual simplicity, both training and generation represent fundamental open problems that intersect multiple research fields. In essence, sampling from Boltzmann-Gibbe ensemble is challenging task in general, and unfortunately, it is necessary even during the training phase. For this reason, the use of Boltzmann Machines was limited to toy models until the proposal of the Contrastive Divergence algorithm by Hinton HHinton 2002).
This procedure, along with its generalizations, made it possible to apply EBMs to practical problems. Morcover, thanks the adoption of a deep neural network (Xie et al.] 2016 as Ug. the interest towards this class of generative models critically increased and in 2010s the use of E EBMs for state-of-the-arl tasks became standard. However, all that glitters is not gold. Despite its success in generating high-quality individual samples, the use of Contrastive Divergence is known to be biased. For instance, it could happen that indi- vidual images are correctly generated, but ensemble properties as the relative proportion of the two species is incorrect. Although Hinton et al. originally claimed that this bias is generally small (Carreira-Perpinan & Hinton 2005), numerous counterexamples have been shown in the more than 20 years since their original paper. The absence of novel paradigm shifts, coupled with the rise of alternative generative models (e.g., ceffusion-based ones (Song et al.) (020b)), ) has reduced attention on EBMs and consequently on Boltzmann Machines. Nevertheless, recent advances in sampling algorithms, optimization techniques, and theoretical understanding particularly from the perspective of statistical physics have sparked renewed interest in EBMs. Their interpretability, connection with energy landscapes, and potential for modeling struc- tured and multi-modee modal data suggest that EBMs still hold substantial promise in modern generative modeling.
B Stochastic Interpolants
Definition B.1. Given two probability densities P1.P2 : R + R4, a stochastic interpolant between them is stochastic process X, Rdd such that
X, I(t, +
[0.1]
(111)
where:
The function is of class C on its domain and satisfy the following endpoint conditions
Xx
i=0,1
(112)
as well as
30 < 00 |0,1 (t, X,xx S C1 Xo -X1 (t, (o, X1) [0.1] x Rd Rd
(113)
7: [0, + R is such that 7(0) 7(1) and x() 0 for t E (0,
The pair (Xo; X1) is sampled from measure that marginalizes on Po and Pt, that is v(d.o, Rd)) PodXo and v(RR)) dXi)= p1
The variable z is a Gaussian random variable independent from (Xo,Xi), i.e. N' Ia) and L X1)
Let us focus on the case in which Ā to be a simple base distribution (e.g. a Gaussian) and Pa: that is the data distribution. Equation equation means that we sample a couple X, ~ Po and X₁ from the dataset, the interpolant is a stochastic process that connects the two points. The objective to build generative model that, in some sense, learns from the interpolants the way to map samples from p to px. The first important result. in this sense is the (AlbrroggAlberro Vanden-Fijnden 2022
44

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Proposition B.1. The interpolant Xt is distributed at any time t E [0, 1] following time dependent density p(x,1) 1) such that p(r, 0) Po and p(z, 1) Pt, and also salisfies the following transport equation:
0.pt V =b =0
(114)
where the vector field b(x, 1) is defined by conditional expectation:
b(x, 1) = Xx X, = -| = E [0,7 (1, Xo. X1) +(1x² X x]
(115)
Proof. Let t)k,t) = Ee X the characteristic function of p(z, t), that is
g(k, t) = Ee 1k (10xXox)))+0
(116)
If we compute the time derivative of g, we obtain
deg(k,t) ik mlktt)
(117)
where mkk,t) = B[(d,] (t, Xo. X1) + + (..]). By definition conditional expectation,
m(k,t) t)
E[(O,T (t, Xo, X1) + (tt)2) | X; =1p(xx, t)dr
(118)
[ buu t)p((, t)dx
where we used the definition of b. If we insert m(k,() () in equation 117 and we compute the Fourier anti- transform, we immediately obtain equation 114 in real space.
Other properties of b can be proven, but for the sake of the present summary we will not delve into them. As usual we identify á and p. base and data distributions. Thanks to the previous Proposition we can already define diffusion-based generalive model:
Lemma B.1 (ODE Generative Model). Given Proposition B. and p(ar, 0) p, the choice p(X,, 1) b(X,,)) and (X,,t) t) = 0 in equation 40 satisfies the endpoint condition for T T==b
Differently from score-based diffusion models, such ODE-based formulation does not involve stochasticity during generation. In fact, the ODE X, b(X,,() can be interpreted as Normalizing Flow (see Section where the pushforward is defined via a transport PDE. Interestingly, the ODE formulation is formally equivalent to an SDE formulation:
Lemma B.2 (SDE Generative Model), For e > 0, given Proposition B. p(z, 0) = - and the score s(x, t) V llg, t), the choice u(X,,t) t) = t(xx,))+es(Xx.t) and (X.. = /25 in equation 40 satisfies the endpoint condition for T 1.
Proof. Adding and subtracting the score to equation 114 we obtain for any >> 0
0pp V (bbes- Es)p)
(119)
But sp Vp, that is
+ V. ((b+ <Vp) 0
(120)
Trivially, the solution of the PDE equation 120 is the law of stochastic process solution of an SDE as equation
40
We presented the proof as an example of the standard trick used to convert the diffusion term into a transport term exploiting the score.
We defined the generative model, but similarly to score-based diffusion, we need to clarify how b and s are learned in practice from data. For such purpose, we present. the following result:
45

---

For the sake of the present summary, we will not present the prooo(Albergo & Vanden-Eijnden 2022) The take home message is that one can now propose two neural networks, namely bo( t) and se (z, t), and train them through backpropagation using equation [2I and equation [22 The integrals are estimated using random pairs (1) and times 24(0, 1]. As for score-based diffusion, we avoid delving into practical details regarding the implementation of the neural networks. We emphasize the main message: it is feasible to construct diffusion model defined in finite time interval that does not solely rely the specific instance of stochastic interpolation Published in Transactions on Machine Learning Research (MM/YYYY)
Proposition B.2. The vector field b(x,t) is the unique mindmizer of the following objective loss
6b = ( ( 8 (t, (0.1 (t, X1) ((0)) X.). de
(121)
Simvlarly, the the score s(x,1) the unique minimizer of the following objective loss
/
E 13 (t), + (t)= (1 ((,, dt J
(122)
score function. In fact, score-based diffusson can be viewed or similar methods (refer to Section 3.5 for more details).
Concerning practical aspocts, the froedom choosing the function Xo. X1) as well as (t) can be chal- lenging due to the absence of a general guiding principle. Unfortunately, the structure of the interpolant and the implementation of and sær can significantly impact the efficient training of the model. Regarding the generative phase, the SDE and ODE formulations are formally equivalent, but the practical choice is not straightforward. From numerical perspective, the primary issue lies in the time discretization and in- tegration of the differential equations. The ODE is preferred since integration methods are more stable and precise compared to those for SDEs; this allows for larger time steps and accelerated generation. This is also a significant advantage of stochastic interpolants over score-based diffusion, which is 3DE-based. However, the presence of noise appears to be necessary as regularization: possibly imperfect, any mismatch is 'smoothed' the SDE setting by the presence noisc. The value of functions as hyperparameter in this context.
layman's terms, since b is learned and
46


Reviews:
[{"review_id": "BuOfMKIOCN", "reviewer": "Reviewer_pNBU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The writing style is pedagogical, in the fashion of scientific dissemination books, at risk of being too verbose in certain parts (e.g. in Section 1.1, and at the beginning of Section 3). That may be appreciated by readers that lack prior knowledge of the field or those that look for a historical perspective that is usually lacking in machine learning papers. \n\nThe paper has some weaknesses that I detail in the Requested Changes section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper provides an overview of Energy-Based Models in the context of generative modeling, and shorter overviews on related generative modeling techniques: variational autoencoders, generative adversarial networks, diffusion models, and normalizing flows. Thus, the work can be used as an entry point to learn about EBMs and more generally about the development of the field of generative modeling. The work touches on several other aspects: the origin of EBMs in Hopfield networks and Boltzmann machines, an overview of the MCMC methods to sample EBMs, links to statistical physics, and a description of the contrastive divergence algorithm.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Several (most) citations are not well formatted and should use citep instead of citet. \n\nThe meaning of Table 2 is unclear. It is hard to understand how the fields in each column are analogous, especially in the evaluation column. Unless it can be clarified, it should be removed. \n\nThe sentence “In conclusion, stochastic interpolants provide a general framework closely related to other diffusion models, such as score-based diffusion, flow matching Liu et al. (2022); Lipman et al. (2022), or Schrödinge bridge De Bortoli et al. (2021).” in page 21 needs some work and further elaboration. A sentence that starts with “In conclusion” cannot refer to concepts that have not been discussed before in the text. A couple of suggestions: \n\n- It would be good to mention earlier on that flow matching, rectified flows and stochastic interpolants are essentially the same thing, as these other two terms are just as well-known in the literature as stochastic interpolants (if not more), and to cite Liu et al. (2022); Lipman et al. (2022) earlier on as well.  \n- With regards to the Schrödinger bridge algorithms, while there is a similarity, the algorithms differ: finding Schrödinger bridges is a more challenging task as it involves an optimality condition on top of matching the two marginals. It would be good to discuss that in a sentence or two. \n\nAlso at the bottom of page 21, the authors claim about stochastic interpolants that “some common issues of diffusion-based generative models persist: slow generation, dependence on hyperparameters and neural architectures, and data dependence are the primary drawbacks.” While this is true, these issues are common in all generative models, and are much more acute in most of them; after all, diffusion models and stochastic interpolants / flow matching are the  SOTA approach for most if not all settings that require data generation in continuous domains. For example, with regards to generation speed, sampling EBMs is arguably slower than sampling diffusion models as it requires using MCMC algorithms that have weak convergence guarantees. Moreover, distillation techniques for diffusion models bring down the cost of generating a sample to just a few model evaluations.  \n\nRelated to my previous point, the paper does not emphasize enough the dominance of diffusion models / stochastic interpolants / flow matching as the current generative modeling techniques of choice. The authors need to comment on that, and also on the role of EBMs in the current generative modeling landscape, i.e. where should/are they be used? should/can they be combined with diffusion models? which insights can we take from EBMs?", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "2c14MfVHlc", "reviewer": "Reviewer_BJq4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths** Most of the technical content is correct/rigorously stated. An emphasis is made on links to physical systems (spin lattices, thermodynamic gases etc.), which is helpful to understand links between energy and probability and demystify the term Energy-Based Models. Section 6 contains some recent insights on CD which are not as known as they should be and it is nice to see a paper mentioning them.\n\n\n**Weaknesses**: At a high, level, I don't think the paper lives up to what the title and abstract promise. The paper spends the majority of its page count introducing related concepts (other probabilistic models, maximum entropy principle, sampling techniques). The little space actually detailing their relation to EBMs contains straightforward, high-level content which is far from enough to impose the reader to read such a long background. More specifically:\n\n- Regarding connections between EBMs and other probabilistic models, the paper currently spends nearly 50% of its total page count on introducing other models, and discussing the connection with EBMs in 4 bullets points, which I found to be too straightforward to be informative.\n- Similarly, regarding connections between EBMs and statistical physics, the paper spends multiple pages introducing the maximum entropy principle, but the link to EBMs is limited to\n  - one Lemma which attempts as a link between EBMs and Entropy which relies on an unmotivated identification (F = - \\log Z),\n  - A few high level sentences attempting to relate the Maximum Entropy Principle to training and sampling in EBM at a very high level, which I personally don't understand, and in any case will have at most as much value as two high-level sentences may have. Overall, I fail to see what the maximum entropy principle brings to the table for EBMs.\n- The sampling parts virtually contains only standard textbook content on MH/MALA/ULA.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript contains a description of energy-based models (EBMs) and related physical systems. Additionally, it describes other modern probabilistic models, outlines some points of comparison with EBMs, and describes some EBM training and sampling algorithms.\n\nAfter an introduction highlighting the links between data collection and modelling improvements, the manuscript follows by a review on the probabilistic description of physical systems based on their energy, which inspired EBMs.\nFollowing that, a formal definition of EBMs is given in Section 2.\n\nSection 3 introduces alternative modern probabilistic modelling approaches ---namely Variational Autoencoders, Normalizing Flows, Generative Adverserial Networks and Diffusion Models, and---briefly---compares them to EBMs.\n\nSection 4 is a 5-page primer on basic Monte Carlo Markov Chain techniques used in EBMs, including Metropolis Hasting (MH) Algorithms and the Unadjusted---and Metropolis-Adjusted---Langevin Algorithm (ULA, MALA).\n\nSection 5 (3 pages) contains an introduction of the links between Boltzman-Gibbs distributions and Maximum Entropy.\n\nFinally, Section 6 reviews the Contrastive Divergence algorithm, a training algorithm used to review Energy-Based Models.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "What I would expect, instead, from a review paper on the the relation between EBMs and other generative models is a comprehensive comparison of theoretical and practical insights of the main generative models established by the machine learning community, with as particular focus on EBMs.\n\n- On the theoretical side, I would expect a review on known results regarding some important properties (approximation properties, sample efficiency, ability to handle manifold data, etc.) of some generative models, and whether or not these hold for EBMs, and how do such result compare to analogue results established for other generative models. I would expect formally restating some important results, not only when describing the models, but actually when describing their properties.\n- On the experimental side, I would expect a thorough literature review on works having used EBMs for various applications in recent years, showcasing in which instances they have been a success. A nice touch would be a particular focus on physics/chemistry/thermodynamics applications, given the physics inspiration of EBMs.\n\n**These, in my opinion, form key aspects of a review paper on \"a comprehensive review on the relation with other generative models, sampling and statistical physics\", which need to take up a significant part of the its real estate, and are absent from the current submission.**\n\n\n\nAdditionally, there are some less fundamental, but still important issues with the paper:\n\n- I find the paper to be unnecessarily wordy in places. Conciseness is important to not waste the readers's time, and I encourage the authors to try to systematically reduce redundancy, wordiness, and analogies which sometimes do not add enough to justify the time spent introducing them.\n- Some mathematical results are not properly stated (Lemma 5.1 for instance: \"Maximum Entropy principle\" under which constraints?) \"Constrained minimization of energy functional\" under which constraints?).\n- I appreciate that the focus on CD due to its popularity and connection with sampling. I find it odd to not spend a few paragraphs on score matching and noise contrastive estimation, which are two other popular methods for training EBMs, and arguably simpler to use in practice.\n- I understand that review papers can have a broad introduction and focus on approachability to a large audience, however I believe that this manuscript has sacrificed too much conciseness, precision, clarity and scientific rigor in the process. Below is a thorough list of concerns related to the introduction.\n\n\t1.1 Generative models\n\n\t- \"An intellect which at a certain moment would know all forces [data] that set nature in motion [...] would be uncertain and the future just like the past could be present before its eyes\": the way the quote is reduced is very confusing. Add \"nothing\" before \"uncertain\".\n\t- \"This line of thought was boosted in the previous century with the advent of automatic calculators, and the velocity of development becomes astounding\": syntactically odd sentence.\n\t- \"For instance, consider the remarkable computational power difference between your smartphone and the computer used for the Apollo program by NASA in the 1960s\": probably better to explain which is better explicitly. Also, let's not assume the reader has a smartphone.\n\t- Hence, the quest for data has become an indispensable aspect of contemporary science: logical link is not completely obvious.\n\t- \"To delve deeper into this issue, let us construct a historical metaphor\": not a metaphor, rather an analogy.\n\t- Non-conventional combination of line breaks and newlines.\n\t- \"In recent years, particularly during the 2000s, we have witnessed a profound paradigm shift represented by the Big Data Era. Thanks to the aforementioned technological advancements in computer science, the volume of generated scientific (and not) data has dramatically increased, resulting from advancements in simulation and storage capabilities. Furthermore, there has been a growing collection of data on human activities, including images, text, sounds, and more\". This whole cluster of paragraph needs some restructuring. the logical flow gets very interrupted, there's a lot of jumping back and forth.\n\t- \"Returning to the historical analogy, it is akin to Brahe suddenly providing Kepler with a thousand times the amount of data that the latter was accustomed to. This shift posed a methodological problem in what we now refer to as data science\". The second sentence relates to the previous paragraph. It's odd to have it bundled with the first sentence which is unrelated.\n\t- \"not just on the internet. \" Don't see what this brings to the point. Also, the internet was not mentioned before, its not clear why the sentence deserves a clause like this.\n\t- \"Generative models represent, in this sense, the most recent breakthrough in technological advancement towards intelligent-like machines. It is complicated to provide a general definition, and there are already many available from different sources5. However, if we informally focus on those already known to the general public, such as Generative Pre-Trained Transformers (GPT)6, the common traits of most definitions are few. Firstly, generative models require a substantial amount of data for training, in addition to the selection of a precise architecture, which goes far beyond the original perceptron. Secondly, the training is probably not biologically inspired, i.e., we do not learn through backpropagationGrossberg (1987), which is the most commonly used training technique in machine learning. For completeness, it is worth noting that this thesis is still debated in neuroscienceLillicrap et al. (2020). Thirdly, a generative model is not necessarily informative about the data distribution; for instance, ChatGPT could achieve astounding results in text generation, but the training machine does not provide knowledge about some general features of text generated by humans\". How about: \"generative models model the observed data through a probabilistic program, able to output samples from the distributions\"? The GPT-class of models, being autoregressive, fit that definition. I don't find the \"training not biologically inspired\" or \"large training set\" aspects relevant to the definition. For that reason, I'm not a big fan of placing that much importance on \"generative modelling\" as IMO, EBMs are not generative models per se, they are **probabilistic models** (e.g. models of the probability of some random variable). They don't define a generative process.\n\t- \"For completeness, it is worth noting that this thesis is still debated in neuroscienceLillicrap et al. (2020). \" this probably should be put in a footnote to not disturb the flow.\n\t- \"Thirdly, a generative model is not necessarily informative about the data distribution; for instance, ChatGPT could achieve astounding results in text generation, but the training machine does not provide knowledge about some general features of text generated by humans.\". True, but a negative definition, so not a super strong point to make when defining generative models.\n\t- \"BraheGPT,\"  -> \"BraheGPT\",\n\t- \"The generative models that excel in terms of interpretability, which form the main focus of the present work, are precisely the Energy- Based Models (EBMs). These models offer a unique advantage in their ability to provide insights into the underlying mechanisms of the data they generate.\" I think the arguments in favor of the interpretability properties of EBMs should be clearly explained. Right now, this sentence is not backed up.\n\t- \"In adopting EBMs, researchers and practitioners gain not only the capacity to generate high-quality data but also a clearer understanding of the factors influencing the generated outputs. This interpretability is indispensable in domains where the model’s ability to convey meaningful information about the dataset is paramount\" lots of overlap with the previous paragraph.\n\n\n\t1.2\n\n\t- The first ingredient of the story: does not really motivate how this ``ingredient'''will matter. Actually, by the end of this paragraph (after equation 1), we still don't get why Boltzmann and Gibbs contributions are relevant to EBMs.\n\t- \"The analysis of the impact of Boltzmann-Gibbs ensemble on physics would require a full monography per se; for the sake of the present work\" this is not very concise. I'd suggest to just remove this sentence.\n\t-  \"particularly in understanding the behavior of spins in a lattice Λ — for simplicity, we can imagine a graph with N nodes\" -> \"which the Ising Model represents as a graph with N nodes\".\n\t- Maybe \"usually denoted as \"up\" or \"down\"\"  -> \"... two possible values: +1 (\"up\") and -1 (\"down\")\", yielding a $\\Omega = \\{ -1, 1\\}^N$ as set of possible lattice configurations.\n\t- \"The interactions between spins are typically modeled using a simple energy function, namely\" -> \"The energy of the system is modeled as\": ...\n\t- \"Let us briefly clarify the notation:\" -> \"where\" (for conciseness).\n\t- <ij>: is this standard notation? I've never seen it. Otherwise, I would advise the authors use standard notation especially that <...> is sometimes used to denote average in physics.\n\t- \"In considering equation 1 as associated to UIs\": not the most elegant formulation\n\t- \"magnetization\" and \"long-range orders\" are not defined, so the sentences mentioning it feel a bit obscure.\n\n\tVery minor:\n\t- Typo: This variational formulation identify  -> \"identifies\".\n\t- Please use \\citep to cite papers.\n\t- \"monography\": monograph is more standard I believe. (section 1.2)", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QV2z16D6C1", "reviewer": "Reviewer_izai", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is a survey/review paper. This reviewer is not exactly familiar the criterions for reviewing an review paper, but felt the paper is overall a bit unorganized and it is unclear what to take away from it. To be specific:\n\nStrength:\n\n- The paper is quite comprehensive in the materials it presents as it covers many aspects of EBMs. \n\nWeaknesses:\n\n- The text in the paper is mainly about general information about EBMs and other types of generative models. It is not very dense in the state-of-the-art approaches in EBMs, or the connections between EBMs and other types of generative models.\n- This reviewer wasn't able to take away key insights from the paper. This reviewer felt a lot of the content are more like a tutorial or general introduction to the topic, which may be found in existing literature.\n- With a lot of general information, the paper is rather lenghty, of 41 pages in general. The paper is aimed to be a \"Hitchhiker’s guide on Energy-Based Models\". However,  I felt the information flow is not very condensed or easy to follow.\n- Some definitions and theorems are not stated in an organized and rigorous way.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a comprehensive review of Energy-Based Models (EBMs) and their relation to other generative models and statistical mechanics principles. The paper discusses various EBM training and sampling techniques and highlights the parallels between EBM concepts and statistical mechanics.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "I think key insights and key results, as well as practical implications needed to be better highlighted. In the current form I felt the paper is not particularly helpful for a researcher trying to learn about EBMs -- what they are good for, when do they fail, how do they connect to other state-of-the-art techniques.", "broader_impact_concerns": "There is no ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper provides an interesting and comprehensive resource on energy-based models viewed from a physicist's point of view, along with links to other generative models. In this sense, it could be a valuable contribution to TMLR.

Nevertheless, reviewers pointed out some concerns about the manuscript in its current form, and some major changes are necessary in order to make it suitable for publication. Here are some recommendations (inspired by the reviewers' final recommendations, see below):
* There is currently a little too much broad introductory material, e.g. in Sections 1.1 and 1.2, which should be shortened. Also please include introductory text before 1.1 which clarifies the goal of the survey, ideally comparing to related surveys on EBMs, such as [this one](https://arxiv.org/abs/2101.03288). Also, move the organization bullets in p.5 to a separate section.
* Sections 3 and 4 are somewhat unrelated to the main focus on EBMs: these could be merged into a single section about links with other generative models, and significantly shortened, given that some of these (e.g., GANs and interpolants) aren't so closely related to EBMs.
* Section 7 could be expanded to cover more recent developments in EBM training, which would make the survey more valuable to the community. The term "state-of-the-art" seems somewhat misplaced to described an old method like CD, perhaps "Recent progress in EBM training" could be more appropriate?
* It would be useful to add a final section highlighting potential new directions and open problems that arise based on the proposed viewpoint, and that could benefit from more exchange between the ML and statistical physics communities.


-------
For reference, I am including here some comments by two out of three reviewers who are leaning on the negative side:
>Currently, a large part of the paper is dedicated to introducing other models, or introducing components related to EBMs (section 6, section 5), with only little content left focused on EBMs themselves, apart from section 7, which overlaps with existing review papers on training energy based models (Song et. al, 2021). Therefore, I am still leaning towards rejection. To be accepted, the paper should either:
>
> * include more new EBM-specific content (I would not hesitate to state technical results in their entirety in order to anchor clearly the properties of EBMs), and less non-EBM specific/introductory content.
> * or position itself as a general introduction to generative models (in which case, the sections related to components of EBMs should be removed)

> I think the key weakness still remains as in it's unclear what a reader can take away from a long read. As another reviewer also pointed out, there is a lot of introductory material, while the discussion on the connection between EBMs and other topics is currently NOT taking a significant enough portion of the paper. A "Hitchhiker’s guide" should articulate the relations very clearly and perhaps discuss practical implications. While introductory materials might be useful to some readers, in this reviewer's opinion, such materials may be put in an appendix and referenced when needed. The main paper can then focus on the relations/insights.
Decision: Accept (poster)