Original: G ENERALIZED E NERGY B ASED M ODELS
Revised: G ENERALIZED E NERGY B ASED M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce the Generalized Energy Based Model (GEBM) for generative modelling.
Revised: We introduce the Generalized Energy Based Model (GEBM) for generative modelling.

Original: These models combine two trained components: a base distribution (generally animplicitmodel), whichcanlearnthesupportofdatawithlowintrinsicdimensionin ahighdimensionalspace; andanenergyfunction, toreﬁnetheprobabilitymassonthe learned support.
Revised: These models combine two trained components: a base distribution (generally animplicitmodel), whichcanlearnthesupportofdatawithlowintrinsicdimensionin ahighdimensionalspace; andanenergyfunction, toreﬁnetheprobabilitymassonthe learned support.

Original: Both the energy function and base jointly constitute the ﬁnal model, unlike GANs, which retain only the base distribution (the "generator").
Revised: Both the energy function and base jointly constitute the ﬁnal model, unlike GANs, which retain only the base distribution (the "generator").

Original: GEBMs are trained by alternating between learning the energy and the base.
Revised: GEBMs are trained by alternating between learning the energy and the base.

Original: We show that both training stages are well-deﬁned: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base.
Revised: We show that both training stages are well-deﬁned: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base.

Original: Samples from the posterior on the latent space of the trained model canbeobtainedviaMCMC,thusﬁndingregionsinthisspacethatproducebetterqualitysamples.
Revised: Samples from the posterior on the latent space of the trained model canbeobtainedviaMCMC,thusﬁndingregionsinthisspacethatproducebetterqualitysamples.

Original: Empirically,theGEBMsamplesonimage-generationtasksareofmuch better quality than those from the learned generator alone, indicating that all else being equal, the GEBM will outperform a GAN of the same complexity.
Revised: Empirically,theGEBMsamplesonimage-generationtasksareofmuch better quality than those from the learned generator alone, indicating that all else being equal, the GEBM will outperform a GAN of the same complexity.

Original: When using normalizing ﬂows as base measures, GEBMs succeed on density modelling tasks, returning comparable performance to direct maximum likelihood of the same networks.
Revised: When using normalizing ﬂows as base measures, GEBMs succeed on density modelling tasks, returning comparable performance to direct maximum likelihood of the same networks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Energy-based models (EBMs) have a long history in physics, statistics and machine learning (LeCun et al., 2006).
Revised: Energy-based models (EBMs) have a long history in physics, statistics and machine learning (LeCun et al., 2006).

Original: They belong to the class of explicit models, and can be described by a family of energies
Revised: They belong to the class of explicit models, and can be described by a family of energies

Original: E which deﬁne probability distributions with density proportional to exp( − E ) .
Revised: E which deﬁne probability distributions with density proportional to exp( − E ) .

Original: Those models are often known up to a normalizing constant Z ( E ) , also called the partition function .
Revised: Those models are often known up to a normalizing constant Z ( E ) , also called the partition function .

Original: The learning task consists of ﬁnding an optimal function that best describes a given system or target distribution P .
Revised: The learning task consists of ﬁnding an optimal function that best describes a given system or target distribution P .

Original: This can be achieved using maximum likelihood estimation (MLE), however the intractability of the normalizing partition function makes this learning task challenging.
Revised: This can be achieved using maximum likelihood estimation (MLE), however the intractability of the normalizing partition function makes this learning task challenging.

Original: Thus, various methods have been proposed to address this (Hinton, 2002; Hyvärinen, 2005; Gutmann and Hyvärinen, 2012; Dai et al., 2019a;b).
Revised: Thus, various methods have been proposed to address this (Hinton, 2002; Hyvärinen, 2005; Gutmann and Hyvärinen, 2012; Dai et al., 2019a;b).

Original: All these methods estimate EBMs that are supported over the whole space.
Revised: All these methods estimate EBMs that are supported over the whole space.

Original: In many applications, however, P is believed to be supported on an unknown lower dimensional manifold.
Revised: In many applications, however, P is believed to be supported on an unknown lower dimensional manifold.

Original: This happens in particular when there are strong dependencies between variables in the data, and suggests incorporating a low-dimensionality hypothesis in the model.
Revised: This happens in particular when there are strong dependencies between variables in the data, and suggests incorporating a low-dimensionality hypothesis in the model.

Original: Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are a particular way to enforce low dimensional structure in a model.
Revised: Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are a particular way to enforce low dimensional structure in a model.

Original: They rely on an implicit model, the generator, to produce samples supported on a low-dimensional manifold by mapping a pre-deﬁned latent noise to the sample space using a trained function.
Revised: They rely on an implicit model, the generator, to produce samples supported on a low-dimensional manifold by mapping a pre-deﬁned latent noise to the sample space using a trained function.

Original: GANs have been very successful in generating high-quality samples on various tasks, especially for unsupervised image generation (Brock et al., 2018).
Revised: GANs have been very successful in generating high-quality samples on various tasks, especially for unsupervised image generation (Brock et al., 2018).

Original: The generator is trained adversarially against a discriminator network whose goal is to distinguish samples produced by the generator from the target data.
Revised: The generator is trained adversarially against a discriminator network whose goal is to distinguish samples produced by the generator from the target data.

Original: This has inspired further research to extend the training procedure to more general losses (Nowozin et al., 2016; Arjovsky et al., 2017; Li et al., 2017; Bi´nkowski et al., 2018;
Revised: This has inspired further research to extend the training procedure to more general losses (Nowozin et al., 2016; Arjovsky et al., 2017; Li et al., 2017; Bi´nkowski et al., 2018;

Original: Arbel et al., 2018) and to improve its stability (Miyato et al., 2018;
Revised: Arbel et al., 2018) and to improve its stability (Miyato et al., 2018;

Original: Gulrajani et al., 2017; Nagarajan and Kolter, 2017; Kodali et al., 2017).
Revised: Gulrajani et al., 2017; Nagarajan and Kolter, 2017; Kodali et al., 2017).

Original: While the generator of a GAN has effectively a low-dimensional support, it remains challenging to reﬁne the distribution of mass on that support using pre-deﬁned latent noise.
Revised: While the generator of a GAN has effectively a low-dimensional support, it remains challenging to reﬁne the distribution of mass on that support using pre-deﬁned latent noise.

Original: For instance, as shown by Cornish et al.
Revised: For instance, as shown by Cornish et al.

Original: (2020) for normalizing ﬂows, when the latent distribution is unimodal and the target distribution possesses multiple disconnected low-dimensional components, the generator, as a continuous map, compensates for this mismatch using steeper slopes.
Revised: (2020) for normalizing ﬂows, when the latent distribution is unimodal and the target distribution possesses multiple disconnected low-dimensional components, the generator, as a continuous map, compensates for this mismatch using steeper slopes.

Original: In practice, this implies the need for more complicated generators.
Revised: In practice, this implies the need for more complicated generators.

Original: In the present work, we propose a new class of models, called Generalized Energy Based Models (GEBMs), which can represent distributions supported on low-dimensional manifolds, while offering more ﬂexibility in reﬁning the mass on those manifolds.
Revised: In the present work, we propose a new class of models, called Generalized Energy Based Models (GEBMs), which can represent distributions supported on low-dimensional manifolds, while offering more ﬂexibility in reﬁning the mass on those manifolds.

Original: GEBMs combine the strength of both implicit and explicit models in two separate components: a base distribution (often chosen to be an implicit model) which learns the low-dimensional support of the data, and an energy function that can reﬁne the probability mass on that learned support.
Revised: GEBMs combine the strength of both implicit and explicit models in two separate components: a base distribution (often chosen to be an implicit model) which learns the low-dimensional support of the data, and an energy function that can reﬁne the probability mass on that learned support.

Original: We propose to train the GEBM by alternating between learning the energy and the base, analogous to f -GAN training (Goodfellow et al., 2014; Nowozin et al., 2016).
Revised: We propose to train the GEBM by alternating between learning the energy and the base, analogous to f -GAN training (Goodfellow et al., 2014; Nowozin et al., 2016).

Original: The energy is learned by maximizing a generalized notion of likelihood which we relate to the Donsker-Varadhan lower-bound (Donsker and Varadhan, 1975) and Fenchel duality , as in (Nguyen et al., 2010; Nowozin et al., 2016).
Revised: The energy is learned by maximizing a generalized notion of likelihood which we relate to the Donsker-Varadhan lower-bound (Donsker and Varadhan, 1975) and Fenchel duality , as in (Nguyen et al., 2010; Nowozin et al., 2016).

Original: Although the partition function is intractable in general, we propose a method to learn it in an amortized fashion without introducing additional surrogate models, as done in variational inference (Kingma and Welling, 2014; Rezende et al., 2014) or by Dai et al.
Revised: Although the partition function is intractable in general, we propose a method to learn it in an amortized fashion without introducing additional surrogate models, as done in variational inference (Kingma and Welling, 2014; Rezende et al., 2014) or by Dai et al.

Original: (2019a;b).
Revised: (2019a;b).

Original: The resulting maximum likelihood estimate, the KL Approximate Lower-bound Estimate ( KALE ) , is then used as a loss for training the base.
Revised: The resulting maximum likelihood estimate, the KL Approximate Lower-bound Estimate ( KALE ) , is then used as a loss for training the base.

Original: When the class of energies is rich and smooth enough, we show that KALE leads to a meaningful criterion for measuring weak convergence of probabilities.
Revised: When the class of energies is rich and smooth enough, we show that KALE leads to a meaningful criterion for measuring weak convergence of probabilities.

Original: Following recent work by Chu et al.
Revised: Following recent work by Chu et al.

Original: Sanjabi et al.
Revised: Sanjabi et al.

Original: , we show that KALE possesses well deﬁned gradients w.r.t.
Revised: , we show that KALE possesses well deﬁned gradients w.r.t.

Original: the parameters of the base, ensuring well-behaved training.
Revised: the parameters of the base, ensuring well-behaved training.

Original: We also provide convergence rates for the empirical estimator of KALE when the variational family is sufﬁciently well behaved, which may be of independent interest.
Revised: We also provide convergence rates for the empirical estimator of KALE when the variational family is sufﬁciently well behaved, which may be of independent interest.

Original: The main advantage of GEBMs becomes clear when sampling from these models: the posterior over the latents of the base distribution incorporates the learned energy, putting greater mass on regions in this latent space that lead to better quality samples.
Revised: The main advantage of GEBMs becomes clear when sampling from these models: the posterior over the latents of the base distribution incorporates the learned energy, putting greater mass on regions in this latent space that lead to better quality samples.

Original: Sampling from the GEBM can thus be achieved by ﬁrst sampling from the posterior distribution of the latents via MCMC in the low-dimensional latent space, then mapping those latents to the input space using the implicit map of the base.
Revised: Sampling from the GEBM can thus be achieved by ﬁrst sampling from the posterior distribution of the latents via MCMC in the low-dimensional latent space, then mapping those latents to the input space using the implicit map of the base.

Original: This is in contrast to standard GANs, where the latents of the base have a ﬁxed distribution.
Revised: This is in contrast to standard GANs, where the latents of the base have a ﬁxed distribution.

Original: We focus on a class of samplers that exploit gradient information, and show that these samplers enjoy fast convergence properties by leveraging the recent work of Eberle et al.
Revised: We focus on a class of samplers that exploit gradient information, and show that these samplers enjoy fast convergence properties by leveraging the recent work of Eberle et al.

Original: While there has been recent interest in using the discriminator to improve the quality of the generator during sampling (Azadi et al., 2019;
Revised: While there has been recent interest in using the discriminator to improve the quality of the generator during sampling (Azadi et al., 2019;

Original: Turner et al., 2019; Neklyudov et al., 2019; Grover et al., 2019; Tanaka, 2019; Wu et al., 2019b), our approach emerges naturally from the model we consider.
Revised: Turner et al., 2019; Neklyudov et al., 2019; Grover et al., 2019; Tanaka, 2019; Wu et al., 2019b), our approach emerges naturally from the model we consider.

Original: We begin in Section 2 by introducing the GEBM model.
Revised: We begin in Section 2 by introducing the GEBM model.

Original: In Section 3, we describe the learning procedure using KALE, then derive a method for sampling from the learned model in Section 4.
Revised: In Section 3, we describe the learning procedure using KALE, then derive a method for sampling from the learned model in Section 4.

Original: In
Revised: In

Original: Section 5 we discuss related work.
Revised: Section 5 we discuss related work.

Original: Finally, experimental results are presented in Section 6.
Revised: Finally, experimental results are presented in Section 6.

Original:  G ENERALIZED E NERGY -B ASED M ODELS
Revised:  G ENERALIZED E NERGY -B ASED M ODELS

Original: Models are learned using either a GAN, GEBM, or EBM.
Revised: Models are learned using either a GAN, GEBM, or EBM.

Original: More details are provided in Appendix G.3.
Revised: More details are provided in Appendix G.3.

Original: In this section, we introduce generalized energy based models (GEBM), that combine the strengths of both energy-based models and implicit generative models, and admit the ﬁrst of these as a special case.
Revised: In this section, we introduce generalized energy based models (GEBM), that combine the strengths of both energy-based models and implicit generative models, and admit the ﬁrst of these as a special case.

Original: An energy-based model (EBM) is deﬁned by a set E of real valued functions called energies , where each E ∈E speciﬁes a probability density over the data space X ⊂ R d up to a normalizing constant,
Revised: An energy-based model (EBM) is deﬁned by a set E of real valued functions called energies , where each E ∈E speciﬁes a probability density over the data space X ⊂ R d up to a normalizing constant,

Original: [Equation]
Revised: [Equation]

Original: While EBMs have been shown recently to be powerful models for representing complex high dimensional data distributions, they still unavoidably lead to a blurred model whenever data are concentrated on a lower-dimensional manifold.
Revised: While EBMs have been shown recently to be powerful models for representing complex high dimensional data distributions, they still unavoidably lead to a blurred model whenever data are concentrated on a lower-dimensional manifold.

Original: This is the case in Figure 1(a), where the ground truth distribution is supported on a 1-D line and embedded in a 2-D space.
Revised: This is the case in Figure 1(a), where the ground truth distribution is supported on a 1-D line and embedded in a 2-D space.

Original: The EBM in Figure 1(d) learns to give higher density to a halo surrounding the data, and thus provides a blurred representation.
Revised: The EBM in Figure 1(d) learns to give higher density to a halo surrounding the data, and thus provides a blurred representation.

Original: That is a consequence of
Revised: That is a consequence of

Original: EBM having a density deﬁned over the whole space, and can result in blurred samples for image models.
Revised: EBM having a density deﬁned over the whole space, and can result in blurred samples for image models.

Original: An implicit generative model (IGM) is a family of probability distributions G θ parametrized by a learnable generator function G : Z (cid:55)→ X that maps latent samples z from a ﬁxed latent distribution η to the data space X .
Revised: An implicit generative model (IGM) is a family of probability distributions G θ parametrized by a learnable generator function G : Z (cid:55)→ X that maps latent samples z from a ﬁxed latent distribution η to the data space X .

Original: The latent distribution η is required to have a density over the latent space Z and is often easy to sample from.
Revised: The latent distribution η is required to have a density over the latent space Z and is often easy to sample from.

Original: Thus, Sampling from G is simply achieved by ﬁrst sampling z from η then applying G ,
Revised: Thus, Sampling from G is simply achieved by ﬁrst sampling z from η then applying G ,

Original: [Equation]
Revised: [Equation]

Original: GANs are popular instances of these models, and are trained adversarially (Goodfellow et al., 2014).
Revised: GANs are popular instances of these models, and are trained adversarially (Goodfellow et al., 2014).

Original: When the latent space Z has a smaller dimension than the input space X , the IGM will be supported on a lower dimensional manifold of X , and thus will not possess a Lebesgue density on X (Bottou et al., 2017).
Revised: When the latent space Z has a smaller dimension than the input space X , the IGM will be supported on a lower dimensional manifold of X , and thus will not possess a Lebesgue density on X (Bottou et al., 2017).

Original: IGMs are therefore good candidates for modelling low dimensional distributions.
Revised: IGMs are therefore good candidates for modelling low dimensional distributions.

Original: While
Revised: While

Original: GANs can accurately learn the low-dimensional support of the data, they can have limited power for representing the distribution of mass on the support.
Revised: GANs can accurately learn the low-dimensional support of the data, they can have limited power for representing the distribution of mass on the support.

Original: This is illustrated in Figure 1(b).
Revised: This is illustrated in Figure 1(b).

Original: A generalized energy-based model (GEBM) Q is deﬁned by a combination of a base G and an energy E deﬁned over a subset X of R d .
Revised: A generalized energy-based model (GEBM) Q is deﬁned by a combination of a base G and an energy E deﬁned over a subset X of R d .

Original: The base component can typically be chosen to be an IGM as in (2).
Revised: The base component can typically be chosen to be an IGM as in (2).

Original: The generalized energy component can reﬁne the mass on the support deﬁned by the base .
Revised: The generalized energy component can reﬁne the mass on the support deﬁned by the base .

Original: It belongs to a class E of real valued functions deﬁned on the input space X , and represents the negative log-density of a sample from the GEBM with respect to the base G ,
Revised: It belongs to a class E of real valued functions deﬁned on the input space X , and represents the negative log-density of a sample from the GEBM with respect to the base G ,

Original: [Equation]
Revised: [Equation]

Original: where A G ,E is the logarithm of the normalizing constant of the model w.r.t.
Revised: where A G ,E is the logarithm of the normalizing constant of the model w.r.t.

Original: G .
Revised: G .

Original: Thus, a GEBM Q re-weights samples from the base according to the un-normalized importance weights exp( − E ( x )) .
Revised: Thus, a GEBM Q re-weights samples from the base according to the un-normalized importance weights exp( − E ( x )) .

Original: Using the latent structure of the base G , this importance weight can be pulled-back to the latent space to deﬁne a posterior latent distribution ν ,
Revised: Using the latent structure of the base G , this importance weight can be pulled-back to the latent space to deﬁne a posterior latent distribution ν ,

Original: [Equation]
Revised: [Equation]

Original: Hence, the posterior latent ν can be used instead of the latent noise η for sampling from Q , as summarized by Proposition 1:
Revised: Hence, the posterior latent ν can be used instead of the latent noise η for sampling from Q , as summarized by Proposition 1:

Original: Proposition 1.
Revised: Proposition 1.

Original: Sampling from Q requires sampling a latent z from ν (4) then applying the map G ,
Revised: Sampling from Q requires sampling a latent z from ν (4) then applying the map G ,

Original: [Equation]
Revised: [Equation]

Original: In order to hold, Proposition 1 does not need the generator G to be invertible.
Revised: In order to hold, Proposition 1 does not need the generator G to be invertible.

Original: We provide a proof in
Revised: We provide a proof in

Original: Appendix C.1 which relies on a characterization of probability distribution using generalized moments.
Revised: Appendix C.1 which relies on a characterization of probability distribution using generalized moments.

Original: We will see later in Section 4 how equation (5) can be used to provide practical sampling algorithms from the GEBM.
Revised: We will see later in Section 4 how equation (5) can be used to provide practical sampling algorithms from the GEBM.

Original: Next we discuss the advantages of GEBMs.
Revised: Next we discuss the advantages of GEBMs.

Original: Advantages of Generalized Energy Based Models.
Revised: Advantages of Generalized Energy Based Models.

Original: The GEBM deﬁned by (3) can be related to exponential tilting (re-weighting)
Revised: The GEBM deﬁned by (3) can be related to exponential tilting (re-weighting)

Original: (Siegmund, 1976; Xie et al., 2016) of the base G .
Revised: (Siegmund, 1976; Xie et al., 2016) of the base G .

Original: The important difference over classical EBMs is that the base G is allowed to change its support and shape in space.
Revised: The important difference over classical EBMs is that the base G is allowed to change its support and shape in space.

Original: By learning the base G , GEBMs can accurately learn the low-dimensional support of data, just like
Revised: By learning the base G , GEBMs can accurately learn the low-dimensional support of data, just like

Original: IGMs do.
Revised: IGMs do.

Original: They also beneﬁt from the ﬂexibility of EBMs for representing densities using an energy E to reﬁne distribution of mass on the support deﬁned by G , as seen in Figure 1(c).
Revised: They also beneﬁt from the ﬂexibility of EBMs for representing densities using an energy E to reﬁne distribution of mass on the support deﬁned by G , as seen in Figure 1(c).

Original: Compared to EBMs , that put mass on the whole space by construction (positive density), GEBMs have the additional ﬂexibility to concentrate the probability mass on a low-dimensional support learned by the base G , provided that the dimension of the latent space Z is smaller than the dimension of the ambient space X : see Figure 1(c) vs Figure 1(d).
Revised: Compared to EBMs , that put mass on the whole space by construction (positive density), GEBMs have the additional ﬂexibility to concentrate the probability mass on a low-dimensional support learned by the base G , provided that the dimension of the latent space Z is smaller than the dimension of the ambient space X : see Figure 1(c) vs Figure 1(d).

Original: In the particular case when the dimension of Z is equal to the ambient dimension and G is invertible, the base G becomes supported over the whole space
Revised: In the particular case when the dimension of Z is equal to the ambient dimension and G is invertible, the base G becomes supported over the whole space

Original: X , and GEBM recover usual EBMs.
Revised: X , and GEBM recover usual EBMs.

Original: The next proposition further shows that any EBM can be viewed as a particular cases of GEBMs, as proved in Appendix C.1.
Revised: The next proposition further shows that any EBM can be viewed as a particular cases of GEBMs, as proved in Appendix C.1.

Original: Proposition 2.
Revised: Proposition 2.

Original: Any EBM with energy E (as in (1)) can be expressed as a GEBM with base G given as a normalizingﬂowwithdensity exp ( − r ( x ))
Revised: Any EBM with energy E (as in (1)) can be expressed as a GEBM with base G given as a normalizingﬂowwithdensity exp ( − r ( x ))

Original: andageneralizedenergy ˜
Revised: andageneralizedenergy ˜

Original: E ( x )=
Revised: E ( x )=

Original: E ( x ) − r ( x ) .
Revised: E ( x ) − r ( x ) .

Original: Inthisparticular case, the dimension of the latent is necessarily equal to the data dimension, i.e. dim ( Z )=
Revised: Inthisparticular case, the dimension of the latent is necessarily equal to the data dimension, i.e. dim ( Z )=

Original: dim ( X ) .
Revised: dim ( X ) .

Original: Compared to IGMs , that rely on a ﬁxed pre-determined latent noise distribution η , GEBMs offer the additional ﬂexibility of learning a richer latent noise distribution.
Revised: Compared to IGMs , that rely on a ﬁxed pre-determined latent noise distribution η , GEBMs offer the additional ﬂexibility of learning a richer latent noise distribution.

Original: This is particularly useful when the data is multimodal.
Revised: This is particularly useful when the data is multimodal.

Original: In IGMs, such a GANs, the latent noise η is usually unimodal thus requiring a more sophisticated generator to distort a unimodal noise distribution into a distribution with multiple modes, as shown by Cornish et al.
Revised: In IGMs, such a GANs, the latent noise η is usually unimodal thus requiring a more sophisticated generator to distort a unimodal noise distribution into a distribution with multiple modes, as shown by Cornish et al.

Original: Instead, GEBMs allow to sample from a posterior ν over the latent noise deﬁned in (4).
Revised: Instead, GEBMs allow to sample from a posterior ν over the latent noise deﬁned in (4).

Original: This posterior noise can be multimodal in latent space (by incorporating information from the energy) and thus can put more or less mass in speciﬁc regions of the manifold deﬁned by the base G .
Revised: This posterior noise can be multimodal in latent space (by incorporating information from the energy) and thus can put more or less mass in speciﬁc regions of the manifold deﬁned by the base G .

Original: This allows GEBMs to capture multimodality in data, provided the support of the base is broad enough to subsume the data support Figure 1(c).
Revised: This allows GEBMs to capture multimodality in data, provided the support of the base is broad enough to subsume the data support Figure 1(c).

Original: The base can be simpler, compared to GANs, as it doesn’t need to distort the input noise too much to produce multimodal samples (see
Revised: The base can be simpler, compared to GANs, as it doesn’t need to distort the input noise too much to produce multimodal samples (see

Original: Figure 8 in Appendix G.4).
Revised: Figure 8 in Appendix G.4).

Original: This additional ﬂexibility comes at no additional training cost compared to
Revised: This additional ﬂexibility comes at no additional training cost compared to

Original: GANs.
Revised: GANs.

Original: Indeed, GANs still require another model during training, the discriminator network, but do not use it for sampling.
Revised: Indeed, GANs still require another model during training, the discriminator network, but do not use it for sampling.

Original: Instead, GEBMs avoid this waist since the base and energy can be trained jointly, with no other additional model, and then both are used for sampling.
Revised: Instead, GEBMs avoid this waist since the base and energy can be trained jointly, with no other additional model, and then both are used for sampling.

Original:  L EARNING GEBM S
Revised:  L EARNING GEBM S

Original: In this section we describe a general procedure for learning GEBMs.
Revised: In this section we describe a general procedure for learning GEBMs.

Original: We decompose the learning procedure into two steps: an energy learning step and a base learning step.
Revised: We decompose the learning procedure into two steps: an energy learning step and a base learning step.

Original: The overall learning procedure alternates between these two steps, as done in GAN training (Goodfellow et al., 2014).
Revised: The overall learning procedure alternates between these two steps, as done in GAN training (Goodfellow et al., 2014).

Original: 3. E NERGY LEARNING
Revised: 3. E NERGY LEARNING

Original: When the base G is ﬁxed, varying the energy E leads to a family of models that all admit a density exp( − E − A G ,E ) w.r.t.
Revised: When the base G is ﬁxed, varying the energy E leads to a family of models that all admit a density exp( − E − A G ,E ) w.r.t.

Original: G .
Revised: G .

Original: When the base G admits a density exp( − r ) deﬁned over the whole space, it is possible to learn the energy E by maximizing the likelihood of the model − (cid:82) ( E + r )
Revised: When the base G admits a density exp( − r ) deﬁned over the whole space, it is possible to learn the energy E by maximizing the likelihood of the model − (cid:82) ( E + r )

Original: d P − A G ,E .
Revised: d P − A G ,E .

Original: However, in general G is supported on a lower-dimensional manifold so that r is ill-deﬁned and the usual notion of likelihood cannot be used.
Revised: However, in general G is supported on a lower-dimensional manifold so that r is ill-deﬁned and the usual notion of likelihood cannot be used.

Original: Instead, we introduce a generalized notion of likelihood which does not require a well deﬁned density exp( − r ) for G :
Revised: Instead, we introduce a generalized notion of likelihood which does not require a well deﬁned density exp( − r ) for G :

Original: Deﬁnition 1 (Generalized Likelihood) .
Revised: Deﬁnition 1 (Generalized Likelihood) .

Original: The expected G -log-likelihood under a target distribution P of a GEBM model Q with base G and energy E is deﬁned as
Revised: The expected G -log-likelihood under a target distribution P of a GEBM model Q with base G and energy E is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: To provide intuitions about the generalized likelihood in Deﬁnition 1, we start by discussing the particular case where KL ( P || G ) <
Revised: To provide intuitions about the generalized likelihood in Deﬁnition 1, we start by discussing the particular case where KL ( P || G ) <

Original: + ∞ .
Revised: + ∞ .

Original: We then present the training method in the general case where P and G might not share the same support, i.e. KL ( P || G )=
Revised: We then present the training method in the general case where P and G might not share the same support, i.e. KL ( P || G )=

Original: + ∞ .
Revised: + ∞ .

Original: Special case of ﬁnite KL ( P || G ) .
Revised: Special case of ﬁnite KL ( P || G ) .

Original: When the Kullback-Leibler divergence between P and G is well deﬁned, (6) corresponds to the Donsker-Varadhan (DV) lower bound on the KL (Donsker and Varadhan, 1975), meaning that KL ( P || G ) ≥L P , G ( E ) for all E .
Revised: When the Kullback-Leibler divergence between P and G is well deﬁned, (6) corresponds to the Donsker-Varadhan (DV) lower bound on the KL (Donsker and Varadhan, 1975), meaning that KL ( P || G ) ≥L P , G ( E ) for all E .

Original: Moreover, the following proposition holds:
Revised: Moreover, the following proposition holds:

Original: Proposition 3.
Revised: Proposition 3.

Original: Assume that KL ( P || G ) < + ∞ and 0 ∈E .
Revised: Assume that KL ( P || G ) < + ∞ and 0 ∈E .

Original: If, in addition, E (cid:63) maximizes (6), then:
Revised: If, in addition, E (cid:63) maximizes (6), then:

Original: [Equation]
Revised: [Equation]

Original: In addition, we have that KL ( P || Q )=0 when E (cid:63) is the negative log-density ratio of P w.r.t.
Revised: In addition, we have that KL ( P || Q )=0 when E (cid:63) is the negative log-density ratio of P w.r.t.

Original: G .
Revised: G .

Original: We refer to Appendix C.1 for a proof.
Revised: We refer to Appendix C.1 for a proof.

Original: According to (7), the GEBM systematically improves over the
Revised: According to (7), the GEBM systematically improves over the

Original: IGM deﬁned by G , with no further improvement possible in the limit case when G = P .
Revised: IGM deﬁned by G , with no further improvement possible in the limit case when G = P .

Original: Hence as long as there is an error in mass on the common support of P and G , the GEBM improves over the base G .
Revised: Hence as long as there is an error in mass on the common support of P and G , the GEBM improves over the base G .

Original: Estimating the likelihood in the General setting.
Revised: Estimating the likelihood in the General setting.

Original: Deﬁnition 1 can be used to learn a maximum likelihood energy E (cid:63) by maximizing L P , G ( E ) w.r.t.
Revised: Deﬁnition 1 can be used to learn a maximum likelihood energy E (cid:63) by maximizing L P , G ( E ) w.r.t.

Original: E even when the KL ( P || G ) is inﬁnite and when P and G don’t necessarily share the same support.
Revised: E even when the KL ( P || G ) is inﬁnite and when P and G don’t necessarily share the same support.

Original: Such an optimal solution is well deﬁned whenever the set of energies is suitably constrained.
Revised: Such an optimal solution is well deﬁned whenever the set of energies is suitably constrained.

Original: This is the case if the energies are parametrized by a compact set Ψ with ψ (cid:55)→ E ψ continuous over Ψ .
Revised: This is the case if the energies are parametrized by a compact set Ψ with ψ (cid:55)→ E ψ continuous over Ψ .

Original: Estimating the likelihood is then achieved using i.i.d.
Revised: Estimating the likelihood is then achieved using i.i.d.

Original: samples ( X n ) 1: N , ( Y m ) 1: M from P and G (Tsuboi et al., 2009; Sugiyama et al., 2012; Liu et al., 2017):
Revised: samples ( X n ) 1: N , ( Y m ) 1: M from P and G (Tsuboi et al., 2009; Sugiyama et al., 2012; Liu et al., 2017):

Original: [Equation]
Revised: [Equation]

Original: In the context of mini-batch stochastic gradient methods, however, M typically ranges from 10 to 1000 , which can lead to a poor estimate for the log-partition function A G ,E .
Revised: In the context of mini-batch stochastic gradient methods, however, M typically ranges from 10 to 1000 , which can lead to a poor estimate for the log-partition function A G ,E .

Original: Moreover, (8) doesn’t exploit estimates of A G ,E from previous gradient iterations.
Revised: Moreover, (8) doesn’t exploit estimates of A G ,E from previous gradient iterations.

Original: Instead, we propose an estimator which introduces a variational parameter A ∈ R meant to estimate A G ,E in an amortized fashion.
Revised: Instead, we propose an estimator which introduces a variational parameter A ∈ R meant to estimate A G ,E in an amortized fashion.

Original: The key idea is to exploit the convexity of the exponential which directly implies − A G ,E ≥−
Revised: The key idea is to exploit the convexity of the exponential which directly implies − A G ,E ≥−

Original: A − exp( − A + A G ,E )+ for any A ∈ R , with equality only when
Revised: A − exp( − A + A G ,E )+ for any A ∈ R , with equality only when

Original: A =
Revised: A =

Original: A G ,E .
Revised: A G ,E .

Original: Therefore, (6) admits a lower-bound of the form
Revised: Therefore, (6) admits a lower-bound of the form

Original: [Equation]
Revised: [Equation]

Original: where we introduced the functional F P , G for concision.
Revised: where we introduced the functional F P , G for concision.

Original: Maximizing F P , G ( E + A ) over A recovers the likelihood L P , G ( E ) .
Revised: Maximizing F P , G ( E + A ) over A recovers the likelihood L P , G ( E ) .

Original: Moreover, jointly maximizing over E and A yields the maximum likelihood energy E (cid:63) and its corresponding log-partition function A (cid:63)
Revised: Moreover, jointly maximizing over E and A yields the maximum likelihood energy E (cid:63) and its corresponding log-partition function A (cid:63)

Original: =
Revised: =

Original: A G ,E (cid:63) .
Revised: A G ,E (cid:63) .

Original: This optimization is well-suited for stochastic gradient methods using the following estimator Kanamori et al.
Revised: This optimization is well-suited for stochastic gradient methods using the following estimator Kanamori et al.

Original: [Equation]
Revised: [Equation]

Original: 3. B ASE LEARNING
Revised: 3. B ASE LEARNING

Original: Unlike in Section 3.1, varying the base G does not need to preserve the same support.
Revised: Unlike in Section 3.1, varying the base G does not need to preserve the same support.

Original: Thus, it is generally not possible to use maximum likelihood methods for learning G .
Revised: Thus, it is generally not possible to use maximum likelihood methods for learning G .

Original: Instead, we propose to use the generalized likelihood (6) evaluated at the optimal energy E (cid:63) as a meaningful loss for learning G , and refer to it as the KL Approximate Lower-bound Estimate (
Revised: Instead, we propose to use the generalized likelihood (6) evaluated at the optimal energy E (cid:63) as a meaningful loss for learning G , and refer to it as the KL Approximate Lower-bound Estimate (

Original: [Equation]
Revised: [Equation]

Original: From Section 3.1, KALE ( P || G ) is always a lower bound on KL ( P , G ) .
Revised: From Section 3.1, KALE ( P || G ) is always a lower bound on KL ( P , G ) .

Original: The bound becomes tight whenever the negative log density of P w.r.t.
Revised: The bound becomes tight whenever the negative log density of P w.r.t.

Original: G is well-deﬁned and belongs to E (Appendix A).
Revised: G is well-deﬁned and belongs to E (Appendix A).

Original: Moreover, Proposition 4 shows that KALE is a reliable criterion for measuring convergence, and is a consequence of (Zhang et al., 2017, Theorem B.1), with a proof in Appendix C.2.1:
Revised: Moreover, Proposition 4 shows that KALE is a reliable criterion for measuring convergence, and is a consequence of (Zhang et al., 2017, Theorem B.1), with a proof in Appendix C.2.1:

Original: Proposition 4.
Revised: Proposition 4.

Original: Assume all energies in E are L -Lipschitz and that any continuous function can be well approximated by linear combinations of energies in E (Assumptions ( A ) and ( B ) of Appendix C.2), then KALE ( P || G ) ≥ 0 with equality only if P = G and KALE ( P || G n ) → 0 iff G n → P in distribution.
Revised: Assume all energies in E are L -Lipschitz and that any continuous function can be well approximated by linear combinations of energies in E (Assumptions ( A ) and ( B ) of Appendix C.2), then KALE ( P || G ) ≥ 0 with equality only if P = G and KALE ( P || G n ) → 0 iff G n → P in distribution.

Original: The universal approximation assumption holds in particular when E contains feedforward networks.
Revised: The universal approximation assumption holds in particular when E contains feedforward networks.

Original: In fact networks with a single neuron are enough, as shown in (Zhang et al., 2017, Theorem 2.3).
Revised: In fact networks with a single neuron are enough, as shown in (Zhang et al., 2017, Theorem 2.3).

Original: The
Revised: The

Original: Lipschitz assumption holds when additional regularization of the energy is enforced during training by methods such as spectral normalization (Miyato et al., 2018) or additional regularization I ( ψ ) on the energy E ψ such as the gradient penalty (Gulrajani et al., 2017) as done in Section 6.
Revised: Lipschitz assumption holds when additional regularization of the energy is enforced during training by methods such as spectral normalization (Miyato et al., 2018) or additional regularization I ( ψ ) on the energy E ψ such as the gradient penalty (Gulrajani et al., 2017) as done in Section 6.

Original: Estimating KALE.
Revised: Estimating KALE.

Original: According to Arora et al.
Revised: According to Arora et al.

Original: (2017), accurate ﬁnite sample estimates of divergences that result from an optimization procedures (such as in (10)) depend on the richness of the class E ; and richer energy classes can result in slower convergence.
Revised: (2017), accurate ﬁnite sample estimates of divergences that result from an optimization procedures (such as in (10)) depend on the richness of the class E ; and richer energy classes can result in slower convergence.

Original: Unlike divergences such as Jensen-Shannon,
Revised: Unlike divergences such as Jensen-Shannon,

Original: KL and the Wasserstein distance, which result from optimizing over a non-parametric and rich class of functions, KALE is restricted to a class of parametric energies E ψ .
Revised: KL and the Wasserstein distance, which result from optimizing over a non-parametric and rich class of functions, KALE is restricted to a class of parametric energies E ψ .

Original: Thus, (Arora et al., 2017, Theorem 3.1) applies, and guarantees good ﬁnite sample estimates, provided optimization is solved accurately.
Revised: Thus, (Arora et al., 2017, Theorem 3.1) applies, and guarantees good ﬁnite sample estimates, provided optimization is solved accurately.

Original: In Appendix B, we provide an analysis for the more general case where energies are not necessarily parametric but satisfy some further smoothness properties; we emphasize that our rates do not require the strong assumption that the density ratio is bounded above and below as in (Nguyen et al., 2010).
Revised: In Appendix B, we provide an analysis for the more general case where energies are not necessarily parametric but satisfy some further smoothness properties; we emphasize that our rates do not require the strong assumption that the density ratio is bounded above and below as in (Nguyen et al., 2010).

Original: Smoothness of KALE.
Revised: Smoothness of KALE.

Original: Learning the base is achieved by minimizing K ( θ ):=
Revised: Learning the base is achieved by minimizing K ( θ ):=

Original: KALE ( P || G θ ) over the set of parameters Θ of the generator G θ .
Revised: KALE ( P || G θ ) over the set of parameters Θ of the generator G θ .

Original: This requires K ( θ ) to be smooth enough so that gradient methods converge to local minima and avoid instabilities during training (Chu et al., 2020).
Revised: This requires K ( θ ) to be smooth enough so that gradient methods converge to local minima and avoid instabilities during training (Chu et al., 2020).

Original: Ensuring smoothness of losses that result from an optimization procedure, as in (10), can be challenging.
Revised: Ensuring smoothness of losses that result from an optimization procedure, as in (10), can be challenging.

Original: Results for the regularized Wasserstein are provided by Sanjabi et al.
Revised: Results for the regularized Wasserstein are provided by Sanjabi et al.

Original: (2018), while more general losses are considered by Chu et al.
Revised: (2018), while more general losses are considered by Chu et al.

Original: (2020), albeit under stronger conditions than for our setting.
Revised: (2020), albeit under stronger conditions than for our setting.

Original: Theorem 5 shows that when E , G θ and their gradients are all Lipschitz then K ( θ ) is smooth enough.
Revised: Theorem 5 shows that when E , G θ and their gradients are all Lipschitz then K ( θ ) is smooth enough.

Original: We provide a proof for Theorem 5 in Appendix C.2.1.
Revised: We provide a proof for Theorem 5 in Appendix C.2.1.

Original: Theorem 5.
Revised: Theorem 5.

Original: Under Assumptions ( I ) to ( III ) of Appendix C.2, sub-gradient methods on K converge to local optima.
Revised: Under Assumptions ( I ) to ( III ) of Appendix C.2, sub-gradient methods on K converge to local optima.

Original: Moreover, K is Lipschitz and differentiable for almost all θ ∈ Θ with:
Revised: Moreover, K is Lipschitz and differentiable for almost all θ ∈ Θ with:

Original: [Equation]
Revised: [Equation]

Original: Estimating the gradient in (11) is achieved by ﬁrst optimizing over E ψ and A using(9), withadditionalregularization I ( ψ ) .
Revised: Estimating the gradient in (11) is achieved by ﬁrst optimizing over E ψ and A using(9), withadditionalregularization I ( ψ ) .

Original: The resulting estimators ˆ E (cid:63) and ˆ A (cid:63) are plugged in (12) to estimate ∇K ( θ ) using samples ( Z m ) 1: M from η .
Revised: The resulting estimators ˆ E (cid:63) and ˆ A (cid:63) are plugged in (12) to estimate ∇K ( θ ) using samples ( Z m ) 1: M from η .

Original: Unlike for learning the energy E (cid:63) , which beneﬁts from using the amortized estimator of the log-partition function, we found that using the empirical log-partition for learning the base was more stable.
Revised: Unlike for learning the energy E (cid:63) , which beneﬁts from using the amortized estimator of the log-partition function, we found that using the empirical log-partition for learning the base was more stable.

Original: We summarize the training procedure in Algorithm 1, which alternates between learning the energy and the base in a similar fashion to adversarial training .
Revised: We summarize the training procedure in Algorithm 1, which alternates between learning the energy and the base in a similar fashion to adversarial training .

Original: Update ψ and A using g ψ and g A .
Revised: Update ψ and A using g ψ and g A .

Original: 11: end for 12:
Revised: 11: end for 12:

Original: Set ˆ E (cid:63) ← E ψ and ˆ A (cid:63) ← A .
Revised: Set ˆ E (cid:63) ← E ψ and ˆ A (cid:63) ← A .

Original: Update θ using ÷ ∇K ( θ ) from (12) 14: end for
Revised: Update θ using ÷ ∇K ( θ ) from (12) 14: end for

Original: [Equation]
Revised: [Equation]

Original:  S AMPLING FROM GEBM S
Revised:  S AMPLING FROM GEBM S

Original: A simple estimate of the empirical distribution of observations under the GEBM is via importance sampling (IS).
Revised: A simple estimate of the empirical distribution of observations under the GEBM is via importance sampling (IS).

Original: This consists in ﬁrst sampling multiple points from the base G , and then re-weighting the samples according to the energy E .
Revised: This consists in ﬁrst sampling multiple points from the base G , and then re-weighting the samples according to the energy E .

Original: Although straightforward, this approach can lead to highly unreliable estimates, a well known problem in the Sequential Monte Carlo (SMC) literature which employs IS extensively(Doucetetal.,2001;DelMoraletal.
Revised: Although straightforward, this approach can lead to highly unreliable estimates, a well known problem in the Sequential Monte Carlo (SMC) literature which employs IS extensively (Doucet et al., 2001; Del Moral et al., 2006).

Revised: Other methods such as rejection sampling are known tobeinefﬁcientinhighdimensionsHaugh(2017).

Original: Instead, weproposetosamplefromtheposterior ν using MCMC.
Revised: Instead, weproposetosamplefromtheposterior ν using MCMC.

Original: Recall from (5) that a sample x from Q is of the form x = G ( z ) with z sampled from the posteriorlatent ν of(4)insteadoftheprior η .
Revised: Recall from (5) that a sample x from Q is of the form x = G ( z ) with z sampled from the posteriorlatent ν of(4)insteadoftheprior η .

Original: Whilesamplingfrom η isoftenstraightforward(forinstanceif η is a Gaussian), sampling from ν is generally harder, due to dependence of its density on complex functions E and G .
Revised: Whilesamplingfrom η isoftenstraightforward(forinstanceif η is a Gaussian), sampling from ν is generally harder, due to dependence of its density on complex functions E and G .

Original: ItisstillpossibletouseMCMCmethodstosamplefrom
Revised: ItisstillpossibletouseMCMCmethodstosamplefrom

Original: ν , however, sincewehaveaccess to its density up to a normalizing constant (4).
Revised: ν , however, sincewehaveaccess to its density up to a normalizing constant (4).

Original: In particular, we are interested in methods that exploit the gradient of ν , and consider two classes of samplers:
Revised: In particular, we are interested in methods that exploit the gradient of ν , and consider two classes of samplers:

Original: Overdamped samplers and Kinetic samplers .
Revised: Overdamped samplers and Kinetic samplers .

Original: Overdamped samplers are obtained as a time-discretization of the Overdamped Langevin dynamics :
Revised: Overdamped samplers are obtained as a time-discretization of the Overdamped Langevin dynamics :

Original: [Equation]
Revised: [Equation]

Original: where w t is a standard Brownian motion.
Revised: where w t is a standard Brownian motion.

Original: The simplest sampler arising from (13) is the Unadjusted
Revised: The simplest sampler arising from (13) is the Unadjusted

Original: Langevin Algorithm (ULA):
Revised: Langevin Algorithm (ULA):

Original: [Equation]
Revised: [Equation]

Original: where ( W k ) k ≥ 0 are i.i.d.
Revised: where ( W k ) k ≥ 0 are i.i.d.

Original: standard Gaussians and λ is the step-size.
Revised: standard Gaussians and λ is the step-size.

Original: For large k , Z k is an approximate sample from ν (Raginsky et al., 2017, Proposition 3.3).
Revised: For large k , Z k is an approximate sample from ν (Raginsky et al., 2017, Proposition 3.3).

Original: Hence, setting X =
Revised: Hence, setting X =

Original: G ( Z k ) for a large enough k provides an approximate sample from the GEBM Q , as summarized in Algorithm 2 of Appendix F.
Revised: G ( Z k ) for a large enough k provides an approximate sample from the GEBM Q , as summarized in Algorithm 2 of Appendix F.

Original: Kinetic samplers arise from the Kinetic Langevin dynamics which introduce a momentum variable:
Revised: Kinetic samplers arise from the Kinetic Langevin dynamics which introduce a momentum variable:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with friction coefﬁcient γ ≥ 0 , inverse mass u ≥ 0 , momentum vector v t and standard Brownian motion w t .
Revised: with friction coefﬁcient γ ≥ 0 , inverse mass u ≥ 0 , momentum vector v t and standard Brownian motion w t .

Original: When the mass u − 1 becomes negligible compared to the friction coefﬁcient γ , i.e. uγ − 2 ≈ 0 , standard results show that (14) recovers the Overdamped dynamics (13).
Revised: When the mass u − 1 becomes negligible compared to the friction coefﬁcient γ , i.e. uγ − 2 ≈ [Equation] standard results show that (14) recovers the Overdamped dynamics (13).

Original: Discretization in time of (14) leads to Kinetic samplers similar to Hamiltonian Monte Carlo (Cheng et al., 2017; Sachs et al., 2017).
Revised: Discretization in time of (14) leads to Kinetic samplers similar to Hamiltonian Monte Carlo (Cheng et al., 2017; Sachs et al., 2017).

Original: We consider a particular algorithm from Sachs et al.
Revised: We consider a particular algorithm from Sachs et al.

Original: (2017) which we call Kinetic Langevin Algorithm (KLA) (see Algorithm 3 in Appendix F).
Revised: (2017) which we call Kinetic Langevin Algorithm (KLA) (see Algorithm 3 in Appendix F).

Original: Kinetic samplers were shown to better explore the modes of the invariant distribution ν compared to Overdamped ones (see (Neal, 2010;
Revised: Kinetic samplers were shown to better explore the modes of the invariant distribution ν compared to Overdamped ones (see (Neal, 2010;

Original: Betancourt et al., 2017) for empirical results and (Cheng et al., 2017) for theory), as also conﬁrmed empirically in Appendix D for image generation tasks using GEBMs.
Revised: Betancourt et al., 2017) for empirical results and (Cheng et al., 2017) for theory), as also conﬁrmed empirically in Appendix D for image generation tasks using GEBMs.

Original: Next, we provide the following convergence result:
Revised: Next, we provide the following convergence result:

Original: Proposition 6.
Revised: Proposition 6.

Original: Assume that log η ( z ) is strongly concave and has a Lipschitz gradient, that E , G and their gradients are all L -Lipschitz.
Revised: Assume that log η ( z ) is strongly concave and has a Lipschitz gradient, that E , G and their gradients are all L -Lipschitz.

Original: Set x t = G ( z t ) , where z t is given by (14) and call P t the probability distribution of x t .
Revised: Set x t = G ( z t ) , where z t is given by (14) and call P t the probability distribution of x t .

Original: Then P t converges to Q in the Wasserstein sense,
Revised: Then P t converges to Q in the Wasserstein sense,

Original: [Equation]
Revised: [Equation]

Original: where c and C are positive constants independent of t , with c
Revised: where c and C are positive constants independent of t , with c

Original: = O (exp( − dim ( Z ))) .
Revised: = O (exp( − dim ( Z ))) .

Original: Proposition 6 is proved in Appendix C.1 using (Eberle et al., 2017, Corollary 2.6), and implies that ( x t ) t ≥ 0 converges at the same speed as ( z t ) t ≥ 0 .
Revised: Proposition 6 is proved in Appendix C.1 using (Eberle et al., 2017, Corollary 2.6), and implies that ( x t ) t ≥ 0 converges at the same speed as ( z t ) t ≥ 0 .

Original: When the dimension q of Z is orders of magnitude smaller than the input space dimension d , the process ( x t ) t ≥ 0 converges faster than typical sampling methods on X , for which the exponent controlling the convergence rate is of order O
Revised: When the dimension q of Z is orders of magnitude smaller than the input space dimension d , the process ( x t ) t ≥ 0 converges faster than typical sampling methods on X , for which the exponent controlling the convergence rate is of order O

Original: (exp( − d )) .
Revised: (exp( − d )) .

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Energy based models.
Revised: Energy based models.

Original: Usually, energy based models are required to have a density w.r.t.
Revised: Usually, energy based models are required to have a density w.r.t.

Original: to a
Revised: to a

Original: Lebesgue measure, and do not use a learnable base measure; in other words, models are supported on the whole space.
Revised: Lebesgue measure, and do not use a learnable base measure; in other words, models are supported on the whole space.

Original: Various methods have been proposed in the literature to learn EBMs.
Revised: Various methods have been proposed in the literature to learn EBMs.

Original: Contrastive
Revised: Contrastive

Original: Divergence (Hinton, 2002) approximates the gradient of the log-likelihood by sampling from the energy model with MCMC.
Revised: Divergence (Hinton, 2002) approximates the gradient of the log-likelihood by sampling from the energy model with MCMC.

Original: More recently, (Belanger and McCallum, 2016; Xie et al., 2016; 2017; 2018c; 2019; Tu and Gimpel, 2018; Du and Mordatch, 2019; Deng et al., 2020) extend the idea using more sophisticated models and MCMC sampling strategies that lead to higher quality estimators.
Revised: More recently, (Belanger and McCallum, 2016; Xie et al., 2016; 2017; 2018c; 2019; Tu and Gimpel, 2018; Du and Mordatch, 2019; Deng et al., 2020) extend the idea using more sophisticated models and MCMC sampling strategies that lead to higher quality estimators.

Original: Score Matching (Hyvärinen, 2005) calculates an alternative objective (the score ) to the log-likelihood which is independent of the partition function, and was recently used in the context non-parametric energy functions to provide estimators of the energy that are provably consistent (Sriperumbudur et al., 2017;
Revised: Score Matching (Hyvärinen, 2005) calculates an alternative objective (the score ) to the log-likelihood which is independent of the partition function, and was recently used in the context non-parametric energy functions to provide estimators of the energy that are provably consistent (Sriperumbudur et al., 2017;

Original: Sutherland et al., 2018; Arbel and Gretton, 2018; Wenliang et al., 2019).
Revised: Sutherland et al., 2018; Arbel and Gretton, 2018; Wenliang et al., 2019).

Original: In Noise-Contrastive
Revised: In Noise-Contrastive

Original: Estimation (Gutmann and Hyvärinen, 2012), a classiﬁer is trained to distinguish between samples from a ﬁxed proposal distribution and the target P .
Revised: Estimation (Gutmann and Hyvärinen, 2012), a classiﬁer is trained to distinguish between samples from a ﬁxed proposal distribution and the target P .

Original: This provides an estimate for the density ratio between the optimal energy model and the proposal distribution.
Revised: This provides an estimate for the density ratio between the optimal energy model and the proposal distribution.

Original: In a similar spirit, Cranmer et al.
Revised: In a similar spirit, Cranmer et al.

Original: (2016) uses a classiﬁer to learn likelihood ratios.
Revised: (2016) uses a classiﬁer to learn likelihood ratios.

Original: Conversely, Grathwohl et al.
Revised: Conversely, Grathwohl et al.

Original: (2020) interprets the logits of a classiﬁer as an energy model obtained after marginalization over the classes.
Revised: (2020) interprets the logits of a classiﬁer as an energy model obtained after marginalization over the classes.

Original: The resulting model is then trained using Contrastive Divergence.
Revised: The resulting model is then trained using Contrastive Divergence.

Original: In more recent work, Dai et al.
Revised: In more recent work, Dai et al.

Original: (2019a;b) exploit a dual formulation of the logarithm of the partition function as a supremum over the set of all probability distributions of some functional objective.
Revised: (2019a;b) exploit a dual formulation of the logarithm of the partition function as a supremum over the set of all probability distributions of some functional objective.

Original: Yu et al.
Revised: Yu et al.

Original: (2020) explore methods for using general f-divergences, such as Jensen-Shannon, to train EBMs.
Revised: (2020) explore methods for using general f-divergences, such as Jensen-Shannon, to train EBMs.

Original: Generative Adversarial Networks.
Revised: Generative Adversarial Networks.

Original: Recent work proposes using the discriminator of a trained
Revised: Recent work proposes using the discriminator of a trained

Original: GAN to improve the generator quality.
Revised: GAN to improve the generator quality.

Original: Rejection sampling (Azadi et al., 2019) and MetropolisHastings correction (Turner et al., 2019; Neklyudov et al., 2019) perform sampling directly on the high-dimensional input space without using gradient information provided by the discriminator.
Revised: Rejection sampling (Azadi et al., 2019) and MetropolisHastings correction (Turner et al., 2019; Neklyudov et al., 2019) perform sampling directly on the high-dimensional input space without using gradient information provided by the discriminator.

Original: Moreover, the data distribution is assumed to admit a density w.r.t.
Revised: Moreover, the data distribution is assumed to admit a density w.r.t.

Original: the generator.
Revised: the generator.

Original: Ding et al.
Revised: Ding et al.

Original: (2019) perform sampling on the feature space of some auxiliary pre-trained network; while Lawson et al.
Revised: (2019) perform sampling on the feature space of some auxiliary pre-trained network; while Lawson et al.

Original: (2019) treat the sampling procedure as a model on its own, learned by maximizing the ELBO.
Revised: (2019) treat the sampling procedure as a model on its own, learned by maximizing the ELBO.

Original: In our case, no auxiliary model is needed.
Revised: In our case, no auxiliary model is needed.

Original: In the present work, sampling doesn’t interfere with training, in contrast to recently considered methods to optimize over the latent space during training Wu et al.
Revised: In the present work, sampling doesn’t interfere with training, in contrast to recently considered methods to optimize over the latent space during training Wu et al.

Original: (2019b;a).
Revised: (2019b;a).

Original: In Tanaka (2019), the discriminator is viewed as an optimal transport map between the generator and the data distribution and is used to compute optimized samples from latent space.
Revised: In Tanaka (2019), the discriminator is viewed as an optimal transport map between the generator and the data distribution and is used to compute optimized samples from latent space.

Original: This is in contrast to the diffusion-based sampling that we consider.
Revised: This is in contrast to the diffusion-based sampling that we consider.

Original: In (Xie et al., 2018b;a), two independent models, a full support EBM and a generator network, are trained cooperatively using MCMC.
Revised: In (Xie et al., 2018b;a), two independent models, a full support EBM and a generator network, are trained cooperatively using MCMC.

Original: By contrast, in the present work, the energy and base are part of the same model, and the model support is lower-dimensional than the target space X .
Revised: By contrast, in the present work, the energy and base are part of the same model, and the model support is lower-dimensional than the target space X .

Revised: While we do not address the mode collapse problem,

Original:  The closest related approach appears in a study concurrent to the present work (Che et al., 2020), where the authors propose to use Langevin dynamics on the latent space of a GAN generator, but with a different discriminator to ours (derived from the Jensen-Shannon divergence or a Wasserstein-based divergence).
Revised: Xu et al. ; Nguyen et al. (2017) showed that KL-based losses are resilient to it thanks to the zero-avoiding property of the KL, a good sign for KALE which is derived from KL by Fenchel duality. The closest related approach appears in a study concurrent to the present work (Che et al., 2020), where the authors propose to use Langevin dynamics on the latent space of a GAN generator, but with a different discriminator to ours (derived from the Jensen-Shannon divergence or a Wasserstein-based divergence).

Original: Our theory results showing the existence of the loss gradient (Theorem 5), establishing weak convergence of distributions under KALE (Proposition 4), and demonstrating consistency of the KALE estimator (Appendix B) should transfer to the JS and Wasserstein criteria used in that work.
Revised: Our theory results showing the existence of the loss gradient (Theorem 5), establishing weak convergence of distributions under KALE (Proposition 4), and demonstrating consistency of the KALE estimator (Appendix B) should transfer to the JS and Wasserstein criteria used in that work.

Original: Subsequent to the present work, an alternative approach has been recently proposed, based on normalising ﬂows, to learn both the low-dimensional support of the data and the density on this support (Brehmer and Cranmer, 2020).
Revised: Subsequent to the present work, an alternative approach has been recently proposed, based on normalising ﬂows, to learn both the low-dimensional support of the data and the density on this support (Brehmer and Cranmer, 2020).

Original: This approach maximises the explicit likelihood of a data projection onto a learned manifold, and may be considered complementary to our approach.
Revised: This approach maximises the explicit likelihood of a data projection onto a learned manifold, and may be considered complementary to our approach.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: 6. I MAGE GENERATION .
Revised: [Figure]

Original: Experimental setting.
Revised: Experimental setting.

Original: We train a GEBM on unsupervised image generation tasks, and compare the quality of generated samples with other methods using the FID score (Heusel et al., 2017) computed on 5 × 10 4 generated samples.
Revised: We train a GEBM on unsupervised image generation tasks, and compare the quality of generated samples with other methods using the FID score (Heusel et al., 2017) computed on 5 × 10 4 generated samples.

Original: We consider CIFAR-10 (Krizhevsky, 2009), LSUN (Yu et al., 2015), CelebA (Liu et al., 2015) and ImageNet (Russakovsky et al., 2014) all downsampled to 32x32 resolution to reduce computational cost.
Revised: We consider CIFAR-10 (Krizhevsky, 2009), LSUN (Yu et al., 2015), CelebA (Liu et al., 2015) and ImageNet (Russakovsky et al., 2014) all downsampled to 32x32 resolution to reduce computational cost.

Original: We consider two network architectures for each of the base and energy, a smaller one (SNGAN ConvNet) and a larger one (SNGAN ResNet), both of which are from Miyato et al.
Revised: We consider two network architectures for each of the base and energy, a smaller one (SNGAN ConvNet) and a larger one (SNGAN ResNet), both of which are from Miyato et al.

Original: For the base we used the SNGAN generator networks from
Revised: For the base we used the SNGAN generator networks from

Original: Miyato et al.
Revised: Miyato et al.

Original: (2018) with a 100 -dimensional Gaussian for the latent noise η .
Revised: (2018) with a 100 -dimensional Gaussian for the latent noise η .

Original: For the energy we used the SNGAN discriminator networks from Miyato et al.
Revised: For the energy we used the SNGAN discriminator networks from Miyato et al.

Original: (Details of the networks in Appendix G.1).
Revised: (Details of the networks in Appendix G.1).

Original: We train the models for 150000 generator iterations using Algorithm 1.
Revised: We train the models for 150000 generator iterations using Algorithm 1.

Original: After training is completed, we rescale the energy by β =100 to get a colder version of the GEBM and sample from it using either
Revised: After training is completed, we rescale the energy by β =100 to get a colder version of the GEBM and sample from it using either

Original: Algorithm 2 (ULA) or Algorithm 3 (KLA) with parameters ( γ =100 ,u =1) .
Revised: Algorithm 2 (ULA) or Algorithm 3 (KLA) with parameters ( γ =100 ,u =1) .

Original: This colder temperature leads to an improved FID score, and needs relatively few MCMC iterations, as shown in Figure 6 of
Revised: This colder temperature leads to an improved FID score, and needs relatively few MCMC iterations, as shown in Figure 6 of

Original: Appendix D. Sampler convergence to visually plausible modes at low tempteratures is demonstrated in
Revised: Appendix D. Sampler convergence to visually plausible modes at low tempteratures is demonstrated in

Original: Figure 2.
Revised: Figure 2.

Original: We perform 1000 MCMC iterations with initial step-size of λ =10 − 4 decreased by 10 every 200 iterations.
Revised: We perform 1000 MCMC iterations with initial step-size of λ =10 − 4 decreased by 10 every 200 iterations.

Original: As a baseline we consider samples generated from the base of the GEBM only (without using information from the energy) and call this KALE-GAN.
Revised: As a baseline we consider samples generated from the base of the GEBM only (without using information from the energy) and call this KALE-GAN.

Original: More details are given in Appendix G.
Revised: More details are given in Appendix G.

Original: Results: Table 1 shows that GEBM outperforms both KALE and standard GANs when using the same networks for the base/generator and energy/critic.
Revised: Results: Table 1 shows that GEBM outperforms both KALE and standard GANs when using the same networks for the base/generator and energy/critic.

Original: Moreover, KALE-GAN matches the performance of a standard GAN (with Jensen-Shannon critic), showing that the improvement of GEBM cannot be explained by the switch from Jensen-Shannon to a KALE-based critic.
Revised: Moreover, KALE-GAN matches the performance of a standard GAN (with Jensen-Shannon critic), showing that the improvement of GEBM cannot be explained by the switch from Jensen-Shannon to a KALE-based critic.

Original: Rather, the improvement is largely due to incorporating the energy function into the model, and sampling using Algorithm 3.
Revised: Rather, the improvement is largely due to incorporating the energy function into the model, and sampling using Algorithm 3.

Original: This ﬁnding experimentally validates our claim that incorporating the energy improves the model, and that all else being equal, a GEBM outperforms a GAN with the same generator and critic architecture.
Revised: This ﬁnding experimentally validates our claim that incorporating the energy improves the model, and that all else being equal, a GEBM outperforms a GAN with the same generator and critic architecture.

Original: Indeed, if the critic is not zero at convergence, then by deﬁnition it contains information on the remaining mismatch between the generator (base) and data mass, which the GEBM incorporates, but the GAN does not.
Revised: Indeed, if the critic is not zero at convergence, then by deﬁnition it contains information on the remaining mismatch between the generator (base) and data mass, which the GEBM incorporates, but the GAN does not.

Original: The GEBM also outperforms an EBM even when the latter was trained using a larger network (ResNet) with supervision (S) on ImageNet, which is an easier task ( Chen et al.
Revised: The GEBM also outperforms an EBM even when the latter was trained using a larger network (ResNet) with supervision (S) on ImageNet, which is an easier task ( Chen et al.

Original: More comparisons on Cifar10 and ImageNet are provided in Table 4 of Appendix D.
Revised: More comparisons on Cifar10 and ImageNet are provided in Table 4 of Appendix D.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: Table 2 shows different sampling methods using the same trained networks (generator and critic), with KALE-GAN as a baseline.
Revised: Table 2 shows different sampling methods using the same trained networks (generator and critic), with KALE-GAN as a baseline.

Original: All energy-exploiting methods outperform the unmodiﬁed
Revised: All energy-exploiting methods outperform the unmodiﬁed

Original: KALE-GAN with the same architecture.
Revised: KALE-GAN with the same architecture.

Original: That said, our method (both ULA and
Revised: That said, our method (both ULA and

Original: KLA) outperforms both (IHM) (Turner et al., 2019) and (DOT)
Revised: KLA) outperforms both (IHM) (Turner et al., 2019) and (DOT)

Original: (Tanaka, 2019), which both use the energy information.
Revised: (Tanaka, 2019), which both use the energy information.

Original: [Table]
Revised: [Table]

Original: In Table 2, KLA was used in the high friction regime γ =100 and thus behaves like ULA.
Revised: In Table 2, KLA was used in the high friction regime γ =100 and thus behaves like ULA.

Original: This allows to obtain sharper samples concentrated around the modes of the GEBM thus improving the FID score.
Revised: This allows to obtain sharper samples concentrated around the modes of the GEBM thus improving the FID score.

Original: If, instead, the goal is to encourage more exploration of the modes of the GEBM, then KLA with a smaller γ is a better alternative than ULA, as the former can explore multiple modes/images within the same
Revised: If, instead, the goal is to encourage more exploration of the modes of the GEBM, then KLA with a smaller γ is a better alternative than ULA, as the former can explore multiple modes/images within the same

Original: MCMC chain, unlike (ULA): see Figures 3 to 5 of Appendix D. Moving from one mode to another results in an increased FID score while between modes, however, which can be avoided by decreasing λ .
Revised: MCMC chain, unlike (ULA): see Figures 3 to 5 of Appendix D. Moving from one mode to another results in an increased FID score while between modes, however, which can be avoided by decreasing λ .

Original: 6. D ENSITY E STIMATION
Revised: 6. D ENSITY E STIMATION

Original: Motivation.
Revised: Motivation.

Original: We next consider the particular setting where the likelihood of the model is well-deﬁned, and admits a closed form expression.
Revised: We next consider the particular setting where the likelihood of the model is well-deﬁned, and admits a closed form expression.

Original: This is intended principally as a sanity check that our proposed training method in Algorithm 1 succeeds in learning maximum likelihood solutions.
Revised: This is intended principally as a sanity check that our proposed training method in Algorithm 1 succeeds in learning maximum likelihood solutions.

Original: Outside of this setting, closed form expressions of the normalizing constant are not available for generic GEBMs.
Revised: Outside of this setting, closed form expressions of the normalizing constant are not available for generic GEBMs.

Original: While this is not an issue (since the proposed method doesn’t require a closed form expression for the normalizing constant), in this experiment only, we want to have access to closed form expressions, as they enable a direct comparison with other density estimation methods.
Revised: While this is not an issue (since the proposed method doesn’t require a closed form expression for the normalizing constant), in this experiment only, we want to have access to closed form expressions, as they enable a direct comparison with other density estimation methods.

Original: Experimental setting.
Revised: Experimental setting.

Original: To have a closed-form likelihood, we consider the case where the dimension of the latent space is equal to data-dimension, and choose the base G of the GEBM to be a Real NVP (Ding et al.
Revised: To have a closed-form likelihood, we consider the case where the dimension of the latent space is equal to data-dimension, and choose the base G of the GEBM to be a Real NVP (Ding et al.

Original: with density exp( − r ( x ))
Revised: with density exp( − r ( x ))

Original: and energy E ( x )= h ( x ) − r ( x ) .
Revised: and energy E ( x )= h ( x ) − r ( x ) .

Original: Thus, in this particular case, the GEBM has a well deﬁned likelihood over the whole space, and we are precisely in the setting of
Revised: Thus, in this particular case, the GEBM has a well deﬁned likelihood over the whole space, and we are precisely in the setting of

Original: Proposition 2, which shows that this GEBM is equal to an EBM with density proportional to exp ( − h ) .
Revised: Proposition 2, which shows that this GEBM is equal to an EBM with density proportional to exp ( − h ) .

Original: We further require the EBM to be a second Real NVP so that its density has a closed form expression.
Revised: We further require the EBM to be a second Real NVP so that its density has a closed form expression.

Original: We consider 5 UCI datasets (Dheeru and Taniskidou, 2017) for which we use the same pre-processing as in (Wenliang et al., 2019).
Revised: We consider 5 UCI datasets (Dheeru and Taniskidou, 2017) for which we use the same pre-processing as in (Wenliang et al., 2019).

Original: For comparison, we train the EBM by direct maximum likelihood (ML) and contrastive divergence (CD).
Revised: For comparison, we train the EBM by direct maximum likelihood (ML) and contrastive divergence (CD).

Original: To train the GEBM, we use Algorithm 1, which doesn’t directly exploit the closed-form expression of the likelihood (unlike direct ML).
Revised: To train the GEBM, we use Algorithm 1, which doesn’t directly exploit the closed-form expression of the likelihood (unlike direct ML).

Original: We thus use either (8) (KALE-DV) or (9) (KALE-F) to estimate the normalizing constant.
Revised: We thus use either (8) (KALE-DV) or (9) (KALE-F) to estimate the normalizing constant.

Original: More details are given in Appendix G.2.
Revised: More details are given in Appendix G.2.

Original: Results.
Revised: Results.

Original: Table 3 reports the Negative Log-Likelihood (NLL) evaluated on the test set and corresponding tothebestperformanceonthevalidationset.
Revised: Table 3 reports the Negative Log-Likelihood (NLL) evaluated on the test set and corresponding tothebestperformanceonthevalidationset.

Original: TrainingtheGEBMusingAlgorithm1leadstocomparable performance to (CD) and (ML).
Revised: TrainingtheGEBMusingAlgorithm1leadstocomparable performance to (CD) and (ML).

Original: As shown in Figure 7 of Appendix E, (KALE-DV) and (KALEF) maintain a small error gap between the training and test NLL and, as discussed in Section 3. and Appendix F, (KALE-F) leads to more accurate estimates of the log-partition function, with a relative error of order 0 .
Revised: As shown in Figure 7 of Appendix E, (KALE-DV) and (KALEF) maintain a small error gap between the training and test NLL and, as discussed in Section 3. and Appendix F, (KALE-F) leads to more accurate estimates of the log-partition function, with a relative error of order 0 .

Original: 1% compared to 10%
Revised: 1% compared to 10%

Original: [Table]
Revised: [Table]

Original: A KL A PPROXIMATE L OWER - BOUND E STIMATE
Revised: A KL A PPROXIMATE L OWER - BOUND E STIMATE

Original: We discuss the relation between KALE (10) and the Kullback-Leibler divergence via Fenchel duality.
Revised: We discuss the relation between KALE (10) and the Kullback-Leibler divergence via Fenchel duality.

Original: Recall that a distribution P is said to admit a density w.r.t.
Revised: Recall that a distribution P is said to admit a density w.r.t.

Original: G if there exists a real-valued measurable function r 0 that is integrable w.r.t.
Revised: G if there exists a real-valued measurable function r 0 that is integrable w.r.t.

Original: G and satisﬁes d P = r 0 d G .
Revised: G and satisﬁes d P = r 0 d G .

Original: Such a density is also called the
Revised: Such a density is also called the

Original: Radon-Nikodym derivative of P w.r.t.
Revised: Radon-Nikodym derivative of P w.r.t.

Original: G .
Revised: G .

Original: In this case, we have:
Revised: In this case, we have:

Original: [Equation]
Revised: [Equation]

Original: Nguyen et al.
Revised: Nguyen et al.

Original: ; Nowozin et al.
Revised: ; Nowozin et al.

Original: (2016) derived a variational formulation for the KL using Fenchel duality.
Revised: (2016) derived a variational formulation for the KL using Fenchel duality.

Original: By the duality theorem (Rockafellar, 1970), the convex and lower semi-continuous function ζ : u (cid:55)→ u log( u ) that appears in (15) can be expressed as the supremum of a concave function:
Revised: By the duality theorem (Rockafellar, 1970), the convex and lower semi-continuous function ζ : u (cid:55)→ u log( u ) that appears in (15) can be expressed as the supremum of a concave function:

Original: [Equation]
Revised: [Equation]

Original: The function ζ (cid:63) is called the Fenchel dual and is deﬁned as ζ (cid:63) ( v )=
Revised: The function ζ (cid:63) is called the Fenchel dual and is deﬁned as ζ (cid:63) ( v )=

Original: sup u uv − ζ ( u ) .
Revised: sup u uv − ζ ( u ) .

Original: By convention, the value of the objective is set to −∞ whenever u is outside of the domain of deﬁnition of ζ (cid:63) .
Revised: By convention, the value of the objective is set to −∞ whenever u is outside of the domain of deﬁnition of ζ (cid:63) .

Original: When ζ ( u ) =
Revised: When ζ ( u ) =

Original: u log( u ) , the Fenchel dual ζ (cid:63) ( v ) admits a closed form expression of the form ζ (cid:63) ( v )=
Revised: u log( u ) , the Fenchel dual ζ (cid:63) ( v ) admits a closed form expression of the form ζ (cid:63) ( v )=

Original: exp( v − 1) .
Revised: exp( v − 1) .

Original: Using the expression of ζ in terms of its Fenchel dual ζ (cid:63) , it is possible to express
Revised: Using the expression of ζ in terms of its Fenchel dual ζ (cid:63) , it is possible to express

Original: KL ( P || G ) as the supremum of the variational objective (16) over all measurable functions h .
Revised: KL ( P || G ) as the supremum of the variational objective (16) over all measurable functions h .

Original: [Equation]
Revised: [Equation]

Original: Nguyen et al.
Revised: Nguyen et al.

Original: (2010) provided the variational formulation for the reverse KL using a different choice for ζ : ( ζ ( u ) = − log( u ) ).
Revised: (2010) provided the variational formulation for the reverse KL using a different choice for ζ : ( ζ ( u ) = − log( u ) ).

Original: We refer to (Nowozin et al., 2016) for general f -divergences.
Revised: We refer to (Nowozin et al., 2016) for general f -divergences.

Original: Choosing a smaller set of functions H in the variational objective (16) will lead to a lower bound on the KL.
Revised: Choosing a smaller set of functions H in the variational objective (16) will lead to a lower bound on the KL.

Original: This is the KL Approximate Lower-bound Estimate (KALE):
Revised: This is the KL Approximate Lower-bound Estimate (KALE):

Original: [Equation]
Revised: [Equation]

Original: In general, KL ( P || G ) ≥ KALE ( P || G ) .
Revised: In general, KL ( P || G ) ≥ KALE ( P || G ) .

Original: The bound is tight whenever the negative log-density h 

Revised: The bound is tight whenever the negative log-density h

Original: = − log r 0 belongs to H ; however, we do not require r 0 to be well-deﬁned in general.
Revised: = − log r 0 belongs to H ; however, we do not require r 0 to be well-deﬁned in general.

Original: Equation (17) has the advantage that it can be estimated using samples from P and G .
Revised: Equation (17) has the advantage that it can be estimated using samples from P and G .

Original: Given i.i.d.
Revised: Given i.i.d.

Original: samples ( X 1 ,...,X N ) and ( Y 1 ,...,Y M ) from P and G , we denote by ˆ P and ˆ G the corresponding empirical distributions.
Revised: samples ( X 1 ,...,X N ) and ( Y 1 ,...,Y M ) from P and G , we denote by ˆ P and ˆ G the corresponding empirical distributions.

Original: A simple approach to estimate KALE ( P || G ) is to use an M -estimator.
Revised: A simple approach to estimate KALE ( P || G ) is to use an M -estimator.

Original: This is achieved by optimizing the penalized objective
Revised: This is achieved by optimizing the penalized objective

Original: [Equation]
Revised: [Equation]

Original: (18) where “ F is an empirical version of F and I 2 ( h ) is a penalty term that prevents overﬁtting due to ﬁnite samples.
Revised: (18) where “ F is an empirical version of F and I 2 ( h ) is a penalty term that prevents overﬁtting due to ﬁnite samples.

Original: The penalty I 2 ( h ) acts as a regularizer favoring smoother solutions while the parameter λ determines the strength of the smoothing and is chosen to decrease as the sample size N and M increase.
Revised: The penalty I 2 ( h ) acts as a regularizer favoring smoother solutions while the parameter λ determines the strength of the smoothing and is chosen to decrease as the sample size N and M increase.

Original: The M -estimator of KALE ( P || G ) is obtained simply by plugging in ˆ h into the empirical objective “ F ( h ) :
Revised: The M -estimator of KALE ( P || G ) is obtained simply by plugging in ˆ h into the empirical objective “ F ( h ) :

Original: [Equation]
Revised: [Equation]

Original: We defer the consistency analysis of (19) to Appendix B where we provide convergence rates in a setting where the set of functions H is a Reproducing Kernel Hilbert Space and under weaker assumptions that were not covered by the framework of Nguyen et al.
Revised: We defer the consistency analysis of (19) to Appendix B where we provide convergence rates in a setting where the set of functions H is a Reproducing Kernel Hilbert Space and under weaker assumptions that were not covered by the framework of Nguyen et al.

Original: B C ONVERGENCE RATES OF KALE
Revised: B C ONVERGENCE RATES OF KALE

Original: In this section, we provide a convergence rate for the estimator in (19) when H is an RKHS.
Revised: In this section, we provide a convergence rate for the estimator in (19) when H is an RKHS.

Original: The theory remains the same whether H contains constants or not.
Revised: The theory remains the same whether H contains constants or not.

Original: With this choice, the Representer Theorem allows us to reduce the potentially inﬁnite-dimensional optimization problem in (18) to a convex ﬁnite-dimensional one.
Revised: With this choice, the Representer Theorem allows us to reduce the potentially inﬁnite-dimensional optimization problem in (18) to a convex ﬁnite-dimensional one.

Original: We further restrict ourselves to the well-speciﬁed case where the density rof P w.r.t.
Revised: We further restrict ourselves to the well-speciﬁed case where the density rof P w.r.t.

Original: G is well-deﬁned and belongs to H , so that KALE matches the KL.
Revised: G is well-deﬁned and belongs to H , so that KALE matches the KL.

Original: While Nguyen et al.
Revised: While Nguyen et al.

Original: (2010) (Theorem 3) provides a convergence rate of 1 / √ N for a related M -estimator, this requires the density r 0 to be lower-bounded by 0 as well as (generally) upper-bounded.
Revised: (2010) (Theorem 3) provides a convergence rate of 1 / √ N for a related M -estimator, this requires the density r 0 to be lower-bounded by 0 as well as (generally) upper-bounded.

Original: This can be quite restrictive if, for instance, r 0 is the density ratio of two gaussians.
Revised: This can be quite restrictive if, for instance, r 0 is the density ratio of two gaussians.

Original: In Theorem 7, we provide a similar convergence rate for the estimator deﬁned in (19) without requiring r 0 to be bounded.
Revised: In Theorem 7, we provide a similar convergence rate for the estimator deﬁned in (19) without requiring r 0 to be bounded.

Original: We start by brieﬂy introducing some notations, the working assumptions and the statement of the convergence result in Appendix B.1 and provide the proofs in Appendix B.2.
Revised: We start by brieﬂy introducing some notations, the working assumptions and the statement of the convergence result in Appendix B.1 and provide the proofs in Appendix B.2.

Original: B. S TATEMENT OF THE RESULT
Revised: B. S TATEMENT OF THE RESULT

Original: We recall that an RKHS H of functions deﬁned on a domain X ⊂ R d and with kernel k is a Hilbert space with dot product (cid:104) .
Revised: We recall that an RKHS H of functions deﬁned on a domain X ⊂ R d and with kernel k is a Hilbert space with dot product (cid:104) .

Original: (cid:105) ,
Revised: (cid:105) ,

Original: such that y (cid:55)→ k
Revised: such that y (cid:55)→ k

Original: ( x,y ) belongs to H for any x ∈X , and k ( x,y )
Revised: ( x,y ) belongs to H for any x ∈X , and k ( x,y )

Original: = (cid:104) k
Revised: = (cid:104) k

Original: ( x,. ) ,k ( y,. )
Revised: ( x,. ) ,k ( y,. )

Original: (cid:105) , ∀ x,y ∈X .
Revised: (cid:105) , ∀ x,y ∈X .

Original: Any function h in H satisﬁes the reproducing property f ( x )=
Revised: Any function h in H satisﬁes the reproducing property f ( x )=

Original: (cid:104) f,k ( x,. )
Revised: (cid:104) f,k ( x,. )

Original: (cid:105) for any x ∈X .
Revised: (cid:105) for any x ∈X .

Original: Recall that KALE ( P || G ) is obtained as an optimization problem
Revised: Recall that KALE ( P || G ) is obtained as an optimization problem

Original: [Equation]
Revised: [Equation]

Original: where F is given by:
Revised: where F is given by:

Original: [Equation]
Revised: [Equation]

Original: Since the negative log density ratio h 0 is assumed to belong to H , this directly implies that the supremum of F is achieved at h 0 and F ( h 0 ) =
Revised: Since the negative log density ratio h 0 is assumed to belong to H , this directly implies that the supremum of F is achieved at h 0 and F ( h 0 ) =

Original: KALE ( P || G ) .
Revised: KALE ( P || G ) .

Original: We are interested in estimating
Revised: We are interested in estimating

Original: KALE ( P || G ) using the empirical distributions ˆ P and ˆ G ,
Revised: KALE ( P || G ) using the empirical distributions ˆ P and ˆ G ,

Original: [Equation]
Revised: [Equation]

Original: where ( X n ) 1 ≤ n ≤ N and ( Y n ) 1 ≤ n ≤ N are i.i.d.
Revised: where ( X n ) 1 ≤ n ≤ N and ( Y n ) 1 ≤ n ≤ N are i.i.d.

Original: samples from P and G .
Revised: samples from P and G .

Original: For this purpose we introduce the empirical objective functional,
Revised: For this purpose we introduce the empirical objective functional,

Original: [Equation]
Revised: [Equation]

Original: The proposed estimator is obtained by solving a regularized empirical problem,
Revised: The proposed estimator is obtained by solving a regularized empirical problem,

Original: [Equation]
Revised: [Equation]

Original: with a corresponding population version,
Revised: with a corresponding population version,

Original: [Equation]
Revised: [Equation]

Original: Finally, we introduce D ( h,δ ) and Γ( h,δ ) :
Revised: Finally, we introduce D ( h,δ ) and Γ( h,δ ) :

Original: [Equation]
Revised: [Equation]

Original: The empirical versions of D ( h,δ ) and Γ( h,δ ) are denoted ˆ D ( h,δ ) and ˆΓ( h,δ ) .
Revised: The empirical versions of D ( h,δ ) and Γ( h,δ ) are denoted ˆ D ( h,δ ) and ˆΓ( h,δ ) .

Original: Later, we will show that D ( h,δ ) ˆ D ( h,δ ) are in fact the gradients of F ( h ) and “ F ( h ) along the direction δ .
Revised: Later, we will show that D ( h,δ ) ˆ D ( h,δ ) are in fact the gradients of F ( h ) and “ F ( h ) along the direction δ .

Original: We state now the working assumptions:
Revised: We state now the working assumptions:

Original: ( i )
Revised: ( i )

Original: The supremum of F over H is attained at h 0 .
Revised: The supremum of F over H is attained at h 0 .

Original: ( ii )
Revised: ( ii )

Original: The following quantities are ﬁnite for some positive (cid:15) :
Revised: The following quantities are ﬁnite for some positive (cid:15) :

Original: [Equation]
Revised: [Equation]

Original: ( iii )
Revised: ( iii )

Original: For any h ∈H , if D ( h,δ )=

Revised: For any h ∈H , if D ( h,δ )=

Original: for all δ then h = h 0 .
Revised: for all δ then h = h 0 .

Original: Theorem 7.
Revised: Theorem 7.

Original: Fix any 1 > η > 0 .
Revised: Fix any 1 > η > 0 .

Original: Under Assumptions ( i ) to ( iii ), and provided that λ = 1 √ N , it holds with probability at least 1 − 2 η that
Revised: Under Assumptions ( i ) to ( iii ), and provided that λ = 1 √ N , it holds with probability at least 1 − 2 η that

Original: [Equation]
Revised: [Equation]

Original: for a constant M (cid:48) ( η,h 0 ) that depends only on η and h 0 .
Revised: for a constant M (cid:48) ( η,h 0 ) that depends only on η and h 0 .

Original: The assumptions in Theorem 7 essentially state that the kernel associated to the RKHS H needs to satisfy some integrability requirements.
Revised: The assumptions in Theorem 7 essentially state that the kernel associated to the RKHS H needs to satisfy some integrability requirements.

Original: That is to guarantee that the gradient δ (cid:55)→∇F ( h )( δ ) and its empirical version are well-deﬁned and continuous.
Revised: That is to guarantee that the gradient δ (cid:55)→∇F ( h )( δ ) and its empirical version are well-deﬁned and continuous.

Original: In addition, the optimality condition ∇F ( h )=0 is assumed to characterize the global solution h 0 .
Revised: In addition, the optimality condition ∇F ( h )=0 is assumed to characterize the global solution h 0 .

Original: This will be the case if the kernel is characteristic Simon-Gabriel and Scholkopf (2018).
Revised: This will be the case if the kernel is characteristic Simon-Gabriel and Scholkopf (2018).

Original: The proof of Theorem 7, in Appendix B.2, takes advantage of the Hilbert structure of the set H , the convexity of the functional F and the optimality condition ∇ “ F (ˆ h )
Revised: The proof of Theorem 7, in Appendix B.2, takes advantage of the Hilbert structure of the set H , the convexity of the functional F and the optimality condition ∇ “ F (ˆ h )

Original: =
Revised: =

Original: λ
Revised: λ

Original: ˆ h of the regularized problem, all of which turn out to be sufﬁcient for controlling the error of (19).
Revised: ˆ h of the regularized problem, all of which turn out to be sufﬁcient for controlling the error of (19).

Original: B. P ROOFS
Revised: B. P ROOFS

Original: We state now the proof of Theorem 7 with subsequent lemmas and propositions.
Revised: We state now the proof of Theorem 7 with subsequent lemmas and propositions.

Original: Proof of Theorem 7.
Revised: Proof of Theorem 7.

Original: We begin with the following inequalities:
Revised: We begin with the following inequalities:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality is by deﬁnition of ˆ h while the second is obtained by concavity of “ F .
Revised: The ﬁrst inequality is by deﬁnition of ˆ h while the second is obtained by concavity of “ F .

Original: For simplicity we write B = (cid:107) ˆ h − h 0 (cid:107) and C =
Revised: For simplicity we write B = (cid:107) ˆ h − h 0 (cid:107) and C =

Original: (cid:107)∇ “ F ( h 0 )
Revised: (cid:107)∇ “ F ( h 0 )

Original: −L ( h 0 ) (cid:107) .
Revised: −L ( h 0 ) (cid:107) .

Original: Using Cauchy-Schwarz and triangular inequalities, it is easy to see that
Revised: Using Cauchy-Schwarz and triangular inequalities, it is easy to see that

Original: [Equation]
Revised: [Equation]

Original: Moreover, by triangular inequality, it holds that
Revised: Moreover, by triangular inequality, it holds that

Original: [Equation]
Revised: [Equation]

Original: Lemma 11 ensures that A ( λ )= (cid:107) h λ − h 0 (cid:107) converges to 0 as λ → 0 .
Revised: Lemma 11 ensures that A ( λ )= (cid:107) h λ − h 0 (cid:107) converges to 0 as λ → 0 .

Original: Furthermore, by Proposition 12, we have (cid:107) ˆ h − h λ (cid:107)≤ 1 λ D where D ( λ )=
Revised: Furthermore, by Proposition 12, we have (cid:107) ˆ h − h λ (cid:107)≤ 1 λ D where D ( λ )=

Original: (cid:107)∇ “ F ( h λ ) −∇L
Revised: (cid:107)∇ “ F ( h λ ) −∇L

Original: ( h λ ) (cid:107) .
Revised: ( h λ ) (cid:107) .

Original: Now choosing λ = 1 √ N and applying
Revised: Now choosing λ = 1 √ N and applying

Original: Chebychev inequality in Lemma 8, it follows that for any 1 >η> 0 , we have with probability greater than 1 − 2 η that both
Revised: Chebychev inequality in Lemma 8, it follows that for any 1 >η> 0 , we have with probability greater than 1 − 2 η that both

Original: [Equation]
Revised: [Equation]

Original: where C ( (cid:107) h 0 (cid:107) ,η ) is deﬁned in Lemma 8.
Revised: where C ( (cid:107) h 0 (cid:107) ,η ) is deﬁned in Lemma 8.

Original: This allows to conclude that for any η > 0 , it holds with probability at least 1 − 2 η that | “ F (ˆ h ) − “ F ( h
Revised: This allows to conclude that for any η > 0 , it holds with probability at least 1 − 2 η that | “ F (ˆ h ) − “ F ( h

Original: [Equation]
Revised: [Equation]

Original: where M (cid:48)
Revised: where M (cid:48)

Original: ( η,h 0 ) depends only on η and h 0 .
Revised: ( η,h 0 ) depends only on η and h 0 .

Original: We proceed using the following lemma, which provides an expression for D ( h,δ ) and ˆ D ( h,δ ) along with a probabilistic bound:
Revised: We proceed using the following lemma, which provides an expression for D ( h,δ ) and ˆ D ( h,δ ) along with a probabilistic bound:

Original: Lemma 8.
Revised: Lemma 8.

Original: Under Assumptions ( i ) and ( ii ), for any h ∈H such that (cid:107) h (cid:107)≤(cid:107) h 0 (cid:107) + (cid:15) , there exists D ( h ) in H satisfying
Revised: Under Assumptions ( i ) and ( ii ), for any h ∈H such that (cid:107) h (cid:107)≤(cid:107) h 0 (cid:107) + (cid:15) , there exists D ( h ) in H satisfying

Original: [Equation]
Revised: [Equation]

Original: and for any h ∈H , there exists “ D ( h ) satisfying
Revised: and for any h ∈H , there exists “ D ( h ) satisfying

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: Moreover, for any 0 <η < 1 and any h ∈H such that (cid:107) h (cid:107)≤(cid:107) h 0 (cid:107) + (cid:15) :
Revised: Moreover, for any 0 <η < 1 and any h ∈H such that (cid:107) h (cid:107)≤(cid:107) h 0 (cid:107) + (cid:15) :

Original: = M , it holds with probability greater than 1 − η that
Revised: = M , it holds with probability greater than 1 − η that

Original: [Equation]
Revised: [Equation]

Original: where C ( M,η ) depends only on M and η .
Revised: where C ( M,η ) depends only on M and η .

Original: Proof.
Revised: Proof.

Original: First, we show that δ (cid:55)→ D ( h,δ ) is a bounded linear operator.
Revised: First, we show that δ (cid:55)→ D ( h,δ ) is a bounded linear operator.

Original: Indeed, Assumption ( ii ) ensures that k ( x,. )
Revised: Indeed, Assumption ( ii ) ensures that k ( x,. )

Original: and k ( x,. )exp( − h ( x )) are Bochner integrable w.r.t.
Revised: and k ( x,. )exp( − h ( x )) are Bochner integrable w.r.t.

Original: P and G (Retherford (1978)), hence
Revised: P and G (Retherford (1978)), hence

Original: D ( h,δ ) is obtained as
Revised: D ( h,δ ) is obtained as

Original: [Equation]
Revised: [Equation]

Original: where µ exp( − h ) G = (cid:82) k ( x,. )
Revised: where µ exp( − h ) G = (cid:82) k ( x,. )

Original: exp( − h ( x ))
Revised: exp( − h ( x ))

Original: d G and µ P = (cid:82) k
Revised: d G and µ P = (cid:82) k

Original: ( x,. )
Revised: ( x,. )

Original: d P .
Revised: d P .

Original: Deﬁning D ( h ) to be = µ exp( − h )
Revised: Deﬁning D ( h ) to be = µ exp( − h )

Original: G − µ P leads to the desired result.
Revised: G − µ P leads to the desired result.

Original: “ D ( h ) is simply obtained by taking the empirical version of D ( h ) .
Revised: “ D ( h ) is simply obtained by taking the empirical version of D ( h ) .

Original: Finally, the probabilistic inequality is a simple consequence of Chebychev’s inequality.
Revised: Finally, the probabilistic inequality is a simple consequence of Chebychev’s inequality.

Original: The next lemma states that F ( h ) and “ F ( h ) are Frechet differentiable.
Revised: The next lemma states that F ( h ) and “ F ( h ) are Frechet differentiable.

Original: Lemma 9.
Revised: Lemma 9.

Original: Under Assumptions ( i ) and ( ii ) , h (cid:55)→ F ( h ) is Frechet differentiable on the open ball of radius (cid:107) h 0 (cid:107) + (cid:15) while h (cid:55)→ “ F ( h ) is Frechet differentiable on H .
Revised: Under Assumptions ( i ) and ( ii ) , h (cid:55)→ F ( h ) is Frechet differentiable on the open ball of radius (cid:107) h 0 (cid:107) + (cid:15) while h (cid:55)→ “ F ( h ) is Frechet differentiable on H .

Original: Their gradients are given by D ( h ) and “ D ( h ) as deﬁned in Lemma 8,
Revised: Their gradients are given by D ( h ) and “ D ( h ) as deﬁned in Lemma 8,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The empirical functional “ F ( h ) is differentiable since it is a ﬁnite sum of differentiable functions, and its gradient is simply given by “ D ( h ) .
Revised: The empirical functional “ F ( h ) is differentiable since it is a ﬁnite sum of differentiable functions, and its gradient is simply given by “ D ( h ) .

Original: For the population functional, we use second order Taylor expansion of exp with integral remainder, which gives
Revised: For the population functional, we use second order Taylor expansion of exp with integral remainder, which gives

Original: [Equation]
Revised: [Equation]

Original: By Assumption ( ii ) we know that Γ( h,δ ) (cid:107) δ (cid:107) converges to 0 as soon as (cid:107) δ (cid:107) → 0 .
Revised: By Assumption ( ii ) we know that Γ( h,δ ) (cid:107) δ (cid:107) converges to 0 as soon as (cid:107) δ (cid:107) → 0 .

Original: This allows to directly conclude that F is Frechet differentiable, with differential given by δ (cid:55)→ D ( h,δ ) .
Revised: This allows to directly conclude that F is Frechet differentiable, with differential given by δ (cid:55)→ D ( h,δ ) .

Original: By Lemma 8, we conclude the existence of a gradient ∇F ( h ) which is in fact given by ∇F ( h )=
Revised: By Lemma 8, we conclude the existence of a gradient ∇F ( h ) which is in fact given by ∇F ( h )=

Original: D ( h ) .
Revised: D ( h ) .

Original: From now on, we will only use the notation ∇F ( h ) and ∇ “ F ( h ) to refer to the gradients of F ( h ) and “ F ( h ) .
Revised: From now on, we will only use the notation ∇F ( h ) and ∇ “ F ( h ) to refer to the gradients of F ( h ) and “ F ( h ) .

Original: The following lemma states that (21) and (22) have a unique global optimum, and gives a ﬁrst order optimality condition.
Revised: The following lemma states that (21) and (22) have a unique global optimum, and gives a ﬁrst order optimality condition.

Original: Lemma 10.
Revised: Lemma 10.

Original: The problems (21) and (22) admit unique global solutions ˆ h and h λ in H .
Revised: The problems (21) and (22) admit unique global solutions ˆ h and h λ in H .

Original: Moreover, the following ﬁrst order optimality conditions hold:
Revised: Moreover, the following ﬁrst order optimality conditions hold:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: For (21), existence and uniqueness of a minimizer ˆ h is a simple consequence of continuity and strong concavity of the regularized objective.
Revised: For (21), existence and uniqueness of a minimizer ˆ h is a simple consequence of continuity and strong concavity of the regularized objective.

Original: We now show the existence result for (22).
Revised: We now show the existence result for (22).

Original: Let’s introduce
Revised: Let’s introduce

Original: G λ ( h )=
Revised: G λ ( h )=

Original: −F ( h )+ λ 2 (cid:107) h (cid:107) 2 for simplicity.
Revised: −F ( h )+ λ 2 (cid:107) h (cid:107) 2 for simplicity.

Original: Uniqueness is a consequence of the strong convexity of G λ .
Revised: Uniqueness is a consequence of the strong convexity of G λ .

Original: For the existence, consider a sequence of elements f k ∈ H such that G λ ( f k ) → inf h ∈H G λ ( h ) .
Revised: For the existence, consider a sequence of elements f k ∈ H such that G λ ( f k ) → inf h ∈H G λ ( h ) .

Original: If h is not the global solution, then it must hold for k large enough that G λ ( f k ) ≤G λ ( h 0 ) .
Revised: If h is not the global solution, then it must hold for k large enough that G λ ( f k ) ≤G λ ( h 0 ) .

Original: We also know that F ( f k ) ≤ F ( h 0 ) , hence, it is easy to see that (cid:107) f k (cid:107) ≤ (cid:107) h 0 (cid:107) for k large enough.
Revised: We also know that F ( f k ) ≤ F ( h 0 ) , hence, it is easy to see that (cid:107) f k (cid:107) ≤ (cid:107) h 0 (cid:107) for k large enough.

Original: This implies that f k is a bounded sequence, therefore it admits a weakly convergent sub-sequence by weak compactness.
Revised: This implies that f k is a bounded sequence, therefore it admits a weakly convergent sub-sequence by weak compactness.

Original: Without loss of generality we assume that f k weakly converges to some element h λ ∈ H and that (cid:107)
Revised: Without loss of generality we assume that f k weakly converges to some element h λ ∈ H and that (cid:107)

Original: f
Revised: f

Original: k (cid:107)≤(cid:107)
Revised: k (cid:107)≤(cid:107)

Original: h 0 (cid:107) .
Revised: h 0 (cid:107) .

Original: Hence, (cid:107) h λ (cid:107)≤ liminf k (cid:107)
Revised: Hence, (cid:107) h λ (cid:107)≤ liminf k (cid:107)

Original: f
Revised: f

Original: k (cid:107)≤(cid:107)
Revised: k (cid:107)≤(cid:107)

Original: h 0 (cid:107) .
Revised: h 0 (cid:107) .

Original: Recall now that by deﬁnition of weak convergence, we have f k ( x ) → k h λ ( x ) for all x ∈X .
Revised: Recall now that by deﬁnition of weak convergence, we have f k ( x ) → k h λ ( x ) for all x ∈X .

Original: By Assumption ( ii ), we can apply the dominated convergence theorem to ensure that F ( f k ) →F ( h λ ) .
Revised: By Assumption ( ii ), we can apply the dominated convergence theorem to ensure that F ( f k ) →F ( h λ ) .

Original: Taking the limit of G λ f k , the following inequality holds:
Revised: Taking the limit of G λ f k , the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: Finally, by Lemma 9 we know that F is Frechet differentiable, hence we can use Ekeland and Témam (1999)
Revised: Finally, by Lemma 9 we know that F is Frechet differentiable, hence we can use Ekeland and Témam (1999)

Original: (Proposition 2.1) to conclude that ∇F ( h λ )
Revised: (Proposition 2.1) to conclude that ∇F ( h λ )

Original: = λh λ .
Revised: = λh λ .

Original: We use exactly the same arguments for (21).
Revised: We use exactly the same arguments for (21).

Original: Next, we show that h λ converges towards h 0 in H .
Revised: Next, we show that h λ converges towards h 0 in H .

Original: Lemma 11.
Revised: Lemma 11.

Original: Under Assumptions ( i ) to ( iii ) it holds that:
Revised: Under Assumptions ( i ) to ( iii ) it holds that:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We will ﬁrst prove that h λ converges weakly towards h 0 , and then conclude that it must also converge strongly.
Revised: We will ﬁrst prove that h λ converges weakly towards h 0 , and then conclude that it must also converge strongly.

Original: We start with the following inequalities:
Revised: We start with the following inequalities:

Original: [Equation]
Revised: [Equation]

Original: These are simple consequences of the deﬁnitions of h λ and h 0 as optimal solutions to (20) and (21).
Revised: These are simple consequences of the deﬁnitions of h λ and h 0 as optimal solutions to (20) and (21).

Original: This implies that (cid:107) h λ (cid:107) is always bounded by (cid:107) h 0 (cid:107) .
Revised: This implies that (cid:107) h λ (cid:107) is always bounded by (cid:107) h 0 (cid:107) .

Original: Consider now an arbitrary sequence ( λ m ) m ≥converging to 0 .
Revised: Consider now an arbitrary sequence ( λ m ) m ≥converging to 0 .

Original: Since (cid:107) h λ m (cid:107) is bounded by (cid:107) h 0 (cid:107) , it follows by weak-compactness of balls in H that h λ m admits a weakly convergent sub-sequence.
Revised: Since (cid:107) h λ m (cid:107) is bounded by (cid:107) h 0 (cid:107) , it follows by weak-compactness of balls in H that h λ m admits a weakly convergent sub-sequence.

Original: Without loss of generality we can assume that h λ m is itself weakly converging towards an element h ∗ .
Revised: Without loss of generality we can assume that h λ m is itself weakly converging towards an element h ∗ .

Original: We will show now that h ∗ must be equal to h 0 .
Revised: We will show now that h ∗ must be equal to h 0 .

Original: Indeed, by optimality of h λ m , it must hold that
Revised: Indeed, by optimality of h λ m , it must hold that

Original: [Equation]
Revised: [Equation]

Original: This implies that ∇F ( h m ) converges weakly to 0 .
Revised: This implies that ∇F ( h m ) converges weakly to 0 .

Original: On the other hand, by Assumption ( ii ), we can conclude that ∇F ( h m ) must also converge weakly towards ∇F ( h ∗ ) , hence ∇F ( h ∗ )
Revised: On the other hand, by Assumption ( ii ), we can conclude that ∇F ( h m ) must also converge weakly towards ∇F ( h ∗ ) , hence ∇F ( h ∗ )

Original: = 0 .
Revised: = 0 .

Original: Finally by Assumption ( iii ) we know that h 0 is the unique solution to the equation ∇F ( h )=0 , hence h ∗ =
Revised: Finally by Assumption ( iii ) we know that h 0 is the unique solution to the equation ∇F ( h )=0 , hence h ∗ =

Original: h 0 .
Revised: h 0 .

Original: We have shown so far that any subsequence of h λ m that converges weakly, must converge weakly towards h 0 .
Revised: We have shown so far that any subsequence of h λ m that converges weakly, must converge weakly towards h 0 .

Original: This allows to conclude that h λ m actually converges weakly towards h 0 .
Revised: This allows to conclude that h λ m actually converges weakly towards h 0 .

Original: Moreover, we also have by deﬁnition of weak convergence that:
Revised: Moreover, we also have by deﬁnition of weak convergence that:

Original: [Equation]
Revised: [Equation]

Original: Recalling now that (cid:107) h λ m (cid:107) ≤ (cid:107) h 0 (cid:107) it follows that (cid:107) h λ m (cid:107) converges towards (cid:107) h 0 (cid:107) .
Revised: Recalling now that (cid:107) h λ m (cid:107) ≤ (cid:107) h 0 (cid:107) it follows that (cid:107) h λ m (cid:107) converges towards (cid:107) h 0 (cid:107) .

Original: Hence, we have the following two properties:
Revised: Hence, we have the following two properties:

Original: h λ m converges weakly towards h 0 , • (cid:107) h
Revised: h λ m converges weakly towards h 0 , • (cid:107) h

Original: λ m (cid:107)
Revised: λ m (cid:107)

Original: converges towards (cid:107) h 0 (cid:107) .
Revised: converges towards (cid:107) h 0 (cid:107) .

Original: This allows to directly conclude that (cid:107) h
Revised: This allows to directly conclude that (cid:107) h

Original: λ m − h 0 (cid:107) converges to 0 .
Revised: λ m − h 0 (cid:107) converges to 0 .

Original: Proposition 12.
Revised: Proposition 12.

Original: We have that:
Revised: We have that:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By deﬁnition of ˆ h and h λ the following optimality conditions hold:
Revised: By deﬁnition of ˆ h and h λ the following optimality conditions hold:

Original: [Equation]
Revised: [Equation]

Original: We can then simply write:
Revised: We can then simply write:

Original: [Equation]
Revised: [Equation]

Original: Now introducing δ :
Revised: Now introducing δ :

Original: =
Revised: =

Original: ˆ h − h λ and E := ∇ “ F (ˆ h ) −∇ “ F ( h λ ) for simplicity and taking the squared norm of the above equation
Revised: ˆ h − h λ and E := ∇ “ F (ˆ h ) −∇ “ F ( h λ ) for simplicity and taking the squared norm of the above equation

Original: [Equation]
Revised: [Equation]

Original: it follows that
Revised: it follows that

Original: [Equation]
Revised: [Equation]

Original: By concavity of “ F on H we know that −(cid:104)
Revised: By concavity of “ F on H we know that −(cid:104)

Original: ˆ h − h λ ,E (cid:105)≥ 0 .
Revised: ˆ h − h λ ,E (cid:105)≥ 0 .

Original: Therefore:
Revised: Therefore:

Original: [Equation]
Revised: [Equation]

Original: B. E STIMATING K ALE WITH ENERGIES IN AN RKHS
Revised: B. E STIMATING K ALE WITH ENERGIES IN AN RKHS

Original: Recall the expression of the empirical objective:
Revised: Recall the expression of the empirical objective:

Original: [Equation]
Revised: [Equation]

Original: The representer theorem allows to express the optimal h (cid:63) in the following form:
Revised: The representer theorem allows to express the optimal h (cid:63) in the following form:

Original: [Equation]
Revised: [Equation]

Original: Plugging in the above expression in (23)
Revised: Plugging in the above expression in (23)

Original: [Equation]
Revised: [Equation]

Original: and introducing the matrix K and vector m with entries given by K ( Y i ,Y j )
Revised: and introducing the matrix K and vector m with entries given by K ( Y i ,Y j )

Original: and m i
Revised: and m i

Original: = ˆ µ P ( Y i ) it follows:
Revised: = ˆ µ P ( Y i ) it follows:

Original: [Equation]
Revised: [Equation]

Original: The optimal values for α and c for a given β are:
Revised: The optimal values for α and c for a given β are:

Original: [Equation]
Revised: [Equation]

Original: Using those optimal solution in (24) gives another concave objective in β :
Revised: Using those optimal solution in (24) gives another concave objective in β :

Original: [Equation]
Revised: [Equation]

Original: with S ( β ) given by S ( β )=
Revised: with S ( β ) given by S ( β )=

Original: exp( mλ − Kβ ) .
Revised: exp( mλ − Kβ ) .

Original: This can be optimized iteratively using Newton’s method.
Revised: This can be optimized iteratively using Newton’s method.

Original: We provide the expression for the gradient:
Revised: We provide the expression for the gradient:

Original: [Equation]
Revised: [Equation]

Original: The matrix K gets simpliﬁed when performing newton’s updates:
Revised: The matrix K gets simpliﬁed when performing newton’s updates:

Original: [Equation]
Revised: [Equation]

Original: with E ( β )=( diag ( (cid:101) S ( β
Revised: with E ( β )=( diag ( (cid:101) S ( β

Original: [Equation]
Revised: [Equation]

Original: ) −
Revised: ) −

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: S (
Revised: S (

Original: [Equation]
Revised: [Equation]

Original: ) (cid:62) ) where we introduced the normalized vector (cid:101) S ( β )=
Revised: ) (cid:62) ) where we introduced the normalized vector (cid:101) S ( β )=

Original: [Equation]
Revised: [Equation]

Original: S
Revised: S

Original: [Equation]
Revised: [Equation]

Original: β
Revised: β

Original: [Equation]
Revised: [Equation]

Original: S
Revised: S

Original: [Equation]
Revised: [Equation]

Original: β
Revised: β

Original: [Equation]
Revised: [Equation]

Original: The optimal coefﬁcients α and β are obtained by solving the following maximization problem:
Revised: The optimal coefﬁcients α and β are obtained by solving the following maximization problem:

Original: [Equation]
Revised: [Equation]

Original: C L ATENT NOISE SAMPLING AND S MOOTHNESS OF KALE C. L ATENT SPACE SAMPLING
Revised: C L ATENT NOISE SAMPLING AND S MOOTHNESS OF KALE C. L ATENT SPACE SAMPLING

Original: Here we prove Proposition 6 for which we make the assumptions more precise:
Revised: Here we prove Proposition 6 for which we make the assumptions more precise:

Original: Assumption 1.
Revised: Assumption 1.

Original: We make the following assumption:
Revised: We make the following assumption:

Original: • log η is strongly concave and admits a Lipschitz gradient.
Revised: • log η is strongly concave and admits a Lipschitz gradient.

Original: • There exists a non-negative constant L such that for any x,x (cid:48) ∈X and z,z (cid:48) ∈Z : | E ( x ) − E ( x (cid:48) )
Revised: • There exists a non-negative constant L such that for any x,x (cid:48) ∈X and z,z (cid:48) ∈Z : | E ( x ) − E ( x (cid:48) )

Original: |≤(cid:107) x − x (cid:48) (cid:107) ,
Revised: |≤(cid:107) x − x (cid:48) (cid:107) ,

Original: [Equation]
Revised: [Equation]

Original: Throughout this section, we introduce U ( z ):= − log( η ( z ))
Revised: Throughout this section, we introduce U ( z ):= − log( η ( z ))

Original: + E ( G ( z )) for simplicity.
Revised: + E ( G ( z )) for simplicity.

Original: Proof of Proposition 1 .
Revised: Proof of Proposition 1 .

Original: To sample from Q G ,E , we ﬁrst need to identify the posterior latent distribution ν G ,E used to produce those samples.
Revised: To sample from Q G ,E , we ﬁrst need to identify the posterior latent distribution ν G ,E used to produce those samples.

Original: We rely on (25) which holds by deﬁnition of Q G ,E for any test function h on X :
Revised: We rely on (25) which holds by deﬁnition of Q G ,E for any test function h on X :

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Hence, the posterior latent distribution is given by ν ( z )
Revised: Hence, the posterior latent distribution is given by ν ( z )

Original: =
Revised: =

Original: η ( z ) f ( G ( z )) , and samples from GEBM are produced by ﬁrst sampling from ν G ,E , then applying the implicit map G ,
Revised: η ( z ) f ( G ( z )) , and samples from GEBM are produced by ﬁrst sampling from ν G ,E , then applying the implicit map G ,

Original: [Equation]
Revised: [Equation]

Original: Proof of Proposition 2.
Revised: Proof of Proposition 2.

Original: the base distribution G admits a density on the whole space denoted by exp( − r ( x )) and
Revised: the base distribution G admits a density on the whole space denoted by exp( − r ( x )) and

Original: the energy ˜ E is of the form ˜ E ( x ) =
Revised: the energy ˜ E is of the form ˜ E ( x ) =

Original: E ( x ) − r ( x ) for some parametric function E , it is easy to see that Q has a density proportional to exp( − E ) and is therefore equivalent to a standard
Revised: E ( x ) − r ( x ) for some parametric function E , it is easy to see that Q has a density proportional to exp( − E ) and is therefore equivalent to a standard

Original: EBM with energy E .
Revised: EBM with energy E .

Original: The converse holds as well, meaning that for any EBM with energy E , it is possible to construct a
Revised: The converse holds as well, meaning that for any EBM with energy E , it is possible to construct a

Original: GEBM using an importance weighting strategy.
Revised: GEBM using an importance weighting strategy.

Original: This is achieved by ﬁrst choosing a base G , which is required to have an explicit density exp( − r ) up to a normalizing constant, then deﬁning the energy of the GEBM to be ˜ E ( x )=
Revised: This is achieved by ﬁrst choosing a base G , which is required to have an explicit density exp( − r ) up to a normalizing constant, then deﬁning the energy of the GEBM to be ˜ E ( x )=

Original: E ( x ) − r ( x ) so that:
Revised: E ( x ) − r ( x ) so that:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Equation (26) effectively depends only on E ( x ) and not on G since the factor exp( r ) exactly compensates for the density of G .
Revised: Equation (26) effectively depends only on E ( x ) and not on G since the factor exp( r ) exactly compensates for the density of G .

Original: The requirement that the base also admits a tractable implicit map
Revised: The requirement that the base also admits a tractable implicit map

Original: G can be met by choosing G to be a normalizing ﬂow (Rezende and Mohamed, 2015) and does not restrict the class of possible EBMs that can be expressed as GEBMs.
Revised: G can be met by choosing G to be a normalizing ﬂow (Rezende and Mohamed, 2015) and does not restrict the class of possible EBMs that can be expressed as GEBMs.

Original: Proof of Proposition 6.
Revised: Proof of Proposition 6.

Original: Let π t be the probability distribution of ( z t ,v t ) at time t of the diffusion in (14), which we recall that
Revised: Let π t be the probability distribution of ( z t ,v t ) at time t of the diffusion in (14), which we recall that

Original: [Equation]
Revised: [Equation]

Original: We call π ∞ its corresponding invariant distribution given by
Revised: We call π ∞ its corresponding invariant distribution given by

Original: [Equation]
Revised: [Equation]

Original: By Lemma 13 we know that U is dissipative, bounded from below, and has a Lipschitz gradient.
Revised: By Lemma 13 we know that U is dissipative, bounded from below, and has a Lipschitz gradient.

Original: This allows to directly apply (Eberle et al., 2017)(Corollary 2.6.) which implies that
Revised: This allows to directly apply (Eberle et al., 2017)(Corollary 2.6.) which implies that

Original: [Equation]
Revised: [Equation]

Original: where c is a positive constant and C only depends on π ∞ and the initial distribution π 0 .
Revised: where c is a positive constant and C only depends on π ∞ and the initial distribution π 0 .

Original: Moreover, the constant c is given explicitly in (Eberle et al., 2017, Theorem 2.3) and is of order 

Revised: Moreover, the constant c is given explicitly in (Eberle et al., 2017, Theorem 2.3) and is of order

Original: ( e − q ) where q is the dimension of the latent space Z .
Revised: ( e − q ) where q is the dimension of the latent space Z .

Original: We now consider an optimal coupling Π t between π t and π 0 .
Revised: We now consider an optimal coupling Π t between π t and π 0 .

Original: Given joints samples (( z t ,v t ) , ( z,v )) from Π t , we consider the following samples in input space ( x t ,x ) :
Revised: Given joints samples (( z t ,v t ) , ( z,v )) from Π t , we consider the following samples in input space ( x t ,x ) :

Original: = ( G ( z t ) ,G ( z )) .
Revised: = ( G ( z t ) ,G ( z )) .

Original: Since z t and z have marginals π t and π ∞ , it is easy to see that x t ∼ P t and x ∼ Q .
Revised: Since z t and z have marginals π t and π ∞ , it is easy to see that x t ∼ P t and x ∼ Q .

Original: Therefore, by deﬁnition of the
Revised: Therefore, by deﬁnition of the

Original: W 2 distance, we have the following bound:
Revised: W 2 distance, we have the following bound:

Original: [Equation]
Revised: [Equation]

Original: The second line uses the deﬁnition of ( x t ,x ) as joint samples obtained by mapping ( z t ,z ) .
Revised: The second line uses the deﬁnition of ( x t ,x ) as joint samples obtained by mapping ( z t ,z ) .

Original: The third line uses the assumption that B is L -Lipschitz.
Revised: The third line uses the assumption that B is L -Lipschitz.

Original: Finally, the last line uses that Π t is an optimal coupling between π t and π ∞ .
Revised: Finally, the last line uses that Π t is an optimal coupling between π t and π ∞ .

Original: Lemma 13.
Revised: Lemma 13.

Original: Under Assumption 1, there exists A> 0 and λ ∈ (0 , 14 ] such that
Revised: Under Assumption 1, there exists A> 0 and λ ∈ (0 , 14 ] such that

Original: [Equation]
Revised: [Equation]

Original: (27) where γ and u are the coefﬁcients appearing in (14).
Revised: (27) where γ and u are the coefﬁcients appearing in (14).

Original: Moreover, U is bounded bellow and has a
Revised: Moreover, U is bounded bellow and has a

Original: Lipschitz gradient.
Revised: Lipschitz gradient.

Original: Proof.
Revised: Proof.

Original: For simplicity, let’s call u ( z )
Revised: For simplicity, let’s call u ( z )

Original: =
Revised: =

Original: − log η ( z ) , w ( z ) =
Revised: − log η ( z ) , w ( z ) =

Original: E (cid:63) ◦
Revised: E (cid:63) ◦

Original: B θ (cid:63) ( z ) , and denote by M an upper-bound on the Lipschitz constant of w and ∇ w which is guaranteed to be ﬁnite by assumption.
Revised: B θ (cid:63) ( z ) , and denote by M an upper-bound on the Lipschitz constant of w and ∇ w which is guaranteed to be ﬁnite by assumption.

Original: Hence U ( z )=
Revised: Hence U ( z )=

Original: u ( z )+ w ( z ) .
Revised: u ( z )+ w ( z ) .

Original: Equation (27) is equivalent to having
Revised: Equation (27) is equivalent to having

Original: [Equation]
Revised: [Equation]

Original: Using that w is Lipschitz, we have that w ( z ) ≤ w (0) + M (cid:107) z (cid:107) and − z (cid:62) ∇ w ( z ) ≤ M (cid:107)
Revised: Using that w is Lipschitz, we have that w ( z ) ≤ w (0) + M (cid:107) z (cid:107) and − z (cid:62) ∇ w ( z ) ≤ M (cid:107)

Original: z (cid:107) .
Revised: z (cid:107) .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: A .
Revised: A .

Original: Therefore, a sufﬁcient condition for (28) to hold is
Revised: Therefore, a sufﬁcient condition for (28) to hold is

Original: [Equation]
Revised: [Equation]

Original: We will now rely on the strong convexity of u, which holds by assumption, and implies the existence of a positive constant m> 0 such that
Revised: We will now rely on the strong convexity of u, which holds by assumption, and implies the existence of a positive constant m> 0 such that

Original: [Equation]
Revised: [Equation]

Original: This allows to write the following inequality,
Revised: This allows to write the following inequality,

Original: [Equation]
Revised: [Equation]

Original: Combining the previous inequality with (29) and denoting M (cid:48)
Revised: Combining the previous inequality with (29) and denoting M (cid:48)

Original: =
Revised: =

Original: (cid:107)∇ u (0) (cid:107) , it is sufﬁcient to ﬁnd A and λ satisfying
Revised: (cid:107)∇ u (0) (cid:107) , it is sufﬁcient to ﬁnd A and λ satisfying

Original: [Equation]
Revised: [Equation]

Original: The l.h.s.
Revised: The l.h.s.

Original: in the above equation is a quadratic function in (cid:107) z (cid:107) and admits a global minimum when λ<
Revised: in the above equation is a quadratic function in (cid:107) z (cid:107) and admits a global minimum when λ<

Original: Ä m + γ 2 2 u ä − 1 .
Revised: Ä m + γ 2 2 u ä − 1 .

Original: The global minimum is always positive provided that A is large enough.
Revised: The global minimum is always positive provided that A is large enough.

Original: To see that U is bounded below, it sufﬁce to note, by Lipschitzness of w , that w ( z ) ≥ w (0) − M (cid:107) z (cid:107) and by strong convexity of u that
Revised: To see that U is bounded below, it sufﬁce to note, by Lipschitzness of w , that w ( z ) ≥ w (0) − M (cid:107) z (cid:107) and by strong convexity of u that

Original: [Equation]
Revised: [Equation]

Original: Hence, U is lower-bounded by a quadratic function in (cid:107) z (cid:107) with positive leading coefﬁcient m 2 , hence it must be lower-bounded by a constant.
Revised: Hence, U is lower-bounded by a quadratic function in (cid:107) z (cid:107) with positive leading coefﬁcient m 2 , hence it must be lower-bounded by a constant.

Original: Finally, by assumption, u and w have Lipschitz gradients, which directly implies that U has a Lipschitz gradient.
Revised: Finally, by assumption, u and w have Lipschitz gradients, which directly implies that U has a Lipschitz gradient.

Original: Proof of Proposition 3.
Revised: Proof of Proposition 3.

Original: By assumption KL ( P || G ) < + ∞ , this implies that P admits a density w.r.t.
Revised: By assumption KL ( P || G ) < + ∞ , this implies that P admits a density w.r.t.

Original: G which we call r ( x ) .
Revised: G which we call r ( x ) .

Original: As a result P admits also a density w.r.t.
Revised: As a result P admits also a density w.r.t.

Original: Q given by:
Revised: Q given by:

Original: [Equation]
Revised: [Equation]

Original: We can then compute the KL ( P || Q ) explicitly:
Revised: We can then compute the KL ( P || Q ) explicitly:

Original: [Equation]
Revised: [Equation]

Original: Since 0 belongs to E and by optimality of E (cid:63) , we know that L P , G ( E (cid:63) ) ≥L
Revised: Since 0 belongs to E and by optimality of E (cid:63) , we know that L P , G ( E (cid:63) ) ≥L

Original: P , G (0)=0 .
Revised: P , G (0)=0 .

Original: The result then follows directly.
Revised: The result then follows directly.

Original: [Equation]
Revised: [Equation]

Original: T OPOLOGICAL AND SMOOTHNESS PROPERTIES OF KALE
Revised: T OPOLOGICAL AND SMOOTHNESS PROPERTIES OF KALE

Original: Topological properties of KALE.
Revised: Topological properties of KALE.

Original: Denseness and smoothness of the energy class
Revised: Denseness and smoothness of the energy class

Original: E are the key to guarantee that KALE is a reliable criterion for measuring convergence.
Revised: E are the key to guarantee that KALE is a reliable criterion for measuring convergence.

Original: We thus make the following assumptions on E :
Revised: We thus make the following assumptions on E :

Original: ( A ) For all E ∈ E , − E ∈ E
Revised: ( A ) For all E ∈ E , − E ∈ E

Original: and there is C E > 

Revised: and there is C E >

Original: such that cE ∈ E for 0 ≤ c ≤ C E .
Revised: such that cE ∈ E for 0 ≤ c ≤ C E .

Original: For any continuous function g , any compact support K in X and any precision (cid:15) > 0 , there exists a ﬁnite linear combination of energies G = (cid:80) ri =1 a
Revised: For any continuous function g , any compact support K in X and any precision (cid:15) > 0 , there exists a ﬁnite linear combination of energies G = (cid:80) ri =1 a

Original: i E i such that sup x ∈ K | f ( x ) − G ( x ) |≤ (cid:15).
Revised: i E i such that sup x ∈ K | f ( x ) − G ( x ) |≤ (cid:15).

Original: ( B ) All energies E in E are Lipschitz in their input with the same Lipschitz constant L> 0 .
Revised: ( B ) All energies E in E are Lipschitz in their input with the same Lipschitz constant L> 0 .

Original: Assumption ( A ) holds in particular when E contains feedforward networks with a given number of parameters.
Revised: Assumption ( A ) holds in particular when E contains feedforward networks with a given number of parameters.

Original: In fact networks with a single neuron are enough, as shown in (Zhang et al., 2017, Theorem 2.3).
Revised: In fact networks with a single neuron are enough, as shown in (Zhang et al., 2017, Theorem 2.3).

Original: Assumption ( B ) holds when additional regularization of the energy is enforced during training by methods such as spectral normalization Miyato et al.
Revised: Assumption ( B ) holds when additional regularization of the energy is enforced during training by methods such as spectral normalization Miyato et al.

Original: (2018) or gradient penalty Gulrajani et al.
Revised: (2018) or gradient penalty Gulrajani et al.

Original: (2017) as done in Section 6.
Revised: (2017) as done in Section 6.

Original: Proposition 4 states the topological properties of KALE ensuring that it can be used as a criterion for weak convergence.
Revised: Proposition 4 states the topological properties of KALE ensuring that it can be used as a criterion for weak convergence.

Original: A proof is given in Appendix C.2.1 and is a consequence of (Zhang et al., 2017, Theorem B.1).
Revised: A proof is given in Appendix C.2.1 and is a consequence of (Zhang et al., 2017, Theorem B.1).

Original: Proposition 14.
Revised: Proposition 14.

Original: Under Assumptions ( A ) and ( B ) it holds that:
Revised: Under Assumptions ( A ) and ( B ) it holds that:

Original: KALE ( P || G ) ≥ 0 with KALE ( P || G )=

Revised: KALE ( P || G ) ≥ 0 with KALE ( P || G )=

Original: if and only if P = G . 2.
Revised: if and only if P = G . 2.

Original: KALE ( P || G n ) → 0 if and only if G n → P under the weak topology.
Revised: KALE ( P || G n ) → 0 if and only if G n → P under the weak topology.

Original: C.2. T OPOLOGICAL PROPERTIES OF KALE
Revised: C.2. T OPOLOGICAL PROPERTIES OF KALE

Original: In this section we prove Proposition 4.
Revised: In this section we prove Proposition 4.

Original: We ﬁrst start by recalling the required assumptions and make them more precise:
Revised: We ﬁrst start by recalling the required assumptions and make them more precise:

Original: Assumption 2.
Revised: Assumption 2.

Original: Assume the following holds:
Revised: Assume the following holds:

Original: • The set X is compact.
Revised: • The set X is compact.

Original: • For all E ∈ E , − E ∈ E and there is C E > 0 such that cE ∈ E for 0 ≤ c ≤ C E .
Revised: • For all E ∈ E , − E ∈ E and there is C E > 0 such that cE ∈ E for 0 ≤ c ≤ C E .

Original: For any continuous function g , any compact support K in X and any precision (cid:15) > 0 , there exists a ﬁnite linear combination of energies G = (cid:80) ri =1 a i E i such that |
Revised: For any continuous function g , any compact support K in X and any precision (cid:15) > 0 , there exists a ﬁnite linear combination of energies G = (cid:80) ri =1 a i E i such that |

Original: f ( x ) − G ( x ) |≤ (cid:15) on K .
Revised: f ( x ) − G ( x ) |≤ (cid:15) on K .

Original: • All energies E in E are Lipschitz in their input with the same Lipschitz constant L> 0 .
Revised: • All energies E in E are Lipschitz in their input with the same Lipschitz constant L> 0 .

Original: For simplicity we consider the set H =
Revised: For simplicity we consider the set H =

Original: E + R , i.e.: H is the set of functions h of the form h
Revised: E + R , i.e.: H is the set of functions h of the form h

Original: = E
Revised: = E

Original: + c
Revised: + c

Original: where E ∈ E and c ∈ R
Revised: where E ∈ E and c ∈ R

Original: In all what follows P 1 is the set of probability distributions with ﬁnite ﬁrst order moments.
Revised: In all what follows P 1 is the set of probability distributions with ﬁnite ﬁrst order moments.

Original: We consider the notion of weak convergence on P 1 as deﬁned in (Villani, 2009,
Revised: We consider the notion of weak convergence on P 1 as deﬁned in (Villani, 2009,

Original: Deﬁnition 6.8) which is equivalent to convergence in the Wasserstein-1 distance W 1 .
Revised: Deﬁnition 6.8) which is equivalent to convergence in the Wasserstein-1 distance W 1 .

Original: Proof of Proposition 4 .
Revised: Proof of Proposition 4 .

Original: We proceed by proving the separation properties ( 1 st statement), then the metrization of the weak topology ( 2 nd statement).
Revised: We proceed by proving the separation properties ( 1 st statement), then the metrization of the weak topology ( 2 nd statement).

Original: Separation.
Revised: Separation.

Original: We have by Assumption 2 that 0 ∈E , hence by deﬁnition KALE ( PP || G ) ≥F
Revised: We have by Assumption 2 that 0 ∈E , hence by deﬁnition KALE ( PP || G ) ≥F

Original: [Equation]
Revised: [Equation]

Original: , G
Revised: , G

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: On the other hand, whenever P = G , it holds that:
Revised: On the other hand, whenever P = G , it holds that:

Original: [Equation]
Revised: [Equation]

Original: Moreover, by convexity of the exponential, we know that exp( − x )+ x − 1 ≥ 0 for all x ∈ R .
Revised: Moreover, by convexity of the exponential, we know that exp( − x )+ x − 1 ≥ 0 for all x ∈ R .

Original: Hence,
Revised: Hence,

Original: F P , G ( h ) ≤ F P , G (0) = 0 for all h ∈ H .
Revised: F P , G ( h ) ≤ F P , G (0) = 0 for all h ∈ H .

Original: This directly implies that KALE ( P | G ) = 0 .
Revised: This directly implies that KALE ( P | G ) = 0 .

Original: For the converse, we will use the same argument as in the proof of (Zhang et al., 2017, Theorem B.1).
Revised: For the converse, we will use the same argument as in the proof of (Zhang et al., 2017, Theorem B.1).

Original: Assume that
Revised: Assume that

Original: KALE ( P | G )=0 and let h be in H .
Revised: KALE ( P | G )=0 and let h be in H .

Original: By Assumption 2, there exists C h > 0 such that ch ∈H and we have:
Revised: By Assumption 2, there exists C h > 0 such that ch ∈H and we have:

Original: [Equation]
Revised: [Equation]

Original: Now dividing by c and taking the limit to 0 , it is easy to see that − (cid:82) h d P + (cid:82)
Revised: Now dividing by c and taking the limit to 0 , it is easy to see that − (cid:82) h d P + (cid:82)

Original: h d G ≤ 0 .
Revised: h d G ≤ 0 .

Original: Again, by Assumption 2, we also know that − h ∈H , hence, (cid:82)
Revised: Again, by Assumption 2, we also know that − h ∈H , hence, (cid:82)

Original: h
Revised: h

Original: d P − (cid:82) h d G ≤ 0 .
Revised: d P − (cid:82) h d G ≤ 0 .

Original: This necessarily implies that (cid:82) h
Revised: This necessarily implies that (cid:82) h

Original: d P − (cid:82) h
Revised: d P − (cid:82) h

Original: d G =0 for all h ∈H .
Revised: d G =0 for all h ∈H .

Original: By the density of H in the set continuous functions on compact sets, we can conclude that the equality holds for any continuous and bounded function, which in turn implies that P = G .
Revised: By the density of H in the set continuous functions on compact sets, we can conclude that the equality holds for any continuous and bounded function, which in turn implies that P = G .

Original: Metrization of the weak topology.
Revised: Metrization of the weak topology.

Original: We ﬁrst show that for any P and G with ﬁnite ﬁrst moment, it holds that KALE ( P | G ) ≤ LW 1 ( P , G ) , where W 1 ( P , G ) is the Wasserstein-1 distance between P and G .
Revised: We ﬁrst show that for any P and G with ﬁnite ﬁrst moment, it holds that KALE ( P | G ) ≤ LW 1 ( P , G ) , where W 1 ( P , G ) is the Wasserstein-1 distance between P and G .

Original: For any h ∈H the following holds:
Revised: For any h ∈H the following holds:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst inequality results from the convexity of the exponential while the last one is a consequence of h being L -Lipschitz.
Revised: The ﬁrst inequality results from the convexity of the exponential while the last one is a consequence of h being L -Lipschitz.

Original: This allows to conclude that KALE ( P || G ) ≤ LW 1 ( P , G ) after taking the supremum over all h ∈H .
Revised: This allows to conclude that KALE ( P || G ) ≤ LW 1 ( P , G ) after taking the supremum over all h ∈H .

Original: Moreover, since W 1 metrizes the weak convergence on P 1 (Villani, 2009,
Revised: Moreover, since W 1 metrizes the weak convergence on P 1 (Villani, 2009,

Original: Theorem 6.9)
Revised: Theorem 6.9)

Original: , it holds that whenever a sequence G n converges weakly towards P in P 1 we also have
Revised: , it holds that whenever a sequence G n converges weakly towards P in P 1 we also have

Original: W 1 ( P , G n ) → 0 and thus KALE ( P || G n ) → 0 .
Revised: W 1 ( P , G n ) → 0 and thus KALE ( P || G n ) → 0 .

Original: The converse is a direct consequence of (Liu et al., 2017,
Revised: The converse is a direct consequence of (Liu et al., 2017,

Original: Theorem 10) since by assumption X is compact.
Revised: Theorem 10) since by assumption X is compact.

Original: Well-deﬁned learning.
Revised: Well-deﬁned learning.

Original: Assume that for any (cid:15) > 0 and any h and h (cid:48) in E there exists f in 2 E such that (cid:107)
Revised: Assume that for any (cid:15) > 0 and any h and h (cid:48) in E there exists f in 2 E such that (cid:107)

Original: h
Revised: h

Original: + h (cid:48) − f (cid:107) ∞ ≤ (cid:15) then there exists a constant C such that:
Revised: + h (cid:48) − f (cid:107) ∞ ≤ (cid:15) then there exists a constant C such that:

Original: [Equation]
Revised: [Equation]

Original: This means that the proposed learning procedure which ﬁrst ﬁnds the optimal energy E (cid:63) given a base G by maximum likelihood then minimizes KALE ( P , G ) ensures ends up minimizing the distance between the data end the generalized energy-based model Q .
Revised: This means that the proposed learning procedure which ﬁrst ﬁnds the optimal energy E (cid:63) given a base G by maximum likelihood then minimizes KALE ( P , G ) ensures ends up minimizing the distance between the data end the generalized energy-based model Q .

Original: [Equation]
Revised: [Equation]

Original: Let’s choose (cid:15) =
Revised: Let’s choose (cid:15) =

Original: KALE ( P , G ) and let h ∈ 2 E
Revised: KALE ( P , G ) and let h ∈ 2 E

Original: such that (cid:107) h
Revised: such that (cid:107) h

Original: + E (cid:63) − f (cid:107) ∞ ≤ (cid:15) .
Revised: + E (cid:63) − f (cid:107) ∞ ≤ (cid:15) .

Original: We have by concavity of the function ( α,β ) (cid:55)→L P , G ( α ( h + E (cid:63) − f )
Revised: We have by concavity of the function ( α,β ) (cid:55)→L P , G ( α ( h + E (cid:63) − f )

Original: + βf ) we have that:
Revised: + βf ) we have that:

Original: [Equation]
Revised: [Equation]

Original: By assumption, we have that (cid:107) h
Revised: By assumption, we have that (cid:107) h

Original: + E (cid:63) − f (cid:107) ∞ ≤ (cid:15) , thus |L P , G ( h + E (cid:63) − f ) |≤ 2 (cid:15) .
Revised: + E (cid:63) − f (cid:107) ∞ ≤ (cid:15) , thus |L P , G ( h + E (cid:63) − f ) |≤ 2 (cid:15) .

Original: Moreover, we have that L P , G ( 12 f ) ≤ KALE ( P , G ) since 12 f ∈E .
Revised: Moreover, we have that L P , G ( 12 f ) ≤ KALE ( P , G ) since 12 f ∈E .

Original: This ensures that:
Revised: This ensures that:

Original: [Equation]
Revised: [Equation]

Original: Finally, we have shown that:
Revised: Finally, we have shown that:

Original: [Equation]
Revised: [Equation]

Original: Hence, minimizing KALE ( P , G ) directly minimizes KALE ( P , Q ) .
Revised: Hence, minimizing KALE ( P , G ) directly minimizes KALE ( P , Q ) .

Original: C.2. S MOOTHNESS PROPERTIES OF KALE
Revised: C.2. S MOOTHNESS PROPERTIES OF KALE

Original: We will now prove Theorem 5.
Revised: We will now prove Theorem 5.

Original: We begin by stating the assumptions that will be used in this section:
Revised: We begin by stating the assumptions that will be used in this section:

Original: ( I ) E is parametrized by a compact set of parameters Ψ .
Revised: ( I ) E is parametrized by a compact set of parameters Ψ .

Original: ( II ) Functions in E are jointly continuous w.r
Revised: ( II ) Functions in E are jointly continuous w.r

Original: t. ( ψ,x ) and are L -lipschitz and L -smooth w.r.t.
Revised: t. ( ψ,x ) and are L -lipschitz and L -smooth w.r.t.

Original: the input x :
Revised: the input x :

Original: [Equation]
Revised: [Equation]

Original: ( III ) ( θ,z ) (cid:55)→ G θ ( z ) is jointly continuous in θ and z , with z (cid:55)→ G θ ( z ) uniformly Lipschitz w.r.t.
Revised: ( III ) ( θ,z ) (cid:55)→ G θ ( z ) is jointly continuous in θ and z , with z (cid:55)→ G θ ( z ) uniformly Lipschitz w.r.t.

Original: z :
Revised: z :

Original: [Equation]
Revised: [Equation]

Original: There exists non-negative functions a and b deﬁned from Z to R such that θ (cid:55)→ G θ ( z ) are a -Lipschitz
Revised: There exists non-negative functions a and b deﬁned from Z to R such that θ (cid:55)→ G θ ( z ) are a -Lipschitz

Original: and b -smooth in the following sense:
Revised: and b -smooth in the following sense:

Original: [Equation]
Revised: [Equation]

Original: Moreover, a and b are integrable in the following sense:
Revised: Moreover, a and b are integrable in the following sense:

Original: [Equation]
Revised: [Equation]

Original: To simplify notation, we will denote by L θ ( f ) the expected G θ log-likelihood under P .
Revised: To simplify notation, we will denote by L θ ( f ) the expected G θ log-likelihood under P .

Original: In other words,
Revised: In other words,

Original: [Equation]
Revised: [Equation]

Original: We also denote by p E,θ the density of the model w.r.t.
Revised: We also denote by p E,θ the density of the model w.r.t.

Original: G θ ,
Revised: G θ ,

Original: [Equation]
Revised: [Equation]

Original: We write K ( θ ):=
Revised: We write K ( θ ):=

Original: KALE ( P || G θ ) to emphasize the dependence on θ .
Revised: KALE ( P || G θ ) to emphasize the dependence on θ .

Original: Proof of Theorem 5.
Revised: Proof of Theorem 5.

Original: To show that sub-gradient methods converge to local optima, we only need to show that K is Lipschitz continuous and weakly convex.
Revised: To show that sub-gradient methods converge to local optima, we only need to show that K is Lipschitz continuous and weakly convex.

Original: This directly implies convergence to local optima for sub-gradient methods, according to Davis and Drusvyatskiy (2018);
Revised: This directly implies convergence to local optima for sub-gradient methods, according to Davis and Drusvyatskiy (2018);

Original: Thekumparampil et al.
Revised: Thekumparampil et al.

Original: Lipschitz continuity ensures that K is differentiable for almost all θ ∈ Θ , and weak convexity simply means that there exits some positive constant C ≥ 0 such that θ (cid:55)→
Revised: Lipschitz continuity ensures that K is differentiable for almost all θ ∈ Θ , and weak convexity simply means that there exits some positive constant C ≥ 0 such that θ (cid:55)→

Original: K
Revised: K

Original: ( θ )+ C (cid:107) θ (cid:107) 2 is convex.
Revised: ( θ )+ C (cid:107) θ (cid:107) 2 is convex.

Original: We now proceed to show these two properties.
Revised: We now proceed to show these two properties.

Original: We will ﬁrst prove that θ (cid:55)→K ( θ ) is weakly convex in θ .
Revised: We will ﬁrst prove that θ (cid:55)→K ( θ ) is weakly convex in θ .

Original: By Lemma 15, we know that for any E ∈E , the function θ (cid:55)→
Revised: By Lemma 15, we know that for any E ∈E , the function θ (cid:55)→

Original: L θ ( E ) is M -smooth for the same positive constant M .
Revised: L θ ( E ) is M -smooth for the same positive constant M .

Original: This directly implies that it is also weakly convex and the following inequality holds:
Revised: This directly implies that it is also weakly convex and the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: Taking the supremum w.r.t.
Revised: Taking the supremum w.r.t.

Original: E , it follows that
Revised: E , it follows that

Original: [Equation]
Revised: [Equation]

Original: This means precisely that K is weakly convex in θ .
Revised: This means precisely that K is weakly convex in θ .

Original: To prove that K is Lipschitz, we will also use Lemma 15, which states that L θ ( E ) is Lipschitz in θ uniformly on E .
Revised: To prove that K is Lipschitz, we will also use Lemma 15, which states that L θ ( E ) is Lipschitz in θ uniformly on E .

Original: Hence, the following holds:
Revised: Hence, the following holds:

Original: [Equation]
Revised: [Equation]

Original: Again, taking the supremum over E , it follows directly that
Revised: Again, taking the supremum over E , it follows directly that

Original: [Equation]
Revised: [Equation]

Original: We conclude that K is Lipschitz by exchanging the roles of θ and θ (cid:48) to get the other side of the inequality.
Revised: We conclude that K is Lipschitz by exchanging the roles of θ and θ (cid:48) to get the other side of the inequality.

Original: Hence, by the Rademacher theorem, K is differentiable for almost all θ .
Revised: Hence, by the Rademacher theorem, K is differentiable for almost all θ .

Original: We will now provide an expression for the gradient of K .
Revised: We will now provide an expression for the gradient of K .

Original: By Lemma 16 we know that ψ (cid:55)→L θ ( E ψ ) is continuous and by Assumption ( I ) Ψ is compact.
Revised: By Lemma 16 we know that ψ (cid:55)→L θ ( E ψ ) is continuous and by Assumption ( I ) Ψ is compact.

Original: Therefore, the supremum sup E ∈E L θ ( E ) is achieved for some function E (cid:63)θ .
Revised: Therefore, the supremum sup E ∈E L θ ( E ) is achieved for some function E (cid:63)θ .

Original: Moreover, we know by Lemma 15 that L θ ( E ) is smooth uniformly on E , therefore the family ( ∂ θ
Revised: Moreover, we know by Lemma 15 that L θ ( E ) is smooth uniformly on E , therefore the family ( ∂ θ

Original: L θ ( E ))
Revised: L θ ( E ))

Original: E ∈E is equi-differentiable.
Revised: E ∈E is equi-differentiable.

Original: We are in position to apply Milgrom and
Revised: We are in position to apply Milgrom and

Original: Segal (2002)(Theorem 3) which ensures that K ( θ ) admits left and right partial derivatives given by
Revised: Segal (2002)(Theorem 3) which ensures that K ( θ ) admits left and right partial derivatives given by

Original: [Equation]
Revised: [Equation]

Original: where e is a given direction in R r .
Revised: where e is a given direction in R r .

Original: Moreover, the theorem also states that K ( θ ) is differentiable iff t
Revised: Moreover, the theorem also states that K ( θ ) is differentiable iff t

Original: (cid:55)→ E (cid:63) θ + te is continuous at t =0 .
Revised: (cid:55)→ E (cid:63) θ + te is continuous at t =0 .

Original: Now, recalling that K ( θ ) is actually differentiable for almost all θ , it must hold that E (cid:63)θ + te → t → 0 E (cid:63)θ and ∂ + e K ( θ )=
Revised: Now, recalling that K ( θ ) is actually differentiable for almost all θ , it must hold that E (cid:63)θ + te → t → 0 E (cid:63)θ and ∂ + e K ( θ )=

Original: ∂ − e K
Revised: ∂ − e K

Original: ( θ ) for almost all θ .
Revised: ( θ ) for almost all θ .

Original: This implies that the two limits in (30) are actually equal to ∂ θ L θ ( E (cid:63)θ ) (cid:62) e .
Revised: This implies that the two limits in (30) are actually equal to ∂ θ L θ ( E (cid:63)θ ) (cid:62) e .

Original: The gradient of K , whenever deﬁned, in therefore given by
Revised: The gradient of K , whenever deﬁned, in therefore given by

Original: [Equation]
Revised: [Equation]

Original: Lemma 15.
Revised: Lemma 15.

Original: Under Assumptions ( I ) to ( III )
Revised: Under Assumptions ( I ) to ( III )

Original: [Equation]
Revised: [Equation]

Original: the functional L θ ( E ) is Lipschitz and smooth in θ uniformly on E :
Revised: the functional L θ ( E ) is Lipschitz and smooth in θ uniformly on E :

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Lemma
Revised: By Lemma

Original: [Equation]
Revised: [Equation]

Original: we have that L θ ( E ) is differentiable
Revised: we have that L θ ( E ) is differentiable

Original: [Equation]
Revised: [Equation]

Original: and that
Revised: and that

Original: [Equation]
Revised: [Equation]

Original: Lemma 16 ensures that (cid:107) ∂
Revised: Lemma 16 ensures that (cid:107) ∂

Original: θ L θ ( E ) (cid:107) is bounded by some positive constant C that is independent from
Revised: θ L θ ( E ) (cid:107) is bounded by some positive constant C that is independent from

Original: E and θ .
Revised: E and θ .

Original: This implies in particular that L θ ( E ) is Lipschitz with a constant C .
Revised: This implies in particular that L θ ( E ) is Lipschitz with a constant C .

Original: We will now show that it is also smooth.
Revised: We will now show that it is also smooth.

Original: For this, we need to control the difference
Revised: For this, we need to control the difference

Original: [Equation]
Revised: [Equation]

Original: We have by triangular inequality:
Revised: We have by triangular inequality:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term can be upper-bounded using L e -smoothness of E and the fact that G θ is Lipschitz in θ :
Revised: The ﬁrst term can be upper-bounded using L e -smoothness of E and the fact that G θ is Lipschitz in θ :

Original: [Equation]
Revised: [Equation]

Original: The last inequality was obtained by Lemma 17.
Revised: The last inequality was obtained by Lemma 17.

Original: Similarly, using that ∇ θ G θ is Lipschitz, it follows by Lemma 17 that
Revised: Similarly, using that ∇ θ G θ is Lipschitz, it follows by Lemma 17 that

Original: [Equation]
Revised: [Equation]

Original: Finally, for the last term III , we ﬁrst consider a path θ t =
Revised: Finally, for the last term III , we ﬁrst consider a path θ t =

Original: tθ +(1 − t ) θ (cid:48) for t ∈ [0 , 1] , and introduce the function s ( t ):=
Revised: tθ +(1 − t ) θ (cid:48) for t ∈ [0 , 1] , and introduce the function s ( t ):=

Original: p E,θ t ◦ G θ t .
Revised: p E,θ t ◦ G θ t .

Original: We will now control the difference p E,θ ◦ G θ − p E,θ (cid:48) ◦ G θ (cid:48) , also equal to s (1) − s (0) .
Revised: We will now control the difference p E,θ ◦ G θ − p E,θ (cid:48) ◦ G θ (cid:48) , also equal to s (1) − s (0) .

Original: Using the fact that s t is absolutely continuous we have that s (1) − s (0)= (cid:82) 1 0 s (cid:48)
Revised: Using the fact that s t is absolutely continuous we have that s (1) − s (0)= (cid:82) 1 0 s (cid:48)

Original: ( t ) dt .
Revised: ( t ) dt .

Original: The derivative s (cid:48) ( t ) is simply given by s (cid:48) ( t )=( θ − θ (cid:48) )
Revised: The derivative s (cid:48) ( t ) is simply given by s (cid:48) ( t )=( θ − θ (cid:48) )

Original: (cid:62) ( M t − ¯ M t ) s ( t ) where M t =( ∇ x E ◦ B θ t ) ∇ θ G θ t and ¯ M t = (cid:82) M t p E,θ t ◦ G θ t dη .
Revised: (cid:62) ( M t − ¯ M t ) s ( t ) where M t =( ∇ x E ◦ B θ t ) ∇ θ G θ t and ¯ M t = (cid:82) M t p E,θ t ◦ G θ t dη .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: We also know that M t is upper-bounded by La ( z ) , which implies
Revised: We also know that M t is upper-bounded by La ( z ) , which implies

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is obtained using Lemma 17.
Revised: where the last inequality is obtained using Lemma 17.

Original: This allows us to conclude that L θ ( E ) is smooth for any E ∈E and θ ∈ Θ .
Revised: This allows us to conclude that L θ ( E ) is smooth for any E ∈E and θ ∈ Θ .

Original: Lemma 16.
Revised: Lemma 16.

Original: Under Assumptions ( II ) and ( III ), it holds that ψ (cid:55)→
Revised: Under Assumptions ( II ) and ( III ), it holds that ψ (cid:55)→

Original: L θ ( E ψ ) is continuous, and that θ (cid:55)→L θ ( E ψ ) is differentiable in θ with gradient given by
Revised: L θ ( E ψ ) is continuous, and that θ (cid:55)→L θ ( E ψ ) is differentiable in θ with gradient given by

Original: [Equation]
Revised: [Equation]

Original: Moreover, the gradient is bounded uniformly in θ and E :
Revised: Moreover, the gradient is bounded uniformly in θ and E :

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: To show that ψ (cid:55)→L θ ( E ψ ) is continuous, we will use the dominated convergence theorem.
Revised: To show that ψ (cid:55)→L θ ( E ψ ) is continuous, we will use the dominated convergence theorem.

Original: We ﬁx ψ 0 in the interior of Ψ and consider a compact neighborhood W of ψ 0 .
Revised: We ﬁx ψ 0 in the interior of Ψ and consider a compact neighborhood W of ψ 0 .

Original: By assumption, we have that ( ψ,x ) (cid:55)→ E ψ ( x ) and ( ψ,z ) (cid:55)→ E ψ ( G θ ( z )) are jointly continuous.
Revised: By assumption, we have that ( ψ,x ) (cid:55)→ E ψ ( x ) and ( ψ,z ) (cid:55)→ E ψ ( G θ ( z )) are jointly continuous.

Original: Hence, | E ψ (0) | and | E ψ ( G θ (0)) | are bounded on W by some constant C .
Revised: Hence, | E ψ (0) | and | E ψ ( G θ (0)) | are bounded on W by some constant C .

Original: Moreover, by Lipschitz continuity of x (cid:55)→ E ψ , we have
Revised: Moreover, by Lipschitz continuity of x (cid:55)→ E ψ , we have

Original: [Equation]
Revised: [Equation]

Original: Recalling that P admits a ﬁrst order moment and that by Assumption ( III ), exp( L e L b (cid:107) z (cid:107) ) is integrable w.r.t.
Revised: Recalling that P admits a ﬁrst order moment and that by Assumption ( III ), exp( L e L b (cid:107) z (cid:107) ) is integrable w.r.t.

Original: η , it follows by the dominated convergence theorem and by composition of continuous functions that ψ (cid:55)→L θ ( E ψ ) is continuous in ψ 0 .
Revised: η , it follows by the dominated convergence theorem and by composition of continuous functions that ψ (cid:55)→L θ ( E ψ ) is continuous in ψ 0 .

Original: To show that θ (cid:55)→
Revised: To show that θ (cid:55)→

Original: L θ ( E ψ ) is differentiable in θ , we will use the differentiation lemma in (Klenke, 2008, Theorem 6.28).
Revised: L θ ( E ψ ) is differentiable in θ , we will use the differentiation lemma in (Klenke, 2008, Theorem 6.28).

Original: We ﬁrst ﬁx θ 0 in the interior of Θ , and consider a compact neighborhood V of θ 0 .
Revised: We ﬁrst ﬁx θ 0 in the interior of Θ , and consider a compact neighborhood V of θ 0 .

Original: Since θ (cid:55)→ | E ( G θ (0)) | is continuous on the compact neighborhood V
Revised: Since θ (cid:55)→ | E ( G θ (0)) | is continuous on the compact neighborhood V

Original: it admits a maximum value C ; hence we have using Assumptions ( II ) and ( III ) that
Revised: it admits a maximum value C ; hence we have using Assumptions ( II ) and ( III ) that

Original: [Equation]
Revised: [Equation]

Original: Along with the integrability assumption in Assumption ( III ), this ensures that z (cid:55)→ exp( − E ( G θ ( z ))) is integrable w.r.t η for all θ in V .
Revised: Along with the integrability assumption in Assumption ( III ), this ensures that z (cid:55)→ exp( − E ( G θ ( z ))) is integrable w.r.t η for all θ in V .

Original: We also have that exp( − E ( G θ ( z ))) is differentiable, with gradient given by
Revised: We also have that exp( − E ( G θ ( z ))) is differentiable, with gradient given by

Original: [Equation]
Revised: [Equation]

Original: Using that E is Lipschitz in its inputs and G θ ( z ) is Lipschitz in θ , and combining with the previous inequality, it follows that
Revised: Using that E is Lipschitz in its inputs and G θ ( z ) is Lipschitz in θ , and combining with the previous inequality, it follows that

Original: [Equation]
Revised: [Equation]

Original: where a ( z ) is the location dependent Lipschitz constant introduced in Assumption ( III ).
Revised: where a ( z ) is the location dependent Lipschitz constant introduced in Assumption ( III ).

Original: The r.h.s.
Revised: The r.h.s.

Original: of the above inequality is integrable by Assumption ( III ) and is independent of θ on the neighborhood
Revised: of the above inequality is integrable by Assumption ( III ) and is independent of θ on the neighborhood

Original: V .
Revised: V .

Original: Thus (Klenke, 2008, Theorem 6.28) applies, and it follows that
Revised: Thus (Klenke, 2008, Theorem 6.28) applies, and it follows that

Original: [Equation]
Revised: [Equation]

Original: We can now directly compute the gradient of L θ ( E ) ,
Revised: We can now directly compute the gradient of L θ ( E ) ,

Original: [Equation]
Revised: [Equation]

Original: Since E and G θ are Lipschitz in x and θ respectively, it follows that (
Revised: Since E and G θ are Lipschitz in x and θ respectively, it follows that (

Original: cid:107)∇ x E ( G θ 0 ( z )) (cid:107) ≤ L e and (cid:107)∇ θ G θ 

Revised: cid:107)∇ x E ( G θ 0 ( z )) (cid:107) ≤ L e and (cid:107)∇ θ G θ

Original: ( z ) (cid:107)≤ a ( z ) .
Revised: ( z ) (cid:107)≤ a ( z ) .

Original: Hence, we have
Revised: Hence, we have

Original: [Equation]
Revised: [Equation]

Original: Finally, Lemma 17 allows us to conclude that (cid:107)∇ θ L θ ( E ) (cid:107) is bounded by a positive constant C independently from θ and E .
Revised: Finally, Lemma 17 allows us to conclude that (cid:107)∇ θ L θ ( E ) (cid:107) is bounded by a positive constant C independently from θ and E .

Original: Lemma 17.
Revised: Lemma 17.

Original: Under Assumptions ( II ) and ( III ), there exists a constant C independent from θ and E such that
Revised: Under Assumptions ( II ) and ( III ), there exists a constant C independent from θ and E such that

Original: [Equation]
Revised: [Equation]

Original: for i ∈ 1 , 2 .
Revised: for i ∈ 1 , 2 .

Original: Proof.
Revised: Proof.

Original: By Lipschitzness of E and G θ , we have exp( − L e L b (cid:107) z (cid:107) )
Revised: By Lipschitzness of E and G θ , we have exp( − L e L b (cid:107) z (cid:107) )

Original: ≤ exp( E ( G θ (0))
Revised: ≤ exp( E ( G θ (0))

Original: − E ( G θ ( z )) ≤
Revised: − E ( G θ ( z )) ≤

Original: exp( L e L b (cid:107) z (cid:107) ) , thus introducing the factor exp( E ( B θ 0 (0)) in (31) we get (cid:90) a i ( z
Revised: exp( L e L b (cid:107) z (cid:107) ) , thus introducing the factor exp( E ( B θ 0 (0)) in (31) we get (cid:90) a i ( z

Original: [Equation]
Revised: [Equation]

Original: ( p E
Revised: ( p E

Original: [Equation]
Revised: [Equation]

Original: ◦ G
Revised: ◦ G

Original: [Equation]
Revised: [Equation]

Original: ( z
Revised: ( z

Original: [Equation]
Revised: [Equation]

Original: ) dη (
Revised: ) dη (

Original: [Equation]
Revised: [Equation]

Original: exp
Revised: exp

Original: [Equation]
Revised: [Equation]

Original: (cid:107)
Revised: (cid:107)

Original: [Equation]
Revised: [Equation]

Original: (cid:107) )
Revised: (cid:107) )

Original: d
Revised: d

Original: [Equation]
Revised: [Equation]

Original: ã −
Revised: ã −

Original: [Equation]
Revised: [Equation]

Original: a (
Revised: a (

Original: [Equation]
Revised: [Equation]

Original: i exp( L
Revised: i exp( L

Original: [Equation]
Revised: [Equation]

Original: L b (cid:107) z (cid:107) )
Revised: L b (cid:107) z (cid:107) )

Original: d η ( z ) , (cid:90)
Revised: d η ( z ) , (cid:90)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ( p
Revised: ( p

Original: [Equation]
Revised: [Equation]

Original: ◦ G
Revised: ◦ G

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: dη
Revised: dη

Original: [Equation]
Revised: [Equation]

Original: d
Revised: d

Original: [Equation]
Revised: [Equation]

Original: ã −
Revised: ã −

Original: [Equation]
Revised: [Equation]

Original: The r.h.s.
Revised: The r.h.s.

Original: of both inequalities is independent of θ and E, and ﬁnite by the integrability assumptions in Assumption ( III ).
Revised: of both inequalities is independent of θ and E, and ﬁnite by the integrability assumptions in Assumption ( III ).

Original: [Table]
Revised: [Table]

Original: D I MAGE G ENERATION
Revised: D I MAGE G ENERATION

Original: Figures 3 and 4 show sample trajectories using Algorithm 3 with no friction γ =0 for the 4 datasets.
Revised: Figures 3 and 4 show sample trajectories using Algorithm 3 with no friction γ =0 for the 4 datasets.

Original: It is clear that along the same MCMC chain, several image modes are explored.
Revised: It is clear that along the same MCMC chain, several image modes are explored.

Original: We also notice the transition from a mode to another happens almost at the same time for all chains and corresponds to the gray images.
Revised: We also notice the transition from a mode to another happens almost at the same time for all chains and corresponds to the gray images.

Original: This is unlike Langevin or when the friction coefﬁcient γ is large as in Figure 5.
Revised: This is unlike Langevin or when the friction coefﬁcient γ is large as in Figure 5.

Original: In that case each chain remains within the same mode.
Revised: In that case each chain remains within the same mode.

Original: Table 4 shows further comparisons with other methods on Cifar10 and ImageNet 32x32.
Revised: Table 4 shows further comparisons with other methods on Cifar10 and ImageNet 32x32.

Original: E D ENSITY E STIMATION
Revised: E D ENSITY E STIMATION

Original: Figure Figure 7 (left) shows the error in the estimation of the log-partition function using both methods (KALE-DV and KALE-F).
Revised: Figure Figure 7 (left) shows the error in the estimation of the log-partition function using both methods (KALE-DV and KALE-F).

Original: KALE-DV estimates the negative log-likelihood on each batch of size 100 and therefore has much more variance than KALE-F which maintains the amortized estimator of the log-partition function.
Revised: KALE-DV estimates the negative log-likelihood on each batch of size 100 and therefore has much more variance than KALE-F which maintains the amortized estimator of the log-partition function.

Original: Figure Figure 7 (right) shows the evolution of the negative log-likelihood (NLL) on both training and test sets per epochs for RedWine and Whitewine datasets.
Revised: Figure Figure 7 (right) shows the evolution of the negative log-likelihood (NLL) on both training and test sets per epochs for RedWine and Whitewine datasets.

Original: The error decreases steadily in the case of KALE-DV and KALE-F while the error gap between the training and test set remains controlled.
Revised: The error decreases steadily in the case of KALE-DV and KALE-F while the error gap between the training and test set remains controlled.

Original: Larger gaps are observed for both direct maximum likelihood estimation and Contrastive divergence although the training NLL tends to decrease faster than for KALE.
Revised: Larger gaps are observed for both direct maximum likelihood estimation and Contrastive divergence although the training NLL tends to decrease faster than for KALE.

Original: F A LGORITHMS
Revised: F A LGORITHMS

Original: Estimating the variational parameter.
Revised: Estimating the variational parameter.

Original: Optimizing (9) exactly over A yields (8), with the optimal
Revised: Optimizing (9) exactly over A yields (8), with the optimal

Original: A equal to ˜ A = log( 1 M (cid:80)
Revised: A equal to ˜ A = log( 1 M (cid:80)

Original: Mm =1 exp( − E ( Y m ))) .
Revised: Mm =1 exp( − E ( Y m ))) .

Original: However, to maintain an amortized estimator of the log-partition we propose to optimize (9) iteratively using second order updates:
Revised: However, to maintain an amortized estimator of the log-partition we propose to optimize (9) iteratively using second order updates:

Original: [Equation]
Revised: [Equation]

Original: where λ is a learning rate and ˜
Revised: where λ is a learning rate and ˜

Original: A k +1 is the empirical log-partition function estimated from a batch of new samples.
Revised: A k +1 is the empirical log-partition function estimated from a batch of new samples.

Original: By leveraging updates from previous iterations, A can yield much more accurate estimates of the log-partition function as conﬁrmed empirically in Figure 7 of Appendix E.
Revised: By leveraging updates from previous iterations, A can yield much more accurate estimates of the log-partition function as conﬁrmed empirically in Figure 7 of Appendix E.

Original: Tempered GEBM.
Revised: Tempered GEBM.

Original: It can be preferable to sample from a tempered version of the model by rescaling the energy E by an inverse temperature parameter β , thus effectively sampling from Q .
Revised: It can be preferable to sample from a tempered version of the model by rescaling the energy E by an inverse temperature parameter β , thus effectively sampling from Q .

Original: High temperature regimes ( β → 0 ) recover the base model G while low temperature regimes ( β →∞ ) essentially sample from minima of the energy E .
Revised: High temperature regimes ( β → 0 ) recover the base model G while low temperature regimes ( β →∞ ) essentially sample from minima of the energy E .

Original: As shown in Section 6, low temperatures tend to produce better sample quality for natural image generation tasks.
Revised: As shown in Section 6, low temperatures tend to produce better sample quality for natural image generation tasks.

Original: Training In Algorithm 1, we describe the general algorithm for training a GEBM which alternates between gradient steps on the energy and the generator.
Revised: Training In Algorithm 1, we describe the general algorithm for training a GEBM which alternates between gradient steps on the energy and the generator.

Original: An additional regularization, denoted by I ( ψ ) is used to ensure conditions of Proposition 4 and Theorem 5 hold.
Revised: An additional regularization, denoted by I ( ψ ) is used to ensure conditions of Proposition 4 and Theorem 5 hold.

Original: I ( ψ ) can include L 2 regularization over the parameters ψ , a gradient penalty as in Gulrajani et al.
Revised: I ( ψ ) can include L 2 regularization over the parameters ψ , a gradient penalty as in Gulrajani et al.

Original: (2017) or Spectral normalization Miyato et al.
Revised: (2017) or Spectral normalization Miyato et al.

Original: The energy can be trained either using the estimator in (8) (KALE-DV) or the one in (9) (KALE-F) depending on the variable C .
Revised: The energy can be trained either using the estimator in (8) (KALE-DV) or the one in (9) (KALE-F) depending on the variable C .

Original: Sampling In Algorithm 3, we describe the MCMC sampler proposed in Sachs et al.
Revised: Sampling In Algorithm 3, we describe the MCMC sampler proposed in Sachs et al.

Original: (2017) which is a time discretization of (14).
Revised: (2017) which is a time discretization of (14).

Original: Algorithm
Revised: Algorithm

Original: Overdamped Langevin Algorithm
Revised: Overdamped Langevin Algorithm

Original: Algorithm 3 Kinetic Langevin Algorithm
Revised: Algorithm 3 Kinetic Langevin Algorithm

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: G E XPERIMENTAL DETAILS
Revised: G E XPERIMENTAL DETAILS

Original: In all experiments, we use regularization which is a combination of L 2 norm and a variant of the gradient penalty Gulrajani et al.
Revised: In all experiments, we use regularization which is a combination of L 2 norm and a variant of the gradient penalty Gulrajani et al.

Original: For the image generation tasks, we also employ spectral normalization Miyato et al.
Revised: For the image generation tasks, we also employ spectral normalization Miyato et al.

Original: This is to ensure that the conditions in Proposition 4 and Theoremhold.
Revised: This is to ensure that the conditions in Proposition 4 and Theoremhold.

Original: We pre-condition the gradient as proposed in Simsekli et al.
Revised: We pre-condition the gradient as proposed in Simsekli et al.

Original: (2020) to stabilize training, and to avoid taking large noisy gradient steps due to the exponential terms in (8) and (9).
Revised: (2020) to stabilize training, and to avoid taking large noisy gradient steps due to the exponential terms in (8) and (9).

Original: We also use the second-order updates in (32) for the variational constant c whenever it is learned.
Revised: We also use the second-order updates in (32) for the variational constant c whenever it is learned.

Original: G. I MAGE GENERATION
Revised: G. I MAGE GENERATION

Original: z ∈
Revised: z ∈

Original: [Table]
Revised: [Table]

Original: ∼N
Revised: ∼N

Original: [Table]
Revised: [Table]

Original: ,I ) dense → M g × M g ×
Revised: ,I ) dense → M g × M g ×

Original: [Table]
Revised: [Table]

Original: 4 × 4 , stride =2 deconv.
Revised: 4 × 4 , stride =2 deconv.

Original: BN 256 ReLU 4 × 4 , stride =2 deconv. BN 128 ReLU 4 × 4 , stride =2 deconv.
Revised: BN 256 ReLU 4 × 4 , stride =2 deconv. BN 128 ReLU 4 × 4 , stride =2 deconv.

Original: BN 64 ReLU 3 × 3 , stride =1 conv.
Revised: BN 64 ReLU 3 × 3 , stride =1 conv.

Original: 3 Tanh
Revised: 3 Tanh

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Network Architecture Table 5 and Table 6 show the network architectures used for the GEBM in the case of SNGAN ConvNet.
Revised: Network Architecture Table 5 and Table 6 show the network architectures used for the GEBM in the case of SNGAN ConvNet.

Original: Table 5 and Table 6 show the network architectures used for the GEBM in the case of SNGAN ResNet.
Revised: Table 5 and Table 6 show the network architectures used for the GEBM in the case of SNGAN ResNet.

Original: The residual connections of each residual block consists of two convolutional layers proceeded by a BatchNormalization and ReLU activation:
Revised: The residual connections of each residual block consists of two convolutional layers proceeded by a BatchNormalization and ReLU activation:

Original: BN+ReLU+Conv+BN+ReLU+Conv as in (Miyato et al., 2018, Figure 8).
Revised: BN+ReLU+Conv+BN+ReLU+Conv as in (Miyato et al., 2018, Figure 8).

Original: Training: We train both base and energy by alternating 5 gradient steps to learn the energy vs 1 gradient step to learn the base.
Revised: Training: We train both base and energy by alternating 5 gradient steps to learn the energy vs 1 gradient step to learn the base.

Original: For the ﬁrst two gradient iterations and after every 500 gradient iterations on base, we train the energy for 100 gradient steps instead of 5 .
Revised: For the ﬁrst two gradient iterations and after every 500 gradient iterations on base, we train the energy for 100 gradient steps instead of 5 .

Original: We then train the model up to 150000 gradient iterations on the base using a batch-size of 128 and Adam optimizer with initial learning rate of 10 − 4 and parameters (0 . 5 ,. 999) for both energy and base.
Revised: We then train the model up to 150000 gradient iterations on the base using a batch-size of 128 and Adam optimizer with initial learning rate of 10 − 4 and parameters (0 . 5 ,. 999) for both energy and base.

Original: Scheduler: We decrease the learning rate using a scheduler that monitors the FID score in a similar way as in Bi ´ nkowski et al.
Revised: Scheduler: We decrease the learning rate using a scheduler that monitors the FID score in a similar way as in Bi ´ nkowski et al.

Original: Arbel et al.
Revised: Arbel et al.

Original: More precisely, every 2000 gradient iterations on the base, we evaluate the FID score on the training set using 50000 generated samples from the base and check if the current score is larger than the score 20000 iterations before.
Revised: More precisely, every 2000 gradient iterations on the base, we evaluate the FID score on the training set using 50000 generated samples from the base and check if the current score is larger than the score 20000 iterations before.

Original: The learning rate is decreased by a factor of 0 . 8 if the FID score fails to decrease for 3 consecutive times.
Revised: The learning rate is decreased by a factor of 0 . 8 if the FID score fails to decrease for 3 consecutive times.

Original: Sampling:
Revised: Sampling:

Original: For (DOT) Tanaka (2019), we use the following objective:
Revised: For (DOT) Tanaka (2019), we use the following objective:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where z y is sampled from a standard Gaussian, (cid:15) is a perturbation meant to stabilize sampling and k eff is the estimated Lipschitz constant of E ◦ B .
Revised: where z y is sampled from a standard Gaussian, (cid:15) is a perturbation meant to stabilize sampling and k eff is the estimated Lipschitz constant of E ◦ B .

Original: Note that (33) uses a ﬂipped sign for the E ◦ B compared to Tanaka (2019).
Revised: Note that (33) uses a ﬂipped sign for the E ◦ B compared to Tanaka (2019).

Original: This is because E plays the role of − D where D is the discriminator in
Revised: This is because E plays the role of − D where D is the discriminator in

Original: Tanaka (2019).
Revised: Tanaka (2019).

Original: Introducing the minus sign in (33) leads to a degradation in performance.
Revised: Introducing the minus sign in (33) leads to a degradation in performance.

Original: We perform 1000 gradient iterations with a step-size of 0 . 0001 which is also decreased by a factor of 10 everyiterations as done for the proposed method.
Revised: We perform 1000 gradient iterations with a step-size of 0 . 0001 which is also decreased by a factor of 10 everyiterations as done for the proposed method.

Original: As suggested by the authors of Tanaka (2019) we perform the following projection for the gradient before applying it:
Revised: As suggested by the authors of Tanaka (2019) we perform the following projection for the gradient before applying it:

Original: [Equation]
Revised: [Equation]

Original: We set the perturbation (cid:15) to 0 .
Revised: We set the perturbation (cid:15) to 0 .

Original: 001 and k eff to 1 which was also shown in Tanaka (2019) to perform well.
Revised: 001 and k eff to 1 which was also shown in Tanaka (2019) to perform well.

Original: In fact, we found that estimating the Lipschitz constant by taking the maximum value of (cid:107)∇ E ◦ G ( z ) (cid:107) over 1000 latent samples according to η lead to higher values for k eff :
Revised: In fact, we found that estimating the Lipschitz constant by taking the maximum value of (cid:107)∇ E ◦ G ( z ) (cid:107) over 1000 latent samples according to η lead to higher values for k eff :

Original: ( Cifar10: 9 . 4 , CelebA : 7 . 2 ,
Revised: ( Cifar10: 9 . 4 , CelebA : 7 . 2 ,

Original: ImageNet: 4 . 9 , Lsun: 3 . 8 ).
Revised: ImageNet: 4 . 9 , Lsun: 3 . 8 ).

Original: However, those higher values did not perform as well as setting k eff =1 .
Revised: However, those higher values did not perform as well as setting k eff =1 .

Original: For (IHM) Turner et al.
Revised: For (IHM) Turner et al.

Original: we simply run the MCMC chain for 1000 iterations.
Revised: we simply run the MCMC chain for 1000 iterations.

Original: G. D ENSITY ESTIMATION
Revised: G. D ENSITY ESTIMATION

Original: Pre-processing We use code and pre-processing steps from Wenliang et al.
Revised: Pre-processing We use code and pre-processing steps from Wenliang et al.

Original: (2019) which we describe here for completeness.
Revised: (2019) which we describe here for completeness.

Original: For RedWine and WhiteWine, we added uniform noise with support equal to the median distances between two adjacent values.
Revised: For RedWine and WhiteWine, we added uniform noise with support equal to the median distances between two adjacent values.

Original: That is to avoid instabilities due to the quantization of the datasets.
Revised: That is to avoid instabilities due to the quantization of the datasets.

Original: For Hepmass and MiniBoone, we removed ill-conditioned dimensions as also done in Papamakarios et al.
Revised: For Hepmass and MiniBoone, we removed ill-conditioned dimensions as also done in Papamakarios et al.

Original: We split all datasets, except HepMass into three splits.
Revised: We split all datasets, except HepMass into three splits.

Original: The test split consists of 10% of the total data.
Revised: The test split consists of 10% of the total data.

Original: For the validation set, we use 10% of the remaining data with an upper limit of 1000 to reduce the cost of validation at each iteration.
Revised: For the validation set, we use 10% of the remaining data with an upper limit of 1000 to reduce the cost of validation at each iteration.

Original: For HepMass, we used the sample splitting as done in Papamakarios et al.
Revised: For HepMass, we used the sample splitting as done in Papamakarios et al.

Original: Finally, the data is whitened before ﬁtting and the whitening matrix was computed on at most 10000 data points.
Revised: Finally, the data is whitened before ﬁtting and the whitening matrix was computed on at most 10000 data points.

Original: Regularization: We set the regularization parameter to 0 .
Revised: Regularization: We set the regularization parameter to 0 .

Original: 1 and use a combination of L 2 norm and a variant of the gradient penalty Gulrajani et al.
Revised: 1 and use a combination of L 2 norm and a variant of the gradient penalty Gulrajani et al.

Original: [Equation]
Revised: [Equation]

Original: Network Architecture.
Revised: Network Architecture.

Original: For both base and energy, we used an NVP Dinh et al.
Revised: For both base and energy, we used an NVP Dinh et al.

Original: with 5 NVP layers each consisting of a shifting and scaling layer with two hidden layers of 100 neurons.
Revised: with 5 NVP layers each consisting of a shifting and scaling layer with two hidden layers of 100 neurons.

Original: We do not use Batch-normalization.
Revised: We do not use Batch-normalization.

Original: Training:
Revised: Training:

Original: In all cases we use Adam optimizer with learning rate of 0 .
Revised: In all cases we use Adam optimizer with learning rate of 0 .

Original: 001 and momentum parameters (0 . 5 , 0 . 9) .
Revised: 001 and momentum parameters (0 . 5 , 0 . 9) .

Original: For both KALE-DV and KALE-F, we used a batch-size of 100 data samples vsgenerated samples from the base in order to reduce the variance of the estimation of the energy.
Revised: For both KALE-DV and KALE-F, we used a batch-size of 100 data samples vsgenerated samples from the base in order to reduce the variance of the estimation of the energy.

Original: We alternate 50 gradient steps on the energy vs 1 step on the base and further perform 50 additional steps on the energy for the ﬁrst two gradient iterations and after every 500 gradient iterations on base.
Revised: We alternate 50 gradient steps on the energy vs 1 step on the base and further perform 50 additional steps on the energy for the ﬁrst two gradient iterations and after every 500 gradient iterations on base.

Original: For
Revised: For

Original: Contrastive divergence, each training step is performed by ﬁrst producing 100 samples from the model using 100 Langevin iterations with a step-size of 10 − 2 and starting from a batch of 100 data-samples.
Revised: Contrastive divergence, each training step is performed by ﬁrst producing 100 samples from the model using 100 Langevin iterations with a step-size of 10 − 2 and starting from a batch of 100 data-samples.

Original: The resulting samples are then used to estimate the gradient of the of the loss.
Revised: The resulting samples are then used to estimate the gradient of the of the loss.

Original: For (CD), we used 100 Langevin iterations for each learning step to sample from the EBM.
Revised: For (CD), we used 100 Langevin iterations for each learning step to sample from the EBM.

Original: This translates into an improved performance at the expense of increased computational cost compared to the other methods.
Revised: This translates into an improved performance at the expense of increased computational cost compared to the other methods.

Original: All methods are trained for 2000 epochs with batch-size of 100 (1000 on Hepmass and Miniboone datasets) and ﬁxed learning rate 0 .
Revised: All methods are trained for 2000 epochs with batch-size of 100 (1000 on Hepmass and Miniboone datasets) and ﬁxed learning rate 0 .

Original: 001 , which was sufﬁcient for convergence.
Revised: 001 , which was sufﬁcient for convergence.

Original: G. I LLUSTRATIVE EXAMPLE IN F IGURE
Revised: G. I LLUSTRATIVE EXAMPLE IN F IGURE

Original: We consider parametric functions G (1) θ and G (2) θ from R to R of the form:
Revised: We consider parametric functions G (1) θ and G (2) θ from R to R of the form:

Original: [Equation]
Revised: [Equation]

Original: with θ =( W,B,W (cid:48) ,b ) .
Revised: with θ =( W,B,W (cid:48) ,b ) .

Original: we also call θ (cid:63) =(1 , 1 , 1 , 0) .
Revised: we also call θ (cid:63) =(1 , 1 , 1 , 0) .

Original: In addition, we consider a sigmoid like function h from [0 , 1] to [0 , 1] of the form:
Revised: In addition, we consider a sigmoid like function h from [0 , 1] to [0 , 1] of the form:

Original: [Equation]
Revised: [Equation]

Original: Data generation : To generate a data point X =
Revised: Data generation : To generate a data point X =

Original: ( X 1 ,X 2 ) , we consider the following simple generative model:
Revised: ( X 1 ,X 2 ) , we consider the following simple generative model:

Original: • Sample a uniform r.v.
Revised: • Sample a uniform r.v.

Original: Z from [0 , 1] .
Revised: Z from [0 , 1] .

Original: • Apply the distortion function h to get a latent sample Y
Revised: • Apply the distortion function h to get a latent sample Y

Original: = h ( Z ) .
Revised: = h ( Z ) .

Original: • Generate point X using X 1 = G (1) θ (cid:63) ( Y ) and X 2 = G (2) θ (cid:63) ( Y ) .
Revised: • Generate point X using X 1 = G (1) θ (cid:63) ( Y ) and X 2 = G (2) θ (cid:63) ( Y ) .

Original: Hence, the data are supported on the 1 -d line deﬁned by the equation X 2 = G (2) θ (cid:63) ( X 1 ) .
Revised: Hence, the data are supported on the 1 -d line deﬁned by the equation X 2 = G (2) θ (cid:63) ( X 1 ) .

Original: GAN For the generator we sample Z uniformly from [0 , 1] then generate a sample ( X 1 ,X 2 )=(
Revised: GAN For the generator we sample Z uniformly from [0 , 1] then generate a sample ( X 1 ,X 2 )=(

Original: G (1) θ ( Z ) ,G (2) θ ( Z )) .
Revised: G (1) θ ( Z ) ,G (2) θ ( Z )) .

Original: The goal is to learn θ .
Revised: The goal is to learn θ .

Original: For the discriminator, we used an MLP with 6 layers and 10 hidden units.
Revised: For the discriminator, we used an MLP with 6 layers and 10 hidden units.

Original: GEBM
Revised: GEBM

Original: For the base we use the same generator as in the GAN model.
Revised: For the base we use the same generator as in the GAN model.

Original: For the energy we use the same MLP as discriminator of the GAN model.
Revised: For the energy we use the same MLP as discriminator of the GAN model.

Original: EBM
Revised: EBM

Original: To ensure tractability of the likelihood, we use the following model:
Revised: To ensure tractability of the likelihood, we use the following model:

Original: [Equation]
Revised: [Equation]

Original: MoG (( µ 1 ,σ 1 ) , ( µ 2 ,σ 2 )) refers to a Mixture of two gaussians with mean and variances µ i and σ i .
Revised: MoG (( µ 1 ,σ 1 ) , ( µ 2 ,σ 2 )) refers to a Mixture of two gaussians with mean and variances µ i and σ i .

Original: We learn each of the parameters ( θ,σ 0 ,µ 1 ,σ 1 ,µ 2 ,σ 2 ) by maximizing the likelihood.
Revised: We learn each of the parameters ( θ,σ 0 ,µ 1 ,σ 1 ,µ 2 ,σ 2 ) by maximizing the likelihood.

Original: Both GAN and GEBM have the capacity to recover the the exact support by ﬁnding the optimal parameter θ (cid:63) .
Revised: Both GAN and GEBM have the capacity to recover the the exact support by ﬁnding the optimal parameter θ (cid:63) .

Original: For the EBM, when θ = θ (cid:63) , the mean G θ (cid:63) ( X 1 ) of the conditional gaussian X 2 | Xdraws a line which matches the data support exactly, i.e.: X 2 =
Revised: For the EBM, when θ = θ (cid:63) , the mean G θ (cid:63) ( X 1 ) of the conditional gaussian X 2 | Xdraws a line which matches the data support exactly, i.e.: X 2 =

Original: G (2) θ (cid:63) ( X 1 ) .
Revised: G (2) θ (cid:63) ( X 1 ) .

Original: G. B ASE /G ENERATOR COMPLEXITY
Revised: G. B ASE /G ENERATOR COMPLEXITY

Original: To investigate the effect of model complexity of the performance gap between GANs and GEBMs, we performed additional experiments using the setting of Figure 1.
Revised: To investigate the effect of model complexity of the performance gap between GANs and GEBMs, we performed additional experiments using the setting of Figure 1.

Original: Now we allow the generator/base network to better model the hidden transformation h that produces the ﬁrst coordinate X 1 given the latent noise.
Revised: Now we allow the generator/base network to better model the hidden transformation h that produces the ﬁrst coordinate X 1 given the latent noise.

Original: We choose G (1) θ to be  a one hidden layer network  with leaky ReLU activation, instead of a simple linear transform as previously done in Appendix G.3.
Revised: We choose G (1) θ to be either a one hidden layer network or an MLP with 3 hidden layers both with leaky ReLU activation, instead of a simple linear transform as previously done in Appendix G.3.

Original: The network has universal approximation capability that depends on the number of units.
Revised: The network has universal approximation capability that depends on the number of units.

Original: This provides a direct control over the complexity of the generator/base.
Revised: This provides a direct control over the complexity of the generator/base.

Revised: We then varied the number of hidden units from 1 to 5 ∗ 10 4 units for the one hidden layer network and from 10 to 5 ∗ 10 3 units per layer for the MLP.

Revised: Note that the MLP with 5 ∗ 10 3 units per layer stores a matrix of size 2 .

Original: We then varied the number of hidden units from 1 to 5 ∗ 10 4 units and compared the performance of the GAN and GEBM using the Sinkhorn divergence Feydy et al.
Revised: 5 ∗ 10and thus contains 2 orders of magnitudes more parameters than the widest shallow network with 5 ∗ 10units. We then compared the performance of the GAN and GEBM using the Sinkhorn divergence Feydy et al.

Original: (2019) between each model and the data distribution.
Revised: (2019) between each model and the data distribution.

Original: In all experiments, we used the same discriminator/energy network described in Appendix G.3.
Revised: In all experiments, we used the same discriminator/energy network described in Appendix G.3.

Original: Results are provided in Figure 8.
Revised: Results are provided in Figure 8.

Original: Estimating the Sinkhorn divergence.
Revised: Estimating the Sinkhorn divergence.

Original: The Sinkhorn is computed using 6000 samples from the data and the model, with squared euclidean distance as a ground cost and using a regularization (cid:15)
Revised: The Sinkhorn is computed using 6000 samples from the data and the model, with squared euclidean distance as a ground cost and using a regularization (cid:15)

Original: =1 e − 3 .
Revised: =1 e − 3 .

Original: We then repeat the procedure 5 times and average the result to get the ﬁnal estimate of the Sinkhorn distance for a given run.
Revised: We then repeat the procedure 5 times and average the result to get the ﬁnal estimate of the Sinkhorn distance for a given run.

Revised: Training.

Revised: Each run optimizes the parameters of the model using Adam optimizer ( β 1 = .

Revised: 5 ,β 2 = . 99) , learning rate lr =1 e − 4 for the energy/discriminator and lr =1 e − 5 for the base/generator and weight decay of 1 e − 2 for the base/generator.

Revised: Training is performed using KALE for 2000 epochs using a batch size of 5000 and 10 gradient iterations for the energy/discriminator per base/generator iteration.

Original: We use the gradient penalty for the energy discriminator with a penalty parameter of 
Revised: We use the gradient penalty for the energy/discriminator with a penalty parameter of 0 .

Original: We then perform early stopping and retain the best performing model on a validation set.
Revised: We then perform early stopping and retain the best performing model on a validation set.

Original: Observations We make the following observations from Figure 8: the GAN generator indeed improves when we increase the number of hidden units.
Revised: Observations We make the following observations from Figure 8: the GAN generator indeed improves when we increase the number of hidden units.

Original: The performance of the GEBM remains stable as the number of hidden units increases.
Revised: The performance of the GEBM remains stable as the number of hidden units increases.

Original: The performance of the GEBM is always better than the GAN, although we can see the GAN converging towards the GEBM.
Revised: The performance of the GEBM is always better than the GAN, although we can see the GAN converging towards the GEBM.

Original: GEBM with a simpler base already outperforms the GAN with more powerful generators.
Revised: GEBM with a simpler base already outperforms the GAN with more powerful generators.

Original: The gap between the GEBM and the GAN reduces as the GAN becomes more expressive.
Revised: The gap between the GEBM and the GAN reduces as the GAN becomes more expressive.

Revised: Using a deeper network further reduces the gap compared to a shallow network.