Title: Energy-based Generative Adversarial Networks
Authors: Junbo Zhao
Michael Mathieu
Yann LeCun
Venue: ICLR 2017
Keywords: Deep learning
Unsupervised Learning
Semi-Supervised Learning

Abstract: We introduce the "Energy-based Generative Adversarial Network" model (EBGAN) which views the discriminator as an energy function that attributes low energies to the regions near the data manifold and higher energies to other regions. Similar to the probabilistic GANs, a generator is seen as being trained to produce contrastive samples with minimal energies, while the discriminator is trained to assign high energies to these generated samples. Viewing the discriminator as an energy function allows to use a wide variety of architectures and loss functionals in addition to the usual binary classifier with logistic output. Among them, we show one instantiation of EBGAN framework as using an auto-encoder architecture, with the energy being the reconstruction error, in place of the discriminator. We show that this form of EBGAN exhibits more stable behavior than regular GANs during training. We also show that a single-scale architecture can be trained to generate high-resolution images.

TL;DR: We introduce the "Energy-based Generative Adversarial Network" (EBGAN) model.

Review discussion:
* All experiments are based on the autoencoder formulation for D. Did you try directly predicting the energy e.g. as the output of a convnet?
* The autoencoder formulation of D permits it to be pre-trained on x alone (i.e. without G). Do you think there would be any benefit to doing so, or is it important that G and D be trained together from the start?
Thanks for the comments.
* Yes, we tried and managed to achieve several implementations of D including this one. We only present auto-encoder as an example implementation of D in EBGAN in the paper.
* We have tried pre-training D using real samples and then starting the normal adversarial training in a few settings (different architectures and hyperparameters) on MNIST. It works equally well as starting training D and G together from the beginning. It’s just that freezing D after being pre-trained doesn’t work, as explained in the reply to the AnonReviewer2.
Use encoder-decoder model as D is an interesting and reasonable idea. It looks like transfer knowledge in encoder decoder to G. What is the result of use pretrained encoder-decoder, then freeze D while generating gradient to G?
Thanks for the comments.
We tried to experiment on the paradigm to pretrain the discriminator by just using real samples, and freeze D, then train G alone. Unfortunately, due to the fact that such paradigm separates training D and G, it could not achieve the goal of adversarial training, i.e., finding a Nash Equilibrium of system (D, G). The generated samples did not look anything like real samples.
For one thing, on the starting stage of training G, where the produced fake samples $G(z)$ are very far away from the data manifold, the goal of adversarial training is to bring it close to the data manifold by giving G some useful gradient from D.
However, since D is trained beforehand with just real samples, it never sees the statistics of the fake samples.
So without the interaction (letting D be trained with fake samples as in GAN or EBGAN), the gradient is likely degenerated.
For another, from the theoretical perspective, the goal of EBGAN and GAN is to find a Nash Equilibrium (as explained in section 2.2).
Unless training D alone by real dataset converges to a global optimum D* with respect to any G, the (D, G) system cannot reach any Nash Equilibrium because when G reaches a minimum, the frozen D won’t be optimal any more.
We claimed in section 2.3 that using an auto-encoder as the discriminator is a promising choice because the phase of training D on real samples also contributes to the discovery of the data manifold (further we postulate the provided gradient from D is more stable and provides richer information).
Yet to find a Nash Equilibrium, an interactive adversarial training paradigm is still unavoidable.
This paper proposes a novel extension of generative adversarial networks that replaces the traditional binary classifier discriminator with one that assigns a scalar energy to each point in the generator's output domain. The discriminator minimizes a hinge loss while the generator attempts to generate samples with low energy under the discriminator. The authors show that a Nash equilibrium under these conditions yields a generator that matches the data distribution (assuming infinite capacity). Experiments are conducted with the discriminator taking the form of an autoencoder, optionally including a regularizer that penalizes generated samples having a high cosine similarity to other samples in the minibatch.
Pros:
* The paper is well-written.
* The topic will be of interest to many because it sets the stage for the exploration of a wider variety of discriminators than currently used for training GANs.
* The theorems regarding optimality of the Nash equilibrium appear to be correct.
* Thorough exploration of hyperparameters in the MNIST experiments.
* Semi-supervised results show that contrastive samples from the generator improve classification performance.
Cons:
* The relationship to other works that broaden the scope of the discriminator (e.g. [1]) or use a generative network to provide contrastive samples to an energy-based model ([2]) is not made clear in the paper.
* From visual inspection alone it is difficult to conclude whether EB-GANs produce better samples than DC-GANs on the LSUN and CelebA datasets.
* It is difficult to assess the effect of the PT regularizer beyond visual inspection as the Inception score results are computed with the vanilla EB-GAN.
Specific Comments
* Sec 2.3: It is unclear to me why a reconstruction loss will necessarily produce very different gradient directions.
* Sec 2.4: It is confusing that "pulling-away" is abbreviated as "PT".
* Sec 4.1: It seems strange that the Inception model (trained on natural images) is being used to compute KL scores for MNIST. Using an MNIST-trained CNN to compute Inception-style scores seems to be more appropriate here.
* Figure 3: There is little variation across the histograms, so this figure is not very enlightening.
* Appendix A: In the proof of theorem 2, it is unclear to me why a Nash equilibrium of the system exists.
Typos / Minor Comments
* Abstract: "probabilistic GANs" should probably be "traditional" or "classical" GANs.
* Theorem 2: "A Nash equilibrium ... exists"
* Sec 3: Should be "Several papers were presented"
Overall, I have some concerns with the related work and experimental evaluation sections, but I feel the model is novel enough and is well-justified by the optimality proofs and the quality of the generated samples.
[1] Springenberg, Jost Tobias. "Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks." arXiv preprint arXiv:1511.06390 (2015).
[2] Kim, Taesup, and Yoshua Bengio. "Deep Directed Generative Models with Energy-Based Probability Estimation." arXiv preprint arXiv:1606.03439 (2016).
Thanks for the comments.
For the cons presented in the review, they are all good points. We make the following response:
i- The loss function of CatGAN ([1]) is composed from several entropy terms. Since the integration of the partition function is not needed in this case, we see it as a specific form of energy function, as opposed to the probabilistic functions. That being said, CatGAN can be categorized into the EBGAN framework. The unique thing it possesses is its energy function not only takes $x$ (or $G(z)$) but also takes a virtual label variable $y$.
i- The connection to the work [2], is not as strong as implied by the title of the paper. This proposed framework doesn't get rid of the computational challenging partition function, so the choice of the energy function is required to be integratable. Our work is proposed from an opposite angle where the energy function is completely free of the integral of the probability density, which provides more freedom for the choice of loss functional.
ii- Admittedly, we lack a well-formed methodology for comparison besides visual generation.
However, we claim that for LSUN bedroom generation in figure 5, some flaw is seen in the DCGAN generation, such as the blue-bed alike generation appearing in the images located at 1r6c, 2r3c, 3r1c, 4r4c, 4r7c, 7r4c (1r6c means 1st row 6th column). The identically configured EBGAN auto-encoder model trained with PT term (EBGAN-PT) excludes such flaw.
Furthermore, we claim in figure 6, EBGAN-PT produces higher quality generation than DCGAN if examining closely.
iii- Thanks for pointing this out. We're launching some grid search experiments on the EBGAN-PT model. We will update the paper with this result when the experiments are finished.
For the special comments:
i-We made some justification as to why an auto-encoder could work as a discriminator in the upfront two bullets of section 2.3.
Briefly, the idea is that an auto-encoder could provide richer information than a scalar-output network where only a single bit is used at one time. And the auto-encoders are well-studied energy-based models themselves. Training an auto-encoder with real samples alone is also providing information of the data manifold, unlike a binary-classifier to which feeding mere real samples doesn't make sense.
ii-We'll change it.
iii-We indeed use a three-layer ConvNets as the offline trained classifier. We inherit the name “inception-score” from the Improved-GAN paper ([3]) where the score was originally proposed. In our MNIST experiments, the inception score has *nothing* to do with the inception model trained on ImageNet. We apologize for this confusion. Some clarification of this is available in the Appendix C.
iv-We'll try to make the histograms look better in the next version.
v-The existence of Nash Equilibrium is proven by two steps: (i)-given enough capacity of G and D, there must exist a model pair (G*, D*) that can satisfy both condition (a) and (b) in the theorem 2; (ii)-by using the sufficient condition of theorem 2 would give us a Nash Equilibrium which is defined by equation 3 and 4.
Overall, we will update the paper shortly by adding some of the comments above and correcting the typos and minor mistakes in the next revision. Thanks!
[1] Springenberg, Jost Tobias. "Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks." arXiv preprint arXiv:1511.06390 (2015).
[2] Kim, Taesup, and Yoshua Bengio. "Deep Directed Generative Models with Energy-Based Probability Estimation." arXiv preprint arXiv:1606.03439 (2016).
[3] Salimans, Tim, et al. "Improved techniques for training gans." Advances in Neural Information Processing Systems. 2016.
Can you show empirically what impact the introduction of both the hinge loss and the auto-encoder parametrization of energy function have on the training and performance of the EBGAN?
In other words, what happens when you remove these features?
We did an ablation study on the hinge loss alone, in Appendix E, where we show the effect of different margin value $m$ (as indicated by model generations). From the trend presented in figure 9, we postulate that some extreme setting of $m$ would be very close to replacing the hinge loss. Alongside these empirical observations, some theoretical explanation is provided in the same section.
We haven’t experimented with removing both features, but we will try to include some ablation studies on this in a future revision.
Thanks for the suggestion!
Thank you for your comment.
As explained in the paper, the encoder-decoder structure is not necessary to achieve the theoretical guaranty that p_G becomes equal to p_data when the system reach a stable point; the result is proven in a more general margin-loss setting.
We did run experiments with a margin-loss alone (without the auto encoder structure), and the system indeed produces correct samples, so the auto-encoder is not strictly needed (and has no reason to be). We also show comparisons with and without the PT term in the appendix.
We focus our paper on the auto-encoder setting, though. We justify the use of the auto-encoder setting in the paper (section 2.3). We feel it does not make much sense to run the EBGAN without the auto-encoder but with the PT term.
Indeed, the PT term assumes the features planes are different for different "fake" inputs. This in indeed correct when using an auto-encoder, since the information has to be there for the reconstruction.
When using a single-valued discriminator, the only information that matters is "real" versus "fake", so there is no reason for the feature planes of two different "fake" samples to be different. Actually, they are encouraged to be similar.
We acknowledge that there is more experimentation to be done, in particular on other form of discriminators. The scope of this paper is (1) to introduce a general class of discriminators and (2) to explore one particular discriminator in this class.
Further research will explore different architectures for the discriminator.
This paper introduces an energy-based Generative Adversarial Network (GAN) and provides theoretical and empirical results modeling a number of image datasets (including large-scale versions of categories of ImageNet). As far as I know energy-based GANs (EBGAN) were introduced in Kim and Bengio (2016), but the proposed version makes a number of different design choices.
First, it does away with the entropy regularization term that Kim and Bengio (2016) introduced to ensure that the GAN discriminator converged to an energy function proportional to the log density of the data (at optimum). This implies that the discriminator in the proposed scheme will become uniform at convergence as discussed in the theoretical section of the paper, however the introductory text seems to imply otherwise -- that one could recover a meaningful score function from the trained energy-function (discriminator). This should be clarified.
Second, this version of the EBGAN setting includes two innovations: (1) the introduction of the hinge loss in the value function, and (2) the use of an auto-encoder parametrization for the energy function. These innovations are not empirically justified in any way - this is disappointing, as it would be really good to see empirical results supporting the arguments made in support of their introduction.
The two significant contributions of this paper are the theoretical analysis of the energy-baesd GAN formalism (showing that the optimum corresponds to a Nash equilibrium) and the impressive empirical results on large images that set a new standard in what straight GAN-style models can achieve.
The theoretical results seem solid to me and make a nice contribution.
Regarding the quantitative results in Table 2, it seems not appropriate to bold the EBGAN line when it seems to be statistically indistinguishable form the Rasmus et al (2015) results.
Though it is not mentioned here, the use of bold typically indicates the state-of-the-art.
I think this paper could be be much stronger if the two novel contributions to the energy-based GAN setting were more thoroughly explored with ablation experiments. That being said, I think this paper has already become a contribution other are building on (including at least two other ICLR submissions) and so I think it should be accepted for publication at ICLR.
Thanks for the comments.
First, despite the resemblance of the titles, our work is actually quite different from Kim and Bengio (2016) [1]. Their approach uses a standard probabilistic GAN, and cast it into an energy model (using Gibbs distributions). That allows them to learn a discriminator that models the distribution when a Nash equilibrium is reached.
On the other hand, our approach gets rid of the probabilistic setting, while presenting the same Nash equilibrium as a standard GAN, but through a different and more generalized class of loss functionals (experiences also show that it can make the training easier).
Second, as we mentioned in the comment to the pre-review question, we did some near-ablation study on the hinge loss in Appendix E, where it shows up the interpolation between small and large margin value. For the auto-encoder part, the main focus of this paper is to explore one particular form of discriminator and theoretically prove a wide family of the choices for the discriminator.
Furthermore, techniques introduced in this work such as the PT term and the combination with Ladder Network are all grounded on the encoder-decoder reconstruction structure.
We’ll update the paper shortly to reflect the above clarification. Thanks!
[1] Kim, Taesup, and Yoshua Bengio. "Deep Directed Generative Models with Energy-Based Probability Estimation." arXiv preprint arXiv:1606.03439 (2016).
This paper is a parallel work to Improving Generative Adversarial Networks with Denoising Feature Matching. The main solution of both papers is introducing autoencoder into discriminator to improve the stability and quality of GAN. Different to Denoising Feature Matching, EBGAN uses encoder-decoder instead of denoising only, and use hingle loss to replace original loss function.
The theoretical results are good, and empirical result of high resolution image is unique among all recent GAN advantages.
I suggest to introduce Improving Generative Adversarial Networks with Denoising Feature Matching as related work.
Thanks for notifying us this related work, Improving Generative Adversarial Networks with Denoising Feature Matching (we here temporarily abbreviate it into GAN-DFM).
Admittedly, the EBGAN framework bears some resemblance, but the main motivation and procedures are very different, as the authors of GAN-DFM paper have commented at their paper site here (https://openreview.net/forum?id=S1X7nhsxl, the “relations to other work” comment).
However, we do project some possibility of relating GAN-DFM within the EBGAN framework.
For instance, on top of the auto-encoder reconstruction energies in the EBGAN auto-encoder model, we can further add a binary classifier upon the top layer of the encoder, which results in a combinatorial energy function formulation: hierarchical reconstruction losses from all layers of encoder-decoder structures with a logistic loss.
In other words, the discriminator of GAN-DFM can be seen as an energy function, constructed by a discriminative encoder-decoder architecture trained only with real data samples in a layer-wise manner.

Meta-review:
role: PC
data: comment: The authors have proposed an energy-based rendition of probabilistic GANs, with the addition of auto-encoder and hinge loss to improve stability. Theoretical results involving the Nash equilibrium are also given. Solid paper, well-written. Novel contribution with good empirical and theoretical justification.

Decision: Accept (Poster)