Title: Generative Adversarial Networks as Variational Training of Energy Based Models
Authors: Shuangfei Zhai
Yu Cheng
Rogerio Feris
Zhongfei Zhang
Venue: ICLR 2017

Abstract: In this paper, we study deep generative models for effective unsupervised learning. We propose VGAN, which works by minimizing a variational lower bound of the negative log likelihood (NLL) of an energy based model (EBM), where the model density $p(\mathbf{x})$ is approximated by a variational distribution $q(\mathbf{x})$ that is easy to sample from. The training of VGAN takes a two step procedure: given $p(\mathbf{x})$, $q(\mathbf{x})$ is updated to maximize the lower bound; $p(\mathbf{x})$ is then updated one step with samples drawn from $q(\mathbf{x})$ to decrease the lower bound. VGAN is inspired by the generative adversarial networks (GANs), where $p(\mathbf{x})$ corresponds to the discriminator and $q(\mathbf{x})$ corresponds to the generator, but with several notable differences. We hence name our model variational GANs (VGANs). VGAN provides a practical solution to training deep EBMs in high dimensional space, by eliminating the need of MCMC sampling. From this view, we are also able to identify causes to the difficulty of training GANs and propose viable solutions.


Review discussion:
Do you have learning the curves from multiple runs? That could help comparing the stability of the models.
Hi,
We did not record multiple runs in our experiments, but we experienced very similar behaviors across different random initializations. You are also welcome to try out our code, which is available through the link given in the paper, and reproduce our experiments.
Thanks,
Shuangfei
1. You use a transition operator to serve as a generator, and it appears that this choice contributes significantly to the results. To understand the role of the transition operator better, have you tried either training your model without transition operator, or training a regular GAN with transition operator?
2. What is H tilde mentioned in algorithm 2? Is it a parametrized model? How do you optimize it?
Hi, Thanks for the comments!
1. The results in Figure 2 are trained without the transition operator, which demonstrates the importance of using a bounded multi-modal energy formulation. We did not try GAN with the transition operator, mainly because we are interested in a broader family of energy based models, which is not specific to GAN. In our framework, this amounts to directly using Equation (7) as the objective, which is actually slightly different from GAN (in the second term of the objective).
2. H tilde is defined in Equation (9) and the following paragraph, which is the same as used in Algorithm 1. It is a differentiable function of either the discriminator or generator parameters, thus can be optimized by gradient descent.
This paper presents a bridging of energy-based models and GANs, where -- starting from the energy-based formalism -- they derive an additional entropy term that is not present in the original GAN formulation and is also absent in the EBGAN model of Zhao et al (2016, cited work). The relation between GANs and energy-based models was, as far as I know, first described in Kim and Bengio (2016, cited work) who also introduced the entropy regularization. It is also discussed in another ICLR submission (Dai et al. "Calibrating Energy-based Generative Adversarial Networks"). There are two novel contribution of this paper: (1) VGAN: the introduction of a novel entropy approximation; (2) VCD: variational contrastive divergence is introduced as a  novel learning algorithm (however, in fact, a different model). The specific motivation for this second contribution is not particularly clear.
The two contributions offered by the authors are quite interesting and are well motivated in the sense that the address the important problem of avoiding dealing with the generally intractable entropy term.
However, unfortunately the authors present no results directly supporting either contribution.
For example, it is not at all clear what role, if any, the entropy approximation plays in the samples generated from the VGAN model. Especially in the light of the impressive samples from the EBGAN model that has no corresponding term. The results provided to support the VCD algorithm, come in the form of a comparison of samples and reconstructions.
But the samples provided here strike me as slightly less impressive compared to either the VGAN results or the SOTA in the literature - this is, of course, difficult to evaluate.
The results the authors do present include qualitative results in the form of reasonably compelling samples from the model trained on CIFAR-10, MNIST and SVHN datasets. They also present quantitative results int he form of semi-supervised learning tasks on the MNIST and SVHN.
However these
quantitative results are not particularly compelling as they show limited improvement over baselines.
Also, there is no reference to the many
existing semi-supervised results on these datasets.
Summary: The authors identify an important problem and offer two novel and intriguing solutions. Unfortunately the results to not sufficiently support either
contribution.
Thanks for the comments.
The goal of this paper is to provide a principled view to guide the training of GAN-like models, which we interpret as minimizing a variational lower bound of the negative log likelihood of an EBM. From this view, we are able to identify several important issues of training GANs and EMBs that are not revealed in the previous literature.
For example, we identified that minimizing a lower bound is a major source of instability, as when the bound is not tight, there is no guarantee that the true NLL is minimized.
Therefore, it is important to switch the order of the G and D updates to update G K steps to tighten the bound then update D one step, instead of the other way around as proposed in the original GAN paper (see Table 1 and Figure 2). We also identified the importance of bounded energy parameterization in light of this training strategy, which is important but often ignored in related EBM literature.
We did not include the (failed) results trained without the entropy approximation, as the importance of diversity promoting regularizations is well recognized in recent GAN related papers (e.g., batch discrimination in improved gan).
Moreover, the related work EBGAN that the reviewer pointed out actually does have a similar term as well, which they call the "repelling regularizer" (section 2.4 of Zhao et al.). We argue that observing the diversity of the generated samples is sufficient to verify the effectiveness of the proposed approximation, although we make no claim that our proposal is by any means an accurate approximation of the true entropy.
The motivation of VCD comes exactly from the difficulty of accurate approximation of the entropy term. VCD initializes p_g(x) as a distribution close to p_data(x), which essential bypasses the need of directly approximating and maximizing its entropy. The fact that this works well is not surprising either due to its similarity to contrastive divergence and the vast success CD achieves in training EBMs. Our way of evaluating the quality of generated samples with the proposed semi-supervised learning benchmark is also novel, and an arguably more objective metric those used previously such as visual quality, parzen window-based likelihood estimation and inception scores. We argue that the improvements gained by using only generated samples as data augmentation are nontrivial, and demonstrate the usefulness of the generative model in an objective way. We will update our paper with related semi-supervised learning methods in a later version for a more complete comparison.
Thanks,
Shuangfei
Regarding the entropy approximation in equation 9, do you have any reason to think this is a reasonable approximation? You state that there is no theoretical guarantee "recovers the true entropy", but do you have any empirical evidence that this is a reasonable approximation in principle, or that the gradient is a reasonable approximation of the entropy gradient?
You state that maximizing the entropy approximation in equation 9 "serves the same purpose of encouraging the generated samples to be diverse", but it seems like a degenerate solution where all the W_j weights are zero would also maximize equation 9, but not at all encourage diversity.
First, do you agree?
Second, do you have evidence that this does not happen?
Thanks for the questions.
We believe Equation 9 is a reasonable approximation of the entropy in the sense that it encourages diversified samples from the generator. And no it is not a degenerate solution. To see this, when updating G, the objective subject to maximization (shown in algorithm 1) can be rewritten as: \frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{K} KL(\sigma_j^i \| \bar{\sigma_j}), where \sigma_j^i is the j-th sigmoid activation for the i-th generated sample, and \bar{\sigma_j} is the averaged activation over the generated mini-batch for the j-th sigmoid. Maximizing the this KL divergence encourages G to generate samples that cause each sigmoid unit to be either 0 or 1 with probability 0.5. As each W_j is initialized randomly, each generated sample tens to randomly fall into one of the 2^K optimal sigmoid activation patterns. Note that this step does not affect W_j, which is part of D.
When updating D, the same KL divergence is maximized on a training data mini-batch (see algorithm 1), where the case when W_j are all zero yields a minimum KL divergence (least optimal solution) and will never occur with a correct optimization procedure.
The evidence that this does not happen is obvious.
When all W_j is set to all zero vectors, the model crashes and no learning can proceed: the EBM simply encodes a uniform distribution over the input space. The fact that we are able to generate quality samples directly contradicts this hypothesis.
This paper presents an adversarial training formulation for energy models.
Although the relationship between energy-based models and GANs is abundantly clear in the literature, the contributions of this paper seems to be a multimodal energy estimate, and training a transition operator instead of a sampler. The core technical contribution of the paper is not clear. The only two quantitative results in the paper demonstrate that the proposed method learns better features than a traditional GAN, and that the proposed method does a better job in SSL than the authors' chosen baseline.
In either case, no comparison to existing literature is made, even though GANs have been used for SSL previously (https://arxiv.org/abs/1606.03498). The qualitative results (visual comparison of quality of samples) is inconclusive.
The lack of comparison to existing literature makes this paper a clear reject. To improve this paper, the authors need to make the core contribution of the paper much better, and situate the paper very clearly wrt existing literature. The motivation for the development of the model should be clearer.
Further, experiments need to compare with existing literature.
Thanks for the comments. We agree that including more related results would be useful for calibrating the contribution of the paper, we will provide an updated version later.
However, we do not agree with the reviewer's point about "Although the relationship between energy-based models and GANs is abundantly clear in the literature".  We argue that our paper provide novel views to this connection, even considering all the parallel submissions to this conference.
For example, when treating GAN as minimizing a variational lower bound of an NLL, it is critical to update G k steps per D update to tighten the lower bound, which is in contradiction with GAN's proposal. More recent findings, such as [Towards Principled Methods for Training
Generative Adversarial Networks, Arjovsky et al. 2016], also supports our argument that training D multiple steps according to the GAN theory leads to worse results.
Although we do not think that our solution is perfect (simply updating G k steps), this remains an important issue to be solved to fully understand both GAN and the variational training of EBMs. Our other contributions that are non-existing in the previous literature also include the discussion of proper choices of energy functions, which is extremely important for designing and training new families of EBMs.
The paper drives a variant of GAN's min max game optimizations from EBM's NLL minimization and propose a new generative model from this derivation.
In introduction, could you elaborate why accurate distance metric is needed for unsupervised learning.
It's a bit hard to read the paper as it jumps from points to points without clear connection and laying down the background.  The introduction, refers to many different concepts without any clear connection. And other sections as well is very incomprehensible even if one is familiar with the concepts.
There has been lots of interests in similar area recently, Authors do cite some of them in introduction and related work but the relationship and comparison are missing.
In the results, if it's comparison of the quality of samples it has to include other recent works and compare those as well. And if it's semi-supervised learning, again, the numbers should be compared to other works.
In summary, unfortunately the paper is very clear and cumbersome to read which makes it hard to judge it fairly. I strongly suggest re-write of the paper in more coherent matter to make it easier to read. And also extend the experiments with more comparisons with other works.
Thanks for the comments.
About the distance metric: unsupervised learning does not necessarily need an accurate distance metric, it just happens that most of the commonly used methods do assume a simple distance metric (such Euclidean distance), such as K-means, Gaussian Mixture Models, PCA, Autoencoders, VAE and etc.. These models either directly use a predefined distance metric in the input space, or assume equivalent distributions (such as Gaussian distributions) in the input space, which directly limits their usefulness for high dimensional inputs where there exists no reasonable simple distance metric.
We do agree that we could do a better job at elaborating the relationship with existing works, we will provide an updated version to clarify the unclearness.

Meta-review:
role: PC
data: comment: This paper is timely since it addresses the connections between energy-based models, GANS, and the general space of generative models. The two principal concerns about the paper are: lack of clarity and coherence in the paper; inability to effectively judge the effectiveness of the method. The responses of the authors address these concern to some extent, but these concerns remain. The paper will have much higher-impact after further introspection, but at present, the paper is not yet ready for acceptance at the conference.

Decision: Reject