Title: Adversarially Learned Inference
Authors: Vincent Dumoulin
Ishmael Belghazi
Ben Poole
Alex Lamb
Martin Arjovsky
Olivier Mastropietro
Aaron Courville
Venue: ICLR 2017
Keywords: Computer vision
Deep learning
Unsupervised Learning
Semi-Supervised Learning

Abstract: We introduce the adversarially learned inference (ALI) model, which jointly
learns a generation network and an inference network using an adversarial
process. The generation network maps samples from stochastic latent variables to
the data space while the inference network maps training examples in data space
to the space of latent variables. An adversarial game is cast between these two
networks and a discriminative network that is trained to distinguish between
joint latent/data-space samples from the generative network and joint samples
from the inference network.  We illustrate the ability of the model to learn
mutually coherent inference and generation networks through the inspections of
model samples and reconstructions and confirm the usefulness of the learned
representations by obtaining a performance competitive with other recent
approaches on the semi-supervised SVHN task.

TL;DR: We present and adverserially trained generative model with an inference network. Samples quality is high. Competitive semi-supervised results are achieved.

Review discussion:
What is the difference of the ALI model to BiGAN (https://arxiv.org/abs/1605.09782) ?
Both methods were developed independently and published at roughly the same time (BiGAN released May 31 2016, ALI released 2 days later on June 2 2016).  Both papers acknowledge that the methods were developed independently and are very similar.
There are differences in the experiments and ALI considered stochastic encoders, but the two proposed models are essentially the same.
I think the idea of matching the joint distributions in order to learn an inference network for GANs is interesting.
I have a few questions from the authors:
1) I had read the previous version of the paper in which the classification rate on SVHN was 19.14% which is good, but not outstanding. What exactly has changed in the new paper which achieves 7.3% classification rate. Is it only the adaptation of Salimans et al. (2016), better hyper-parameters or something else?
2) I think the main contribution of this paper is proposing a new way to learn an inference network for GANs.
So may I ask if the authors have tried to use this inference network for semi-supervised learning rather than using the GAN discriminative network as done in this paper, in Radford et al. (2015) and Salimans et al. (2016).
3) The conditional generation is interesting, but I have seen similar experiments in papers such as "Autoencoding beyond pixels using a learned similarity metric". Does this paper claim that the image generation results is better than the previous works. If so why?
Thank you for your questions.
1) The adaptation of the Salimans et al. (2016) method is what is responsible for the misclassification rate reduction. Note that the semi-supervised method used in the first version of the ALI arXiv paper differs from the one used in this submission in one key aspect. In the former, semi-supervised learning is done after ALI has been trained. In the latter, semi-supervised learning is integrated into ALI's training procedure. In that sense, it is unsurprising that the Salimans et al. (2016) method performs better than a linear SVM trained on features extracted from ALI's encoder.
2) The inference network is used for both methods mentioned in 1). In the first version of the arXiv paper, the encoder is used after unsupervised training as a feature extractor. We show that it outperforms the Radford et al. (2015) semi-supervised result with GANs. In this submission, the discriminator adapted for the semi-supervised learning task receives the encoder's output as input. We show that doing so allows us to match the Salimans et al. (2016) semi-supervised results without having to do feature matching.
3) This paper does not claim that the conditional image generation results are better than previous work. The results are presented to support the claim that ALI is also amenable to class-conditional generative modeling.
We updated the submission to highlight and clarify the difference between the semi-supervised learning results presented in the first arXiv version of the paper and this version.
This paper extends the GAN framework to allow for latent variables. The observed data set is expanded by drawing latent variables z from a conditional distribution q(z|x). The joint distribution on x,z is then modeled using a joint generator model p(x,z)=p(z)p(x|z).  Both q and p are then trained by trying to fool a discriminator. This constitutes a worthwhile extension of GANs: giving GANs the ability to do inference opens up many applications that could previously only be addressed by e.g. VAEs.
The results are very promising. The CIFAR-10 samples are the best I've seen so far (not counting methods that use class labels). Matching the semi-supervised results from Salimans et al. without feature matching also indicates the proposed method may improve the stability of training GANs.
We thank you for your feedback.
Great paper! I really enjoyed reading it.
The comparison with Salismans et al. might be a little unfair. ALI [1] was trained for 6475 epochs (which is a pretty large number I believe) whereas the Salisman et al. model [2] was trained for 1200 epochs only.
I'm curious about how much of the improvement in the sample quality for cifar10 is because of the model being better and how much of it is due to the longer training regime.
Would it be possible for the authors to share a graph of the Discriminator and Generator losses versus training progress? It would probably help us understand how important it was to run the training procedure for 6475 epochs (which seems like a rather large number).
[1]: https://github.com/IshmaelBelghazi/ALI/blob/master/experiments/ali_cifar10.py#L28
[2]: https://github.com/openai/improved-gan/blob/master/mnist_svhn_cifar10/train_cifar_feature_matching.py#L130
Good point! It might not be clearly specified, but the architectures in the appendix were not used for semi-supervised learning. For comparison with Salisman et al. model, we used an architecture very close to theirs. The differences are that we need to accomodate the encoder. This results in a ConvNet which has exactly the same structure (reversed order of feature maps, nonlinearities, etc) as their generator and with convolution instead of deconvolution. There is also a very small amount of extra parameters added into their discriminator to take this into account.
Also, we did stop training at 1200 epochs and the 6475 mentionned in the appendix is for the samples in the purely unsupervised setting.
After reading the rebuttal, I decided to increase my score. I think ALI somehow stabilizes the GAN training as demonstrated in Fig. 8 and learns a reasonable inference network.
Initial Review:
This paper proposes a new method for learning an inference network in the GAN framework. ALI's objective is to match the joint distribution of hidden and visible units imposed by an encoder and decoder network. ALI is trained on multiple datasets, and it seems to have a good reconstruction even though it does not have an explicit reconstruction term in the cost function. This shows it is learning a decent inference network for GAN.
There are currently many ways to learn an inference network for GANs: One can learn an inference network after training the GAN by sampling from the GAN and learning a separate network to map X to Z. There is also the infoGAN approach (not cited) which trains the inference network at the same time with the generative path. I think this paper should have an extensive comparison with these other methods and have a discussion for why ALI's inference network is superior to previous works.
Since ALI's inference network is stochastic, it would be great if different reconstructions of a same image is included. I believe the inference network of the BiGAN paper is deterministic which is the main difference with this work.
So maybe it is worth highlighting this difference.
The quality of samples is very good, but there is no quantitative experiment to compare ALI's samples with other GAN variants.
So I am not sure if learning an inference network has contributed to better generative samples. Maybe including an inception score for comparison can help.
There are two sets of semi-supervised results:
The first one concatenate the hidden layers of the inference network and uses an L2-SVM afterwards. Ideally, concatenating feature maps is not the best way for semi-supervised learning and one would want to train the semi-supervised path at the same time with the generative path. It would have been much more interesting if part of the hidden code was a categorical distribution and another part of it was a continuous distribution like Gaussian, and the inference network on the categorical latent variable was used directly for classification (like semi-supervised VAE). In this case, the inference network would be trained at the same time with the generative path.
Also if the authors can show that ALI can disentangle factors of variations with a discrete latent variable like infoGAN, it will significantly improve the quality of the paper.
The second semi-supervised learning results show that ALI can match the state-of-the-art.
But my impression is that the significant gain is mainly coming from the adaptation of Salimans et al. (2016) in which the discriminator is used for classification. It is unclear to me why learning an inference network help the discriminator do a better job in classification. How do we know the proposed method is improving the stability of the GAN? My understanding is that one of the main points of learning an inference network is to learn a mapping from the image to the high-level features such as class labels.
So it would have been more interesting if the inference path was directly used for semi-supervised learning as I explained above.
We thank you for you feedback.
REVIEWER POINT
“I think this paper should have an extensive comparison with these other methods and have a discussion for why ALI's inference network is superior to previous works.”
RESPONSE
The reviewer raises an important and salient point that, while we have shown that ALI does learn to do inference reasonably well, the paper doesn’t do enough to directly compare with alternative ways of doing feedforward inference in a GAN setting. To address these concerns, we will shortly add two new sections to the paper: (1) a review of alternative approaches and (2) a new experiment to highlight the role of the inference network during learning. These additions are summarized below.
Here is a list of alternative approaches and why they may or may not be fit for comparison with ALI:
* Learning the inverse mapping from GAN samples: This corresponds to learning an encoder to reconstruct Z, i.e. encode(decode(Z ~ p(Z))) ~= Z. We are not aware of any work that reports results for this approach. Could you point out to such work if it exists?
* InfoGAN: While InfoGAN should be cited as related work, InfoGAN actually does not do inference, it only estimates the discrete latent code which describes specific aspects of the image.  This is why the InfoGAN paper doesn’t show any reconstructions, rather it shows generated samples where they vary the latent code.
Additionally, InfoGAN uses a fixed reconstruction cost for the latent code C and requires a tractable approximate posterior, q(C|X), that can be sampled from and evaluated. ALI only requires that inference networks can be sampled from, allowing it to represent arbitrarily complex posterior distributions. Combining InfoGAN and ALI could be an exciting area for future work.
* Post-hoc learned inference: As verification that learning inference jointly with generation is beneficial, one can first train a GAN and then freeze the decoder and learn the encoder using the procedure proposed by ALI. In this setting, the encoder and the decoder cannot interact together during training and the encoder must work with whatever the decoder has learned during GAN training.
To address this point we performed an experiment on a toy dataset for which q(X) is a 2D gaussian mixture with 25 mixture components laid out on a grid. The covariance matrices and centroids have been chosen such that the distribution exhibits lots of modes separated by large low-probability regions, which makes it a decently hard task despite the 2D nature of the dataset.
We trained ALI and GAN on 100,000 q(X) samples. The decoder and discriminator architectures are identical between ALI and GAN. Each model was trained 10 times using Adam with random learning rate and beta_1 values, and the weights were initialized by drawing from a gaussian distribution with a random standard deviation.
We measured the extent to which the trained models covered all 25 modes by drawing 10,000 samples from their p(X) distribution and assigning each sample to a q(X) mixture component according to the mixture responsibilities. We defined a dropped mode as one that wasn’t assigned to *any* sample (which is a generous definition). Using this definition, we found that ALI models covered 13.4 ± 5.8 modes on average (min: 8, max: 25) while GAN models covered 10.4 ± 9.2 modes on average (min: 1, max: 22).
We then selected the best-covering ALI and GAN models, and the GAN model was augmented with the following inference mechanisms:
* Learned inverse mapping
* Post-hoc learned inference
The encoders learned for GAN inference have the same architecture as ALI’s encoder. We then compared each model’s inference capabilities by reconstructing 10,000 held-out samples from q(X).
A figure summarizing the experiment can be found at https://raw.githubusercontent.com/IshmaelBelghazi/ALI/master/paper/mixture_plot.png.
The three columns correspond to the three different strategies for learning inference:
1. ALI (our proposed strategy).
2. Learning an inverse mapping from GAN samples.
3. Post-hoc learned inference
The five rows correspond to:
1. X ~ q(X) samples, i.e. test set examples, colour-coded by mixture component. They're the same for all three columns.
2. Z_hat ~ q(Z | X) samples, i.e. the latent codes, also colour-coded.
3. X_hat ~ p(X | Z = Z_hat) samples, i.e. the reconstructions, also colour-coded.
4. Z ~ p(Z) samples, i.e. prior samples. They're the same for all three columns.
5. X_tilde ~ p(X | Z) samples, i.e. generator samples.
Here is what we observe:
* The ALI encoder models a marginal distribution q(Z) that matches p(Z) fairly well (row 2, column 1). The learned representation does a decent job at clustering and organizing the different mixture components.
* The GAN generator (row 5, columns 2-3) has more trouble reaching all the modes than the ALI generator (row 5, column 1), even over 10 runs of hyperparameter search.
* Learning an inverse mapping from GAN samples does not work very well: the encoder has trouble covering the prior marginally and the way it clusters mixture components is not very well organized (row 2, column 2).
* Learning inference post-hoc doesn't work as well as training the encoder and the decoder jointly. As had been discussed above, it appears that adversarial training benefits from learning inference at training time in terms of mode coverage. This also negatively impacts how the latent space is organized (row 2, column 3).
However, it appears to be better at matching q(Z) and p(Z) than when inference is learned through inverse mapping from GAN samples.
To summarize, this experiment provides evidence that adversarial training benefits from learning an inference mechanism jointly with the decoder.
Furthermore, it shows that our proposed approach for learning inference in an adversarial setting is superior to the other approaches investigated.
We will update the manuscript shortly to incorporate these additional results and cite the appropriate relevant work.
REVIEWER POINT
“Since ALI's inference network is stochastic, it would be great if different reconstructions of a same image is included. I believe the inference network of the BiGAN paper is deterministic which is the main difference with this work.
So maybe it is worth highlighting this difference.”
RESPONSE
Our experience is that the added stochasticity does not make much of a difference: at the end of training, very little noise ends up being injected at the encoder’s output. This falls in line with the invertibility results derived by Donahue et al. (2016).
We agree that left unexplained, this difference may confuse readers and we will update the manuscript to address this question.
REVIEWER POINT
“The quality of samples is very good, but there is no quantitative experiment to compare ALI's samples with other GAN variants.
So I am not sure if learning an inference network has contributed to better generative samples. Maybe including an inception score for comparison can help.”
RESPONSE
We do not claim that learning an inference network contributes to better generative samples, simply that doing so does not come at the expense of sample quality.
However, as explained above, experiments on a toy dataset suggest that ALI is better at mode coverage than GAN.
Regarding the use of the Inception score, we are hesitant to use it, as Odena et al. (2016) [1] found that “[the] Inception accuracy can not measure whether a model has collapsed. A model that simply memorized one example from each ImageNet class would do very well by this metric.”
REVIEWER POINT
“There are two sets of semi-supervised results [...]”
RESPONSE
The point which we try to make with the semi-supervised results is that:
* In the first method, one can train a shallow classifier using learned features from an unsupervised model.  We used the *exact same* procedure as DCGAN to build an L2-SVM on top of existing features, with the exception that the features were taken from the inference network’s high-level representation as opposed to the discriminator.  In this case ALI outperformed the results reported in the DCGAN paper, which suggests that for this procedure the inference network buys us something over the discriminator. To ensure that the comparison is fair, we are currently running an experiment in which DCGAN and ALI share the same architecture. We will update the manuscript when it is complete.
* In the second method, one co-trains the discriminator for label classification. With this procedure ALI matches the more recent “Improved methods for training GANs” results while using a simpler architecture (no feature matching). In this setting, the inference network *is* used, as it provides one of the two inputs which the discriminator uses to produce its prediction.
While an inference network on categorical variables like in semi-supervised VAEs does sound elegant and straightforward, it is not directly applicable to ALI.
Like GANs, ALI requires that the conditional distributions p(x | z) and q(z | x) can be sampled from in a way that allows gradient backpropagation, which cannot be achieved in a straightforward manner using discrete random variables.
[1] Odena, A., Olah, C., & Shlens, J. (2016). Conditional Image Synthesis With Auxiliary Classifier GANs. arXiv preprint arXiv:1610.09585.
This is a parallel work with BiGAN.  The idea is using auto encoder to provide extra information for discriminator. This approach seems is promising from reported result.
We thank you for your review.
We recently updated the submission taking the reviewers' comments into account. The main change is the addition of a section discussing alternative approaches to feedforward inference in GANs and section with a toy experiment comparing ALI's behaviour with those approaches.

Meta-review:
role: PC
data: comment: The reviewers were positive about this paper and agree that it will make a contribution to the community.

Decision: Accept (Poster)