Venue: ICLR 2017 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: After reading the rebuttal, I decided to increase my score. I think ALI somehow stabilizes the GAN training as demonstrated in Fig. 8 and learns a reasonable inference network.

---------------
Initial Review:

This paper proposes a new method for learning an inference network in the GAN framework. ALI's objective is to match the joint distribution of hidden and visible units imposed by an encoder and decoder network. ALI is trained on multiple datasets, and it seems to have a good reconstruction even though it does not have an explicit reconstruction term in the cost function. This shows it is learning a decent inference network for GAN.

There are currently many ways to learn an inference network for GANs: One can learn an inference network after training the GAN by sampling from the GAN and learning a separate network to map X to Z. There is also the infoGAN approach (not cited) which trains the inference network at the same time with the generative path. I think this paper should have an extensive comparison with these other methods and have a discussion for why ALI's inference network is superior to previous works.

Since ALI's inference network is stochastic, it would be great if different reconstructions of a same image is included. I believe the inference network of the BiGAN paper is deterministic which is the main difference with this work. So maybe it is worth highlighting this difference.

The quality of samples is very good, but there is no quantitative experiment to compare ALI's samples with other GAN variants. So I am not sure if learning an inference network has contributed to better generative samples. Maybe including an inception score for comparison can help.

There are two sets of semi-supervised results: 
The first one concatenate the hidden layers of the inference network and uses an L2-SVM afterwards. Ideally, concatenating feature maps is not the best way for semi-supervised learning and one would want to train the semi-supervised path at the same time with the generative path. It would have been much more interesting if part of the hidden code was a categorical distribution and another part of it was a continuous distribution like Gaussian, and the inference network on the categorical latent variable was used directly for classification (like semi-supervised VAE). In this case, the inference network would be trained at the same time with the generative path. Also if the authors can show that ALI can disentangle factors of variations with a discrete latent variable like infoGAN, it will significantly improve the quality of the paper.

The second semi-supervised learning results show that ALI can match the state-of-the-art. But my impression is that the significant gain is mainly coming from the adaptation of Salimans et al. (2016) in which the discriminator is used for classification. It is unclear to me why learning an inference network help the discriminator do a better job in classification. How do we know the proposed method is improving the stability of the GAN? My understanding is that one of the main points of learning an inference network is to learn a mapping from the image to the high-level features such as class labels. So it would have been more interesting if the inference path was directly used for semi-supervised learning as I explained above.
review_title: official review

final_score: rating: null
review_content: What is the difference of the ALI model to BiGAN (https://arxiv.org/abs/1605.09782) ?
review_title: Pre-review question

final_score: rating: null
review_content: I think the idea of matching the joint distributions in order to learn an inference network for GANs is interesting.

I have a few questions from the authors:
 
1) I had read the previous version of the paper in which the classification rate on SVHN was 19.14% which is good, but not outstanding. What exactly has changed in the new paper which achieves 7.3% classification rate. Is it only the adaptation of Salimans et al. (2016), better hyper-parameters or something else?

2) I think the main contribution of this paper is proposing a new way to learn an inference network for GANs. So may I ask if the authors have tried to use this inference network for semi-supervised learning rather than using the GAN discriminative network as done in this paper, in Radford et al. (2015) and Salimans et al. (2016).

3) The conditional generation is interesting, but I have seen similar experiments in papers such as "Autoencoding beyond pixels using a learned similarity metric". Does this paper claim that the image generation results is better than the previous works. If so why?
review_title: few questions

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper extends the GAN framework to allow for latent variables. The observed data set is expanded by drawing latent variables z from a conditional distribution q(z|x). The joint distribution on x,z is then modeled using a joint generator model p(x,z)=p(z)p(x|z).  Both q and p are then trained by trying to fool a discriminator. This constitutes a worthwhile extension of GANs: giving GANs the ability to do inference opens up many applications that could previously only be addressed by e.g. VAEs.

The results are very promising. The CIFAR-10 samples are the best I've seen so far (not counting methods that use class labels). Matching the semi-supervised results from Salimans et al. without feature matching also indicates the proposed method may improve the stability of training GANs.
review_title: interesting extension of GANs, promising results

final_score: rating: 7: Good paper, accept
review_content: This is a parallel work with BiGAN.  The idea is using auto encoder to provide extra information for discriminator. This approach seems is promising from reported result.
review_title: null

Meta-review: The reviewers were positive about this paper and agree that it will make a contribution to the community.
Decision: Accept (Poster)