Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Thanks for the feedback and for clarifying the 1) algorithm and the assumptions in the multivariate case 2) comparison to RL based methods 3) connection to estimating importance sampling weights using GAN discriminator.

I think the paper contribution is now more clear and strengthened with additional convincing experiments and I am increasing my score to 7.

The paper would still benefit from doing the experiment with importance weights by pixel , rather then a global one as done in the paper now. I encourage the authors to still do the experiment, see if there is any benefit.



==== Original Review =====
Summary of the paper:

The paper presents a method based on importance sampling and reinforcement learning to learn discrete generators in the GAN framework. The GAN uses an  f-divergence cost function for  training the discriminator. The generator is trained to minimize the KL distance between the  discrete generator q_{\theta}(x|z), and the importance weight discrete real distribution estimator w(x|z)q(\theta|z). where w(x|z) is estimated in turn using the discriminator. 
The methodology is also extended to the continuous case. Experiments are conducted on quantized image generation, and text generation.

Quality:

the paper is overall well written and supported with reasonable experiments.

Clarity:

The paper has a lot of typos that make sometimes the paper harder to follow:
- page (2) Eq 3 max , min should be min, max if we want to keep working with f-divergence
- Definition 2.1 \mathbb{Q}_{\theta} --> \mathbb{Q}
- page 5 the definition of \tilde{w}(x^{(m})) in the normalization it is missing \tilde{w}
- Equation (10) \nabla_{\theta}\log(x|z) --> \nabla_{\theta}\log(x^{(m)}|z)
- In algorithm 1, again missing indices in the update of theta  --> \nabla_{\theta}\log(x^{(m|n)}|z^{n})
 
Originality:

The main ingredients of the paper are well known and already used in the literature (Reinforce for discrete GAN with Disc as a reward for e.g GAN for image captioning Dai et al). The perspective from importance sampling coming from f-divergence for discrete GAN has some novelty although the foundations of this work relate also to previous work:
- Estimating ratios using the discriminator is well known for e.g learning implicit models , Mohamed et al 
- The relation of  importance sampling to  reinforce is also well known" On a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient," Tang and Abbeel.

General Review:

- when the generator is producing only *one* discrete distribution the theory is presented in Section 2.3. When we move to experiments, for image generation for example, we need to have a generator that produces a distribution by pixel. It would be important for 1) understanding the work 2) the reproducibility of the work to parallel algorithm 1 and have it *in the paper*, for this 'multi discrete distribution ' generation case.  If we have N pixels    \log(p(x_1,...x_N|z))= \Pi_i g_{\theta}(x_i|z) (this should be mentioned in the paper if it is the case ), it would be instructive to comment on the assumptions on independence/conditional dependence of this model, also to state clearly how the generator is updated in this case and what are importance sampling weights. 

- Would it make sense in this N pixel discrete case generation to have also the discriminator produce N probabilities of real and fake as in PixelGAN in Isola et al? then see in this case what are the importance sampling weights this would parallel the instantaneous reward in RL?



 

review_title: review for boundary seeking GAN

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper introduces a new method for training GANs with discrete data. To this end, the output of the discriminator is interpreted as importance weight and REINFORCE-like updates are used to train the generator.

Despite making interesting connections between different ideas in GAN training, I found the paper to be disorganized and hard to read. My main concern is the fact that the paper does not make any comparison with other methods for handling of discrete data in GANs. In particular, (Gulrajani et al.’17) show that it is possible to train Wasserstein GANs without sampling one-hot vectors for discrete variables during the training. Is there a reason to use REINFORCE-like updates when such a direct approach works?

Minor: 
complex conjugate => convex conjugate 
review_title: Why should I use your method?

final_score: rating: 7: Good paper, accept
review_content: Thank you for the feedback, and I have read the revision.

I would say the revised version has more convincing experimental results (although I'm not sure about the NLP part). The authors have also addressed my concerns on variance reduction, although it's still mysterious to me that the density ratio estimation method seems to work very well even at the begining stage.

Also developing GAN approaches for discrete variables is an important and unsolved problem.

Considering all of the above, I would like to raise the rating to 7, but lower my confidence to 3 (as I'm not an expert for NLP which is the main task for discrete generative models).

==== original review ====

Thank you for an interesting read.

My understanding of the paper is that:

1. the paper proposes a density-ratio estimator via the f-gan approach;
2. the paper proposes a training criterion that matches the generator's distribution to a self-normalised importance sampling (SIS) estimation of the data distribution;
3. in order to reduce the variance of the REINFORCE gradient, the paper seeks out to do matching between conditionals instead.

There are a few things that I expect to see explanations, which are not included in the current version:

1. Can you justify your variance reduction technique either empirically or experimentally? Because your method requires sampling multiple x for a single given z, then in the same wall-clock time I should be able to obtain more samples for the vanilla version eq (8). How do they compare?

2. Why your density ratio estimation methods work in high dimensions, even when at the beginning p and q are so different?

3. It's better to include some quantitative metrics for the image and NLP experiments rather than just showing the readers images and sentences!

4. Over-optimising generators is like solving a max-min problem instead. You showed your method is more robust in this case, can you explain it from the objective you use, e.g. the convex/concavity of your approach in general?

Typo: eq (3) should be min max I believe?

BTW I'm not an expert of NLP so I won't say anything about the quality of the NLP experiment.
review_title: Contains some interesting ideas

Meta-review: Training GANs to generate discrete data is a hard problem. This paper introduces a principled approach to it that uses importance sampling to estimate the gradient of the generator. The quantitative results, though minimal, appear promising and the generated samples look fine. The writing is clear, if unnecessarily heavy on mathematical notation.
Decision: Accept (Poster)