Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The submission proposes to train a GAN on discrete sequences using the straight-through Gumbel estimator introduced in Jang et al. (2016) in combination with gradient centering. The proposed approach is evaluated on COCO and EMNLP News in terms of BLEU and Self-BLEU scores, Fréchet Embedding Distance, Language Model Score, and Reverse Language Model Score.

My assessment is that the submission is below the acceptance bar, mainly due to clarity and novelty concerns. The proposed approach does have empirical backing, but I would argue that it is a very straightforward application of the straight-through Gumbel estimator to GANs, which is itself similar to existing work on applying the Gumbel-softmax estimator to GANs (Kusner & Hernández-Lobato, 2016). Detailed comments can be found below.

The submission does not feel self-contained. For instance, it borrows notation from Jang et al. (2016) without explicitly acknowledging it, and my personal experience is that reading Jang et al. (2016) beforehand makes a big difference in terms of clarity in Section 2.2.

The notation is inconsistent and confusing, and gets in the way of understanding the proposed approach. Here’s a (non-exhaustive) list of examples:

- The reward function is first introduced as f_\phi(\mathbf{x}) above Equation 3, but all subsequent mentions of the reward function use f_\phi(\hat{\mathbf{x}}).
- The \mathbf{m}_\theta variable is introduced in Equation 5 and is immediately replaced with \mathbf{p}_\theta, which adds notational overhead without any benefit.
- The difference between \hat{\mathbf{x}} and \hat{x} is not explained in the text. From the context I understand that \hat{x} is a categorical scalar in {1, …, V}; is this correct?
- In Equation 6, x_1, …, x_V are used to denote the *values* that \hat{x} can take. This clashes with the previous convention that \mathbf{x} is a sequence sampled from p_{data} (Equation 1). Given that convention and the difference between bolded and non-bolded variables discussed above, I would have expected that x_1, …, x_V would correspond to the categorical values of elements of the \mathbf{x} sequence. That contributes to confusion in Equation 9, where \mathbf{e}_{x_t} and p_\theta(x_t) are *not* time-dependent.
- Equation 8 sums over time steps, but the first summation that appears in Equation 8 does not make use of the temporal index. There is also a symbol collision for T, which is used both as the sequence length and as the "transpose" symbol.

As a result, the proposed centering method and the rationale for it is still not entirely clear to me. In particular, is the gradient centering approach necessary to avoid the drawback of score function-based approaches (i.e. the generator is only given feedback on the tokens it samples), or does the non-centered, straight-through variant of the proposed approach also avoid this drawback?

I’m also not convinced that the centering heuristic is a crucial component of the proposed approach when the biggest improvement observed over the straight-through baseline is obtained by adding spectral normalization. I would argue that the proposed approach is a straightforward application of the straight-through Gumbel gradient estimator to GAN training, which is similar in spirit to work by Kusner & Hernández-Lobato (2016) (not cited in the submission) -- the main difference being that the latter uses the Gumbel-softmax distribution directly and anneals the temperature parameter over the course of training. A comparison between the two would be warranted.

References:

- Kusner, M. J., & Hernández-Lobato, J. M. (2016). GANs for sequences of discrete elements with the Gumbel-softmax distribution. arXiv:1611.04051.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper attempts to solve the problem of non-differentiable connection between the generation and discriminator of a GAN. The authors come up with an estimator of the gradient for the generator from the gradient of the discriminator, which was disconnected previously. With this change, the model should be able to  select better tokens than random selection, which could leads to more robust training. The experiment results on both COCO Image Captions and EMNLP 2017 News datasets justify the authors' argument. 
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The authors propose CaptainGAN, a method using the straight-through gradient estimator to improve training of the generator for text generation.

The paper is well-written and the evaluation seems thorough, comparing to relevant baselines.

Comments:

Figure 3: the caption refers to Caccia et al. for results on LeakGAN, MaliGAN and seqGAN, but unless I’ve missed it, RelGAN hasn’t yet been introduced by name as a baseline? The citation is given in the opening part of the introduction, in an enumeration, but isn’t revisited later in the text - not even here where the results of the model are introduced. Given that it seems, according to the presented results, to be the most competitive of the GAN models that the authors are comparing to, maybe it’s worth adding more contextual information on RelGAN to the Background section?

For their method, the authors should report an average performance over several random seeds and provide the standard deviation / confidence intervals, for the readers to be able to assess the stability of the method and the significance of the improvement reported in the results.

I find Section 5.5. particularly interesting, as well as the reported perplexity in Table 2. The authors provide 3 bullet points to explain the unusually high perplexity of the generator on the training and validation data. I feel that the explanations that are given are at the moment vague and not visibly backed by data, therefore being speculative. Obviously, point 1) is hard to quantify - but point 2) could possibly be at least partially quantified - if the hypothesis is that names, places, punctuation marks etc play an important role in the reported perplexity score, then maybe the authors could test this by correlating model perplexity on sentences with whether those sentences contain these types of words?

review_title: Official Blind Review #3

Meta-review: This paper proposes a method to train generative adversarial nets for text generation. The paper proposes to address the challenge of discrete sequences using straight-through and gradient centering. The reviewers found that the results on COCO Image Captions and EMNLP 2017 News were interesting. However, this paper is borderline because it does not sufficiently motivate one of its key contributions: the gradient centering. The paper establishes that it provides an improvement in ablation, but more in-depth analysis would significantly improve the paper. I strongly encourage the authors to resubmit the paper once this has been addressed.
Decision: Reject