Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper provides a method (loss function) for training GAN model for generation of discrete text token generation. The aim of this loss method to control the trade off between quality vs diversity while generating the text data.

For example,
if original sentence is "The company ’ s shares rose 1 . 5 percent to 1 . 81 percent , the highest since the end of the year ." and the output is "The company ’ s shares rose 1 . 5 percent to 1 . 81 percent , the highest since the end of the year ." then the quality of generation is high and diversity is low. 

Pros:
1. The paper is very well written, with importance to the smaller details. It is a very good read for even people who are new to this problem. Especially, I appreciate the part where authors took efforts in writing why a few metrics are not used!
2. The motivation is good and also contributions are explicitly written. The details of the approach are provided clearly.
3. The experiments are provided in two different datasets and also the experiments support the two major claims in the paper.

Cons:
1. The primary concern with this submission is the novelty. The Proposition 1 of using forward-backward JSD based divergence has already been proposed in Li et al. (2019). Also, Li et al. (2019) proposes the entire contribution of this paper. The only difference is the introduction of \pi, which controls the percentage of labelled data to be considered between the generated data and original data. Basically, Li et al. (2019) is a specific case of this paper where \pi = 0.5. Thus, I would consider this paper as one additional experiment in Li et al. (2019) and not a whole paper as such.
2. Also, in the formulation in Eqn 2, the proposition 2 becomes a direct observation when \pi becomes 0 or \pi becomes 1. I would not call this as a proposition but a mere observation of Eqn 2.
3. Thus, taking away proposition 1 (already proposed in Li et al. (2019)) and proposition 2 (which is a mere observation) I do not find any novelty in this paper.
4. From an experiments perspective, Li et al. (2019) performed experiments in 4 datasets: Chinese Poems, MS COCO captions, Obama Speech, and EMNLP2017 WMT news. However, in this paper, results are shown in only two datasets - MS COCO captions and EMNLP2017 WMT news. Was it because that this paper was submitted in haste and/or the results in the other two datasets are not compelling enough to share?
5. The result analysis are poor - the authors have shown only the numbers in the tables, while the interpretation on these numbers and the discussion is left to reviewers discretion. Also, there are no generated examples that the authors are showing in either of the datasets. The authors should further discuss and analyze the results, show generated examples, and explain success and failure cases and the reasons behind them.

Overall, I find the novelty and the experimental analysis of the paper, very weak.
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: This paper introduces a GAN-based text generation approach, where the authors propose to directly optimize a weighted version of JSD replacing p_data with its empirical distribution. I find the theoretical analysis of the approach confusing, thus would like to get clarification from the authors. The experiments largely rely on automatic evaluation, which is known to be unreliable for text generation. I'd like to see human evaluation of the generated sentences, and at least some example outputs should be shown (even if it's cherry-picked). Given that both the theory and the empirical results are not solid in the current version, I intend to reject the submission.

=== Theoretical analysis ===
1. Main question: based on Equation 2, the optimal solution p_G^* is the empirical data distribution. It's unclear if p_G^* goes to the real data distribution when N goes to infinity.
2. Given the definition of the generalized JSD in Equation y, when \pi = 0 and \pi = 1, JSD_\pi is both 0. How does it control the balance between forward and reverse KL? I'm also wondering what's the connection between Proposition 2 and the interpolation between forward and reverse KL (which is implied in the text).
3. In the proof of Proposition 2, what is H? Also in Equation 8, second line, how does the second term disappear? Would be good to have complete proof in the appendix.

=== Empirical results ===
1. The description of NLL_test and NLL_oracle is very brief. Could you specify what are the language model and data used in each case?
2. In Table 2, all numbers are pretty close, are they significantly different? It would be really helpful to show some qualitative results as well.
3. Is there evidence that \pi is controlling the tradeoff between quality and diversity? From the experiments it's mainly controlled by the temperature.
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: This paper proposed to use KL and reversed KL as its new objective function for text generation GAN training.
However, this paper missed a lot important references. Basically, the authors only compare results with seqGAN and leakGAN. MaliGAN (https://arxiv.org/pdf/1702.07983.pdf), TextGAN (https://arxiv.org/pdf/1706.03850.pdf), etc. 
Also, KL + reversed KL training method for GAN framework is first proposed in Symmetric VAE (https://arxiv.org/abs/1709.01846), and the Proposition 2 basically are the same as the Symmetric VAE paper.

Therefore, I think this work is lack of novelty, and still need more time to work on.
review_title: Official Blind Review #2

Meta-review: This paper provides a method (loss function) for training GAN model for generation of discrete text token generation. The aim of this loss method to control the trade off between quality vs diversity while generating the text data.

The paper is generally well written, but the experimental section is not overly good: Interpretation of the results is missing; error bars are missing. 
Decision: Reject