Language GANs Falling Short
Authors: Massimo Caccia
Lucas Caccia
William Fedus
Hugo Larochelle
Joelle Pineau
Laurent Charlin
Venue:  (2020)
Keywords: NLP
GAN
MLE
adversarial
text generation
temperature

Abstract: Traditional natural language generation  (NLG) models are trained using maximum likelihood estimation (MLE) which differs from the sample generation inference procedure. During training the ground truth tokens are passed to the model, however, during inference, the model instead reads its previously generated samples - a phenomenon coined exposure bias. Exposure bias was hypothesized to be a root cause of poor sample quality and thus many generative adversarial networks (GANs) were proposed as a remedy since they have identical training and inference.  However, many of the ensuing GAN variants validated sample quality improvements but ignored loss of sample diversity. This work reiterates the fallacy of quality-only metrics and clearly demonstrate that the well-established technique of reducing softmax temperature can outperform GANs on a quality-only metric. Further, we establish a definitive quality-diversity evaluation procedure using temperature tuning over local and global sample metrics. Under this, we find that MLE models consistently outperform the proposed GAN variants over the whole quality-diversity space.  Specifically, we find that 1) exposure bias appears to be less of an issue than the complications arising from non-differentiable, sequential GAN training;  2) MLE trained models provide a better quality/diversity trade-off compared to their GAN counterparts, all while being easier to train, easier to cross-validate, and less computationally expensive.

Published as a conference paper at ICLR 2020
LANGUAGE GANS FALLING SHORT
Massimo Caccia"
Lucas Caccia"
Mila, Université de Montréal massimo P. caccialgmaillaacccccccccccmm com
Mila, McGill University
lucas page-aaeeeee. saaee--ccciammml.mmo. moci 11 ca
William Fedus
Hugo Larochelle
Mila, Université de Montréal
Google Brain, Montréal
Google Brain, Montréal
Mila, Université de Montréal
Canada CIFAR Al Chair
Joëlle Pineau
Laurent Charlin
Mila, McGill University
Mila, HEC Montréal
Facebook AI Research, Montréal
Canada CIFAR Al Chair
Canada CIFAR AI Chair
ABSTRACT
Traditional natural language generation (NLG) models are trained using maxi- mum likelihood estimation (MLE) which differs from the sample generation infer- ence procedure. During training the ground truth tokens passed to the model, however, during inference, the model instead reads its previously generated sam- ples phenomenon coined exposure bias. Exposure bias hypothesized to be a root cause of poor sample quality and thus many generative adversarial networks (GANs) were proposed as a remedy since they have identical training and infer- ence. However, many of the ensuing GAN variants validated sample quality im- provements but ignored loss of sample diversity. This work reiterates the fallacy of quality-only metrics and clearly demonstrate that the well-established technique of reducing softmax temperature can outperform GANs on a quality-only met- ric. Further, we establish definitive quality-dversiity evaluation procedure using temperature tuning over local and global sample metrics. Under this, we find that MLE models consistently outperform the proposed GAN variants over the whole quveity-diversity space. Specifically, we find that 1) exposure bias appears to be less of issue than the complications arising from non-differentiable, sequential GAN training; 2) MLE trained models provide a better quality/diversity trade- off compared their GAN counterparts, all while being easier to train, easier cross-validate, and less computationally expensive
INTRODUCTION
Generating fluent natural language is a central aim in Natural Language Processing (NLP). Trans- former architectures (Vaswani et al. (017)) with hundreds of millions or billions of parameters regu- larly reestablish state-of-the-an art on held-out validation perplexities, however, the generated samples can still often lack coherence. and inference procedures, is the root cause for poor sample quality. Therefore, numerous GANs (Goodfellow et al., (014)) for text generation were proposed since they do not suffer from exposure bias due training objective that directly seeks to improve sample quality (to a learned critic).
was hypothesized that exposure bias, differences in the training
While many of the early GAN variants demonstrated improvements in sample quality Yuu et al, 2017 Guo al. 2017). they often ignored loss of sample diversity. This is not only theoretical concern because it is well-documented that GANs exhibit mode collapse (Che et al.; 2016 Salimans
et al. 2016) where the generator produces subset of the "modes" in the training data, reducing
Authors contributed equally
Code reproduce experiments available biggtttuu.... wtt.hhub.cmmmccclcccpc. combbb.com/pcuuccsll/occ /aa.l//aaaaaalaaaaaaalaaaaaaalal Gansral lingsshotrtitt ingshrrt

---

Published as a conference paper at ICLR 2020
Diverslt
Diversity
Diversity
Quality
Quality
Quality
Figure 1: The importance of temperature for evaluating NLG models quality and diversity. Each sub-figure plots inverse quality against inverse diversity (lower is better for both metrics). Left: current way of comparing NLG models. In this case, it is impossible to come to any meaningful conclusions about which model (red or blue) dominates the other. Middle: With our proposed NLG evaluation framework, the temperature sweep shines light the relative performance of the models: the red model should be used for high-diversity samples and the blue model for high-quality samples. Right: A second simulated scenario (consistent with the left Figure) where the temperature sweep reveals that the blue model dominates the red. That is, for any desired diversity-level, there is temperature for which the blue model outperforms the red terms of quality (and vice versa).
diversity. Evaluating generated samples remains challenging (Liu et al. 2016 Zhao et al. 2017
but Cifka al. (2018) recommends assessing generated samples along two dimensions: 1) the quality of each sentence; 2) the diversity across sentences. However, this too is problematic because now the assessment relies on two measures which can make it difficult to compare models. Without further information about the relative trade-off between the two dimensions it is impossible to claim which is superior (Figure T. left subplot).
In this work, we build on the natural relationship between model's softmax temperature and the re- sultant quality-diversity trade-off: lower temperatures generate less diverse, higher-quality samples; higher temperatures increase the entropy of the distribution and produce more diverse, lower-quality samples. We propose a temperature sweep as a computationally efficient method to characterize the quality-diversity trade-off of different models. By explicitly controlling the temperature remove a potential source of bias (models may have different "implicit" temperatures) and obtain a more complete understanding of each model's behavior (Figure middle and right).
Additionally, we address that temperature modulation changes the entropy of the conditional dis- tributions, as opposed changing the entropy of the joint distribution. We explore other ways to navigate the quality-diversity space with less bias, including stochastic beam search and generator rejection. Although these methods provide small gains in the quality-diversity trade-off they have other computational limitations.
Despite the dizzying array of text-GAN variants and algorithms Yu al. 2017
Che et al. 2017
Lin et al., 2017; Zhang et al. 2017: Guo et al. 2017 Fedus et al. 2018 Lu et al.
201 Sa Shi et al.,
2018 Xu et al. 2018 Chen et al. 2018 Nie al., 2019 d' Autume et al.) 2019) our conclusions using temperature sweeps are clear: MLE models still dominate GANs. According to all metrics we studied, changing the temperature of MLE models at generation time leads to a better quality- diversity trade-off compared to GANs. This evaluation technique provides definitive boundary for future GAN research and our hope is that it will help the NLP community accurately assess natural-language generation progress.
2 ADVERSARIAL TEXT GENERATION
GANs are implicit generative models learned via a competition between a generator network Go and a discriminator network Dø- The generator network Ga produces samples from probability distribution Pmodet(x)) The discriminator (1) attempts to distinguish whether an input value is real (training data) or generated. Mathematically, the GAN objective can be formulated as minimax game
L = min mar En-peer Dlog (x) + EawGe [1 - log D&
(1)
2

---

Published as a conference paper at ICLR 2020
GANs were first proposed for continuous domains since their training procedure differentiates through the discriminator into the generator. However, modeling text, which is both discrete and typically modeled sequentially requires challenging adaptation for GANs which, in their orig- inal formulation, were built upon "one-shot" continuous data. Discrete (sequential) data require an alternative approach. Yu al. (2017) estimate the gradient to the generator via REINFORCE policy gradients (Williams) 1992). In their formulation, the discriminator evaluates full sequences. Therefore, to provide error attribution earlier for incomplete sequences and to reduce the variance of gradients, they perform l Monte-Carlo rollouts until the sentence is completed.
Yu et al. (2017) advertise their model using two tasks which we argue (with hindsight) are flawed. First, they introduce a synthetic evaluation procedure where the underlying data distribution P is known and can he queried. By representing P with an L.TTM (referred to as an oracle in the lit- erature), they directly compute the likelihood of samples drawn from generative model Go. The issue that they benchmark models against each other on this likelihood alone, i.e., the diagnostic entirely blind diversity. For example, model that always outputs the same highly likely sequence would easily outperform other potentially superior models. For real data, there was no agreed-upon metric to evaluate the quality of unconditional NLG the time. This led the authors propose new metric, Corpus-level BLEU, which computes the fraction of n-grams in a sample that appears in reference corpus. Again, this metric agnostic to diversity. Generating a single good sentence over and over will give perfect BLEU score.
This paper sparked stream of works that adopt their evaluation framework. Notably, RankGAN (Lin al.] (2017), MaliGAN (Che
0.7
FM-GAN
iet al.] 2017). ), TexIGAN (Zhang et al.) (2017).
LeakGAN (Guo al. (017) and IRL-GAN
0.6
(Shi et al. 2018) were proposed soon after.
6BLEU-5
0.5
LeakGAN
All but one of the aforementioned papers eval-
uate sample quality alone. As remedy. Zhu
0.4
al. (2018) propose metric that compares generated sentence with corpus of generated sentences, called Self-BLEU, They, along with Lu al. (2018)), provide extensive compar- ison of GANs using quality (negative BLEU) and diversity (Self-Bleu). However, it is not clear which algorithm is superior, as evidenced model simultaneously outperforms the other on both metrics. It is now standard for language GANs evaluate simul- taneously quality and diversity.
0.3
RankGAN, SeqGAN MallGAN
train data.
0.2
-0.35
-0.30
-0.25
-0,20
-0.15
-0.10
Negative BLEU-5
Negative BLEU-5 versus SBLEU- 5 (lower is better for both metrics) on the EMNLP2017 News dataset taken from (Lu et al.) 2018b) and this work (train data and FM-GAN). These scatter plots do not clearly show which al- gorithm is preferred since none strictly dominates on both metrics simultaneously.
Figure 2:
by Figure because
GANs Trained without RL Reinforcement Learning (RL) is often difficult to optimize, un- stable, and sensitive to hyperparameters. Be- cause of this, GAN-variants have recently been proposed that eschew RL-techniques in favor of fully-differentiable objectives. Cooperative train- ing (CoT) (Lu al. 2018a) and feature mover GAN (FM-GAN) Chen al. (0018 are notable approaches.
Research in both RL and non-RL GANs is still very active (c.g., Xu et (2118; Nie et al. (2019) Li et al. (001)) Bagnon-Marchang et al. (2099) and so our review is not
d' A'ttume et al. (2019) exhaustive.
TEMPERATURE SWEEP: TOWARDS ROBUST NLG EVALUATION
During natural language generation, a single spurious sampled token can lead to an overall low- quality and incoherent sequence. In other words, a high-entropy conditional distribution may result in poor sample quality at inference. To address this problem, one can modulate the entropy of Go | s1a-1) with Boltzmann temperature parameter (Ackley al. 1988). If or is the gen-
3

---

Published as a conference paper at ICLR 2020
erator's pre-logit activation and W is the word embedding matrix, then the conditional distribution of the generator is given by Gg | f1:t-1) = softmax W/a). Decreasing a below 1.0 will in- crease or and thus decrease the entropy of G's conditional probability. Temperature tuning naturally moves the model in quality/diversity space. We demonstrate this in Table
Samples
2.0
(1) If you go at watch crucial characters putting awareness in Washington, forget there are now unique developments organized personally then why charge.
(2) Front wants zero house blood number places than above spin 5 provide school projects which youth particularly teenager temporary dollars plenty of investors enjoy headed Japan about if federal assets own, at 41
1.0 (1) Researchers are expected to comment on where a scheme is sold, but it is no longer this big name at this point.
(2) We know you re going to build the kind of home you re going to expecting it can give better understanding of what ground test we re on this year, he explained
0.7 (1) The other witnesses are believed have been injured the police said in a statement, adding that there was no immediate threat to any other witnesses.
(2) The company net income fell to 5. 29 billion, or cents per share, on the same period last year.
0.0 (1) The company s shares rose 1. 5 percent to 1. 81 percent, the highest since the end of the year.
The company shares rose percent to 1. 81 percent, the highest since the end of the year.
Table 1: The effect of temperature on samples from an language model trained via MLE on the EMNLP17 News dataset. At a temperature of a 1.0 the samples are syntactically correct but often lack in global coherence. The sample quality varies predictably with temperature. At a > 1.0, the syntax breaks down and at a = 0.0 the model always outputs the same sequence. At a = 0.7 the samples are both of high quality and of sufficient diversity.
The evaluation protocol for NLG explained in Section 2 is compare models with respect both quality and diversity metrics. Often this results in a situation where it is impossible to tell which model is superior, as shown in Figure 2 and further exemplified in Figure [ (Left). One can then control the quality-diversity trade-off of autoregressive text generators using temperature. We can leverage this tool to design a new evaluation framework that shines a light on the real performance of each model. More precisely, we propose to generate samples at temperatures for each model in order to compute temperature curves in quality-diversity space. This is exemplified in Figure
We refer to this procedure as temperature sweep. This new way of evaluating NLG models allows practitioners to answer questions such as: which model to use if interested in high quality (or diversity) samples? Does new model improve upon others in the quality/diversity space or is it just reducing the entropy of the distribution? It could also be leveraged as a cross- validation tool e.g., early-stop once the best temperature curve is achieved according to heuristic.
In the next section we show, using temperature sweep, that MLE models consistently outperform the new proposed GAN variants everywhere in the quality- space. MLE performs equally on synthetic data to CoT and outperforms it on real data, whilst being computationally and algorithmi- cally less complicated. Our results are further validated in independent follow-up works (d' Autume ct al. 2019 Alihosseini et al. 2019 where temperature sweeps show MLE outperforming GANs.
To implement temperature sweep we have taken advantage of the fact that our model factorizes the joint distribution over an observation as a product of conditional distributions over single tokens given all previous tokens (this is a property of most autoregressive neural networks). We change the temperature of these conditionals, which is straightforward to implement. However, this is different from changing the temperature of the joint probability distribution. The samples generated this way are biased towards having lower entropy at the beginning of sentences compared to samples ob- tained by changing the entropy of the joint distribution. Changing the entropy of the joint distribution quickly becomes intractable with respect to vocabulary size and/or the total number of timesteps. We show below some entropy reducing techniques are less biased than changing the temperature of the conditionals and might achieve better quality-diversee trade-off. However, we argue that theses methods aren't 't suitable for our evaluation protocol due to computational inefficiency (see Section 5.4].
4

---

Published as a conference paper at ICLR 2020
Stochastic Beam Search is popular approximate decoding technique. It is computationally expen- sive but leads to higher- higher- likelihood samples than greedy decoding. Here, we focus on its stochastic version (not to be confused with the locally optimal one local beam search). In Stochastic beam search with beam size k; the most likely hypotheses (sampled so far) are kept at every decoding steps. In Appendix [ we detail this technique and explain how it is less biased than temperature tuning regarding the reduction of the joint distribution's 's entropy. Finally, the more traditional local (deterministic) beam search removes all sample diversity and is not useful in our setting.
Generator Rejection Sampling works as follow: generate some sentences; compute their likeli- hood under the generator's own distribution; accept/reject the sample given threshold. The thresh- old enables the practitioner to modulate the quality- trade-off. In Appendix [B] we explain how it is less biased than temperature tuning, why it is computationally expensive, and how the discriminator can be used in rejection sampling.
4 RELATED WORK
Concurrent with our work, SerneiSeeeiita et al. (2018) demonstrated the issues of local n-gram metrics. Their extensive empirical evaluation of GAN models and language models (LM) did not result in evidence of GAN-trained models outperforming on the new and improved global metrics from Cífka et al. (2018). Our analysis further explores this path by examining the performance of these models under sweep of temperatures. We believe this difference to be of utmost importance, as is the necessary ingredient towards definitively showing MLE models outperform current GAN variants on quality-diversity global metrics. Work from Ott et al. (2018) thoroughly examines the local beam search strategy for neural machine translation. In their analysis, the authors compare local beam search and generator rejection sampling and find the beam search is quite effective at finding high- likelihood regions. However, their work focuses on conditional text generation, where quality-only metrics measures performance.
Guo al. (0177) suggest that increasing the temperature at training time leads to more diverse samples. However, we argue that this procedure leads to the opposite outcome as a model can adapt to the temperature change. This would have the net result of lowering the entropy at test time. We discuss this issue and propose study in Appendix F
5 EMPIRICAL RESULTS
Using our evaluation approach (Section 3). we examine several recent GAN text generation mod- els and compare against an MLE baseline. The experiments consist of two parts: synthetic data generation and long-text generation. We provide strong empirical evidence for both types of data that MLE trained models reliably outperform textual GANs in the quality-diversity space. For these experiments, we only use temperature tuning, as is the only technique that gives a smooth control over said trade-off whilst being computationally efficient (see Section 5.4 5.4
5.1 EXPERIMENTAL DETAILS
The ever-growing body of RL trained language GANs makes writing, running hyperparameter searches and cross-validations on all the variants prohibitively expensive We implemented our own language GAN, including improvements/unccioaaliee proposed several text GAN papers. We ensure that each functionality can be toggled during training. Then for each dataset, we ran hyperparameter search of 300 trials encompassing all possible combinations of said functionalities. We refer to this model as RL-GAN. It is based on SeqGAN (Yu et al.) 2017) with additional im- provements shown to be useful including: MLE pretraining (Yu al.] 2017), leaky discriminator (Guo et al. 2017), step-level loss instead of sequence level (Fedus al. 2018), Icarned bascline to reduce variance (Fedus et al., 2018), regularizing REINFORCE with maximum-entropy loss Williams & Peng 1991) and alternating adversarial loss with MLE loss (Guo et al. 2017).
2
We found the majority of the official implementations be prone extreme mode collapse thus making it quite hard to reproduce reported results.
5

---

Published as a conference paper at ICLR 2020
Through comparisons with previously reported results and results obtained via running the official repositories. we show that RL-GAN is the state-of-the-ait RL trained GAN. Our MLE model is trained with the same codebase (adversarial training turned off) and only one-sixth of the trials. Moreover, we report SeqGAN and LeakGAN results in the synthetic data experiment. These were obtained with the official repositories that modified in order to obtain samples different tem- peratures. Finally, the SeqGAN, LeakGAN, RankGAN, and MaliG AN results in Figure 4a are taken from (Lu et al. 2018b), which was written by the authors of SeqGAN and LeakGAN
Finally, we conducted experiments differently for non-RL GANs. The CoT (Lu et al. 201 results are obtained with the official repository via careful hyperparameter search guided by discussion with the authors. The FM-GAN (Chen et al., (2018) results were obtained with the best performing model that was provided by the authors.
Selecting temperature sweep range. We describe how we selected the range of temperatures in the temperate sweeps. We found that in practice, both the quality and the diversity axes induce bounds beyond which results are non-informative. On the diversity dimension, we found no value in increasing the entropy of the MLE model because minimizing the forward KL does not lead to an underestimation of the real data entropy. Hence, we decreased the temperatures of the other models until a similar diversity to the MLE model was reached. On the quality dimension, we decreased the temperatures to a 0 but did not report the complete curves for the following reasons. First, with synthetic data (Figure 3) the NL explodes when diversity is decreased too much, so we reduce the temperature of the MLE model until the difference in performance compared to the GAN results was evident. Second, in the real data in Figure 4b] we stopped decreasing the temperature when the models achieved a Reverse LM score equal to the perplexity achieved by I-gram (unigram) model. This means that the generated dataset is almost non-informative for the (Reverse) language model to learn on i.e., is as informative as counting tokens. This also coincides with severe mode collapse and temperature of <0.5 for the MLE model. In Figure [4a] we once more decreased the temperature until the difference in performance of MLE with respect to the other models is unambiguous.
12
Model
NLLoracle
CoT
SeqGAN Yu al. 2017)
8.74
SeqGAN
11
RankGAN Lin et al.
2017
8.25
LeakGAN
LeakGAN (Guo
al. 2017)
7.04
RL-GAN
IRL (Shi et al. 2018)
6.91
MLE
MLE (a = 1.0)
9.40
NLL
MLE (a- 0.4)
5.50
MLE (a = 0.001)
4.58
8
Table 2: NLLorade measured on the synthetic task (lower is better). All results are taken from their respective papers. An MLEE-rained model with reduced temperature easily improves upon these GAN variants, producing the highest qual- ity sample.
10
11
NLL oracle
Figure 3: Effect of temperature tuning the global metrics (lower is better for both metrics) for the synthetic task.
5.2 SYNTHETIC DATA EXPERIMENT
In the synthetic experiment, we learn a generative model of data produced from a fixed LSTM oracle Yu et al. (2017) with hidden dimension of 32 with parameters drawn from a standard normal distribution. This allows us to compute a perfect quality metric, the likelihood under the Oracle NLLorade. In Table 2 we see that artificially reducing the temperature at inference achieves state- of-the-art as evaluated by the NLLoraate- As and others have argued, evaluating quality alone is misleading. The MLE-trained model with extremely low temperature will repeatedly output only a single sequence. It is therefore essential to evaluate the resulting sample diversity which we evaluate using the likelihodd that a generator assigns to held-out data (N L We report the result of temperature sweep in Figure

---

Published as a conference paper at ICLR 2020
6.5
0.7
FM-GAN RL -LAG
6.0
0.6
MLE
Reverse LM score
SBLEU-5
0.5
LeakGAN
5.5
5.0
0.4
FM-GAN
CoT
4.5
0.3
RankGAN
RL-GAN
train data
SeqGAN
MailGAN
4.0
MLE
-0.35
-0.30
-0.25
-0,20
-0.15
-0.10
1.5
2.0
2.5
3.0
3.5
Negative BLEU-5
LM score
(a) Local metrics
(b) Global metrics
Figure 4: Results on the EMNLP 2017 News dataset. (lower is better for all metries). MLE under temperature sweep achieves better quality-diversity trade-off compared to the GAN approaches.
Our RL-GAN benchmark is superior to the official SeqGAN and LeakGAN implementations. Nonetheless, MLE outperforms GANs everywhere in the quality- space. [Semeniuta et al. 2018) suggest that the best-performing GANs tend to stay close to the solution given by maximum- likelihood training. We find support for this conclusion, as the best performing RL-GAN models have the smallest learning rate and a considerable amount of pretraining. CoT achieves similar performance to MLE, but with dramatically increased algorithmic complexity. This is unsurprising as their objectives are somewhat similar (see Section 4.. Alihosseini et al. (2019) replicated this experiment on synthetic data and arrived at the same conclusions.
5.3 LONG-TEXT GENERATION
Next, we study long-text generation using EMNLP News 2017. We first compare an MLE model to the reported GAN results the local metrics Negative BLEU and Self-BLEU. Negative BLEU5 and SBLEU-5 are used for Figure 4a and results for (Self- BLEU-2 to in Appendix D Again, the conclusions are the same: MLE outperforms all RL GANs considered in the quality-diversity trade-off. Moreover, MLE outperforms FM-GAN, the state-of-the-art non-RL. Language GAN. Note that the FM-GAN results were obtained with combination of temperature tuning and noise reduction in order to achieve the best possible results. We provide additional details in Appendix H We compare MLE GANs using recently proposed global metrics, the Language Model score (quality) and Reverse Language Model score (diversity+tuualit)) (Citka et al.) 2018 Semeniuta et al. 2018). See Figure 4b for comparison of the training scheme in quality-diversit space. The conclusion is the same as with BI.EU and Self-BI Self-BIFUU MLE outperforms all GANs everywhere the quality-diversity space. Further results on this dataset as well as on the IMDB movie reviews (Maas et al.) (201)) and Wiki'Text- 103 (Merity et al. (0066) datasets are presented in independent follow-up works (d' (d'Autume et al., 22119] Alihosseini et al. 2019) where MLE is also shown to outperform GANs using temperature sweeps.
are reported
5.4 EMPIRICAL REMARKS
These findings are illustrative: a body of research is predicated exposure bias as the culprit for poor sample quality. Therefore, under this hypothesis, MLE should at its weakest on long-text generation tasks. However, results are evidence that exposure bias is less of an issue than opti- mization problems arising from GAN training combined with the con-differentiability of the original objective function. There is also another way to interpret these results. It seems that MLE (pre- Jtraining leaves the generator with a better policy according to quality-diversity. However, because GAN training removes entropy from the learned distribution (see Fig. 5 in Appendix ??), which results in high-quality samples, it can lead one to believe that GAN trained models are superior.
7

---

Published as a conference paper at ICLR 2020
Temperalure Gonerator Rejection
10"
Temperalute Generator Reyection
6.0
Beam Search
Beemn Search
Decoding Time
5.5
Reveree LM
10*
5.0
10'
4.0
1.5
3.0
3.5
LM score
LM score
(a) Diversity and quality
(b) Decoding time and quality
Figure 6: Analysis of decoding methods. (lower better for all metrics). Left: Less biased methods provided better quality/diversity trade-off. Right: However, they are computationally much more expensive.
Methods to approximately reduce the entropy of the joint distribution We analyze the differ- ent decoding mechanisms as tools to move in the quality-diversity space and as an alternative to temperature sweep. We evaluate these tools by decoding from the best performing MLE model on the E.MNLP2017 News dataset. We also consider the different properties of these tools, including their ability to provide smooth control over the quality-diversity trade-off and their computational efficiency. The purpose of this experiment is thus not to find the best performing strategy i.e., a model combined with a decoding method but rather to compare decoding methods. Further, a com- parison between MLE and RL-GAN using these approaches is in Appendix [ and it supports our prior results.
Figure 6a compares three methods for navigating the quality (Language Model, LM) diversity (Re- verse I.M) space: temperature sweep, generator rejection and beam search. We first note that temperature tuning and generator rejection sampling yield a similar trade-off in the high-diversity regime. However, as the bias incurred by temperature tuning grows, so does the gap between both methods. An important finding is that generation rejection sampling gets exponentially slower the threshold increases (see Figure (6b). It is for this reason that the generator rejection sampling curve doesn't span further in the high-quality part of the space.
Beam search, a less biased method, also seems to of- fer better trade-off compared temperature sweep. However, beam search has a major drawback: unitary increases in beam size leads to drastic leaps in quality- diversity space. In Figure 6a we report beam sizes of (right dot). 2 (middle dot), and (left dot). This study shows that although the less biased methods seem to offer at least small gains in the quality-diversity space, temperature tuning still has important advantages to assess a model's quality-diversie The discreteness of beam search and the computational inefficacy of generator rejection sampling are important drawbacks. However, if one's goal is obtain the best samples possible, we advocate for their use.
NLL 5
Entropy 2
trade-off efficiently.
20
40
60
Epochs
Figure 5: Dotted line indicates the start of GAN training. We notice a clear drop in entropy (spike in NLL when moving from maximum-likelibood to adversarial updates.
GAN Training Reduces Entropy Figure 5 shows the evolution of generator's 's entropy with respect to epochs. We observe that as soon as adversarial training starts, a drastic entropy reduction occurs. Future work investigates if GAN training changes the learned distri- bution or simply reduces the entropy in an ineffective way with respect to the quality-diversiy trade-off.
For computational reasons, generator rejection sampling does not span further in the high-quality regime.
8

---

Published as a conference paper at ICLR 2020
6 DISCUSSION
We demonstrate that well-adjusted language models are a strong baseline and that temperature sweeping can provide unambiguous characterization of model performance in terms of quality and diversity. A well-adjusted language model outperforms the considered GAN variants as evalu- ated on both local, and more surprisingly, global metrics of quality and diversity. Our temperature sweeping framework shares characteristics with Receiver Operating Curve. Analogously, if one needed single scalar compare NLG models, one could compute the area under the curve and seek the model with the smallest value (lower is better for our considered metries).
GAN-based generative models have been proven effective on real-valued data. However, there exist many difficult pernicious issues involved in moving to discrete data. These issues must be overcome before they will improve over the strong MLE baselines for unconditional language generation. On the datasets and tasks considered, potential issues caused by exposure bias were less severe than the ones arising from adversarial training combined with learning the non-differntiiabiliyy of the original objective function. GAN training may prove fruitful eventually, but this research lays forth clear boundaries that it must first surpass.
REFERENCES
David H. Ackley, Geoffrey E. Hinton, and Terrence J. Sejnowski. Connectionist models and their implications: Readings from cognitive science. chapter A Learning Algorithm for Boltzmann Machines, pp. 285-307. Ablex Publishing Corp., Norwood, NJ, USA, 1988. ISBN 0-89391-456-
8. htthttt:: fl. acm. org/ citation. www.ttio....miiddaeeee.ee......e
54465
Danial Alihosseini, Ehsan Montahaei, and Mahdieh Soleymani Baghshah. Jointly measuring di- versity and quality in text generation models. Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation, pp. 90-98. 2019.
Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li. Mode regularized generative adversarial networks. arXiv preprint arXiv:1612.02136, 2016.
Tong Che, Yanran Li, Ruixiang Zhang, R Devon Hjelm, Wenjie Li, Yangqiu Song, and Yoshua Bengio. Maximum-likelihon augmented discrete generative adversarial networks. arXiv preprint 170.00777, 2017.
Liqun Chen, Shuyang Dai, Chenyang Tao, Dinghan Shen, Zhe Gan, Haichao Zhang, Yizhe Zhang, and Lawrence Carin. Adversarial text generation via feature-mover's distance. arXiv preprint arXiv:
arXiv:/809.000 (00..889999999 2018.
Ondřej Cífka, Aliaksei Severyn, Enrique Alfonseca, and Katja Filippova. Eval all, trust a few, do wrong to none: Comparing sentence generation models. arXiv preprint 1XXiv:880.00772) 2018.
Cyprien de Masson d' Autume, Mihacla Rosca, Jack Rae, and Shakir Mohamed. Training language gans from scratch. arXiv preprint arXiv:1905. 09922, 2019.
William Fedus, Ian Goodfellow, and Andrew M Dai. Maskgan: Better text generation via filling in the arXiv preprint arXiv:/801.077 077367 2018.
Jules Mignon-Marchande Hamed Sadeghi, Md Akmal Haidar, and Mehdi Rezagholizadeh. Salsa- text: self attentive latent space based adversarial text generation. In Canadian Conference on Artificial Intelligence, pp. 119-131. Springer, 2019.
Ian Goodfellow, Jean Aouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Yong Yu, and Jun Wang. Long text generation via adversarial training with leaked information. arXiv preprint arXiv: 1709 08624, 2017.
Zhongliang Li, Tian Xia, Xingyu Lou, Kaihe Xu, Shaojun Wang, and Jing Xiao. Adversarial discrete sequence generation without explicit neuralnetworks as discriminators. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 3089-3098, 2019.

---

Published as a conference paper at ICLR 2020
Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming- Sun. Adversarial ranking for language generation. In Advances in Neural Information Processing Systems, pp. 3155-3165, 2017.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Chia-Wei Liu, Ryan Lowe, Julian Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. How not to evaluate your dialogue system: An empirical study of unsupervised eval- uation metries for dialogue response generation. arXiv preprint 1oo::66030022 2016.
Sidi Lu, Lantao Yu, Weinan Zhang, and Yong Yu. Cot: Cooperative training for generative modeling. https://arxxv.org//df// wtt.////88440.3. 8040038.... 2018a.
Sidi Lu, Yaoming Zhu, Weinan Zhang, Jun Wang, and Yong Yu. Neural text generation: Past, present and beyond. arXiv preprint htt3:/800.01/// 2018b.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies Volume I, HLT '11, pp. 142-150, Stroudsburg, PA, USA, 2011. Association for Computational Linguistics. ISBN 978-1-99 99911-9-22-2---- URL http: dl cm http://ctt..ooooooeoee222222222. 20C2491
cfm?
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. ArXiv, abs/1609.07843 2016.
Weili Nie, Nina Narodytska, and Ankit Patel. RelGAN: Relational generative adversarial net-
works for text generation. In International Conference on Learning Representations, 2019. URL
btttps://openrett
net/ wtt.://wwww..eetooooeeeeeeeeeeee
Myle Ott, Michael Auli, David Granger, and Mare' Aurelio Ranzato. Analyzing uncertainty in neural machine translation. arXiv preprint arXiv: 0803/003000000 2018.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pp. 2234 2242, 2016.
Stanislau Semeniuta, Aliaksei Severyn, and Sylvain Gelly. On accurate evaluation of gans for lan- guage generation. arXiv preprint arXiv: (40...9600.. 2018.
Zhan Shi, Xinchi Chen, Qium Xipeng, and Xuanjing Huang. Towards diverse text generation with pdf. 2018.
inverse reinforcement learning. https://arzix.org/////1/111....
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Reinforcement Learning, pp. 5-32. Springer, 1992.
Ronald J Williams and Jing Peng. Function optimization using connectionist reinforcement learning algorithms. Connection Science, 3(3):241-268, 1991.
Jingjing Xu, Xu Sun, Xuancheng Ren, Junyang Lin, Binzhen Wei, and Wei Li. Dp-gan: Diversity- promoting generative adversarial network for generating informative and diversified text. arXiv preprint arxiv:1802.000 0055.013... 2018.
Lantao Yu, Weinan Zhang. Jun Wang, and Yong Yu. Seggan: Sequence generative adversarial nets with policy gradient. 2017.
Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, and Lawrence Carin. Adversarial feature matching for text generation. arXiv preprint arXiv: /tt.://66.....8 2017.
10

---

Published as a conference paper at ICLR 2020
Junbo Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M Rush, and Yann LeCun. Adversarially regularized autoencoders for generating discrete structures. CoRR, abs/1706.04223, 2017.
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models, SIGIR, 2018.

---

Published as a conference paper at ICLR 2020
A STOCHASTIC BEAM SEARCH.
Beam search is popular approximate decoding technique. It is computationally expensive but usually leads to higher-likelihodd samples than greedy decoding. Here, we focus on its stochastic version, not to be confused with the locally optimal one, Local Beam Search. In Stochastic Beam Search with beam size k, words are sampled sequentially starting from the first one. At the first timestep & words are sampled conditioned on each of the & current hypotheses, resulting in k22 hypotheses. The most likely are kept and so on. Because an increase in beam size results in higher-likelihood sam- ples, beam size can be leveraged to modulate the quality beam size would result sampling the most likely sentences under the generator's distribution. This is not what we can expect from setting the temperature to O, which is equivalent to greedy decoding. For this reason, we can hypothesize that Stochastic Beam Search less biased regarding the reduction of the joint distribution's 's entropy. However, unlike temperature tuning, one cannot smoothly trade quality for diversity, as the beam size a discrete parameter. The more traditional local (deterministic) beam search removes all sample diversity. Thus, it is not useful in our setting.
sampled, each represents hypothesis. At subsequent timesteps, k next words
trade-off. Moreover, an infinite
B GENERATOR REJECTION SAMPLING.
Generator Rejection Sampling works as follow: generate some sentences; compute their likelihood under the Generator's 's own distribution; accept/reject the sample given a threshold. The threshold en- ables the practitioner to modulate the quality-diversity trade-off. Similar to Stochastic Beam Search, increasing the threshold to a maximum level would result in always sampling the most likely sen- tence. We can again hypothesize that this method achieves a better quality-divenyy pared to temperature tuning, since is not biased towards having sentences with lower entropy prefix. However, the running time of this procedure increases significantly as the acceptance thresh- old lowers, as discussed Section 5.4 We also experimented with a rejection sampling scheme where the score is determined by discriminator but it did not provide any additional insights.
trade-off com-
C QUALITATIVE ANALYSIS OF THE SAMPLES
For completeness, we show samples of SeqGAN, LeakGAN, and MLE from both datasets in Table 3 In this experiment, we used the Image COCO dataset (Lin al. (014) in order to generate shorter sentences as well. In this case, the samples of all models appear similar. This is because generating captions is a relatively easier task. For the EMNLP2017 News dataset, we would like point out interesting facts from our samples. In our first sample (1), the model generates the sequence "post Brexit strategy". This is n-gram is not present in the training set. our second sample (2). the token "leak" and the n-gram "Freedom of Information request" never appear together in the training dataset. The extrapolations show that the model learns a certain level of generalization beyond the training set. Additional samples presented further in Appendix!]
D FULL BLEU AND SELF-BLEU RESULTS
Full BLEU and Self-BLEU results are shown in Table 4
E THE LIMITATIONS OF BLEU
In this section, we want to highlight an important flaw in using BLEU as a proxy for quality. We tuned the temperature in order to find a MLE model with BLEU score equal to the training data's. 's. We show three randomly sampled sentences from the model in Table Although sometimes gram- matically correct, the samples lack in semantic and/or global coherence. It seems the generated text has poor information content. Surprisingly, in order to get great samples on a consistent basis, the temperature needs to be reduced to level where BLEU-5 is twice large the training data' 's Thus, it scems like BLEU is not always a good proxy of sample quality. Again, we think it is of utmost importance develop better metrics and modernize NL.G's canonical evaluation framework.
12

---

Published as a conference paper at ICLR 2020
Datasets
Image COCO
EMNLP2017 News
SeqGAN
(1) A woman is riding bike on the street next to bus.
(1) You only certainly might not rush it down for those when they were the heads, and when she's name.
circumstances where
(2) A silver stove, the refrigera- sitting in kitchen.
(2) think you should really really leave for because we hadn't 't been busy, where goes to one," he wrote.
LeakGAN
(1) A woman holding an um- brella while standing against the sidewalk.
(1) A man has been arrested at age 28, a resident in Seattle, which was widely reported in 2007.
(2) A bathroom with toilet and sink and mirror
(2) also think that's a good place for us, I'm sure that this would be good opportunity for me to get in touch. (1) The company will he to provide post Brexit strategy which will be published in the coming weeks
MLE
(1) A narrow kitchen with wooden cabinets and white appliances.
(2) There are several bikes parked in front of a tall building with four cars
(2) leak was obtained by a Freedom Informa- tion request, which is based on the number of people claiming to be victim of fraud.
Table 3: Samples from the different models on Image COCO and EMNLP2017 WMT News. For SeqGAN and LeakGAN, samples were taken from (Guo et al. 2017). It's the first two samples found in their appendix. For our samples, we reduced the temperature of the model till we achieved similar BLEU scores the ones reported in (Guo al. 2017) order keep comparison fair.
BLEU
Self-BLEU
2
4
2
4
Training Data
0.86
0.61
0.38
0.23
0.86
0.62
0.38
0.24
SeqGAN Yu al. (2017
0.72
0.42
0.18
0.09
0.91
0.70
0.46
0.27
MaliGAN
Che al.
2017
0.76
0.44
0.17
0.08
0.91
0.72
0.47
0.25
RankGAN
Lin ct al. (017))
0.69
0.39
0.18
0.09
0.90
0.68
0.45
0.30
TextGAN
Jhang et al. (2017
0.21
0.17
0.15
0.13
1.00
0.98
0.97
0.96
LeakGAN
Guo et al.
2017)
0.84
0.65
0.44
0.27
0.94
0.82
0.67
0.51
MLE (a= 1.25-*)
0.93
0.74
0.51
0.32
0.93
0.78
0.59
0.41
Table 4: BLEU (left) and Self-BLEU (right) on test data of EMNLPNEWS 2017.
(Higher BLEU
and lower Self-BLEUU is better).
BLEU
Self-BLEU
2
4
2
3
4
5
Training Data
0.74
0.53
0.34
0.22
0.90
0.75
0.58
0.42
ScqGAN Yu al. (2017)
0.75
0.50
0.29
0.18
0.95
0.84
0.67
0.49
MaliGAN Che et al.
2017
0.67
0.43
0.26
0.16
0.92
0.78
0.61
0.44
RankGAN Lin al. 2017)
0.74
0.47
0.26
0.16
0.96
0.88
0.76
0.62
TexIGAN
Zhang et al.
(2017
0.59
0.46
0.28
0.21
0.94
0.93
0.80
0.75
LeakGAN
Guo et 2017)
0.74
0.52
0.33
0.21
0.93
0.82
0.66
0.51
MLE
0.74
0.52
0.33
0.21
0.89
0.72
0.54
0.38
Table 5: BLEU (left) and Self-BLEU (right) on test data of Image COCO. (Higher BLEU and lower Self-BLEU is better).
F ISSUES OF VARYING TEMPERATURE DURING TRAINING
As penultimate experiment, we analyze the effect of changing the temperature at training instead of inference. [uoo et al.) (2017) suggested that increasing the temperature at training time leads to more diverse samples. However, model can adapt to the temperature change. This would have the net result of lowering the entropy at test time. To examine this, we trained 30 GANs maintaining everything constant except training temperature. Negative BLEU-5 against SBLEU-5 are plotted in Figure 7 The darker the dot, the higher the and consequently the temperature. As hypothesize, models trained with increased
argue that this procedure leads to the opposite outcome
13

---

Published as a conference paper at ICLR 2020
MLE
(1) He explained that the Government s plan to cut tax on unemployment was
3 percent lower than forecast for the first increase of 16 percent in 2015, the fastest rate the state since 2004.
(2) On the policy it no than the amount of money we have of the decades and Senate of our assets
1.05-1
(3) They say was possible supporting the Scottish government make the changes as secret free environment based on competition.
Three randomly sampled sentences from our model with closest BLEU scores to the training set's. The sentences have poor semantics global coherence. They are also not perfect grammati- cally speaking.
Table 6:
temperature at training time adapted to the change and the net result was a colder temperature at inference (hence reduced diversity). We therefore recommend only adjusting the temperature inference. One should consider other techniques to facilitate exploration during training.
3.00
1.0
2.75
2.50
SBLEU-5
2.25
2.00
0.6
1.75
0.4
1.50
1.25
022
1.00
-0.5
-0.4
-0.3
-0.2
-0.1
Negative BLEU-5
Figure 7: Negative BLEU-5 on test data against SBLEUS for models with different temperature applied at training time. The redder the dot, the higher the i.e. more pressure to increase entropy. From these results, it evident that one should not increase the temperature training time as the models adapts and the net results is mode collapse.
G DISCRIMINATOR REJECTION SAMPLING
We experimented with rejection sampling technique base on the Discriminator's signal. For this method, we train a Discriminator on the Generator's s samples without ever updating the Generator. Next, we accept/reject samples in similar fashion to Generator Rejection Sampling however now the score is determined by the Discriminator. The higher the threshold, the more confident the Discriminator needs to be about the realness of the data. This should increase the quality of the samples with the usual downside of reducing diversity. We show results with Discriminator having the same architecture as the Generator (Discriminator Rejection) and the best Discriminator found with an hyper parameter search of 100 trials (Best Discriminator Rejection. Results are shown in Figure Note that the domain of the figure is much smaller that in Figure 4b The reason is that this approach can't move the models further in quality-diversity space. For this reason, we do not advocate for the use of this approach as means to obtain high quality samples.
H MODULATING QUALITIT/DDIVESSINN FOR FM-GAN
For FM-GAN, we observed that lowering the temperature to O didn't completely remove all the stochasticity in the generations. This is because FM-GAN samples noise prior to sampling the first word. This is similar to how image generation GANs are trained. Precisely, 2 ~ N(0, 1) is sampled and concatenated to every token of generated sentence. Because temperature tuning wasn't 't covering enough of the quality/diversity space, we also reduced the variance of the noise as quality modulating tool. Results show in Figure
14

---

Published as a conference paper at ICLR 2020
4.30
Temperature Discriminator Rejection
Reverse LM score
4.25
Best Discriminator Rejection
4.20
4.15
4.10
4.05
4.00
2,6
3.0
3.2
3.6
LM
Figure 8: Discriminator Rejection Sampling is not great tool to navigate in quality-diversity space.
FM-GAN (temp)
FM-GAN (naise)
0.8
SBLEU-5
0.4
-0.40
-0.35
-0.30
-0.25
-0.20
-0.15
Negative BLEU-5
Figure 9: Different strategies to modulate the quality/diversity trade-off in FM-GAN.
I RL-GAN VERSUS MLE WITH OTHER DECODING MECHANISMS
MLE Generator Rejection
Reverse LM score
RL-GAN Generator Rejection
MLE beam
RL-GAN beam
1.5
2.0
2.5
3.0
3.5
LM score
Figure 10: MLE still outperforms RL-GAN under different decoding mechanisms.
15

---

Published as a conference paper at ICLR 2020
J
EMNLP 2017 NEWS SAMPLES
We present additional samples for EMNLP 2017 News dataset.
EMNLP2017 News
SeqGAN
You only certainly might not rush it down for those circumstances where are when they were the heads, and when she name think you should really really leave for because we hadn been busy, where it goes to one, he wrote.
All the study knew was that they are so they continue to provide support service and doesn exist.
can say become up with nothing sales have reached the charge for the other any evidence that been virtually well below the $ 800
Three times before the start of the season is much early on 2015 we are in the third training every year.
That S the idea of strength that decision they said, we haven already lost four or seven, Liverpool team
That is not the time for the cost of changing the system and it was pushing for 20 million
We had to take it good day for a military but nearly 000 ] and prepare for them through.
actually didn tell the background check the difference after my hour was to be recalled and it was great We are thinking about 40, 000 and jobs in what is wrong in the coming and you know That is out how working you can t set out some pretty tight or what going through
wanted to be made you decided to have a crisis that way up and get some sort of weapon not much to give birth to for American room
She had been fined almost 200, 000 with couple of asylum seekers in Syria and Iraq Perhaps not, in looking for housing officials would help the frustration of Govermment, with an FBI shortly before 2020
Once got real show for the young since I sure she went to love it just whether to be late later last year.
But after holiday period we might have to go on total out debate like that could have happened to us.
Table 7: Samples from SeqGAN taken from Guo et al. 2017)
16

---

Published as a conference paper at ICLR 2020
EMNLP2017 News
LeakGAN
A man has been arrested at age 28 a resident in Seattle, which was widely reported in 2007
also think that s good place for us I m sure that this would be a good opportunity for me to get in touch.
What is the biggest problem for Clinton is that Donald Trump will in the race and he unlikely be the nominee.
We re going to do and we re going to put it out and get the ball, he said.
would be afraid to blame the girls go back but I was just disappointed with the race, he said.
not going to work together with a different role and we can win the game, he added
The couple s lives are still missing and they have been killed the city s way to play against them, and because I came out there. For the last three years, we ve got lot of things that we need to with this is based on the financial markets.
Don t ask me but know. if be able to be out of Hillary Clinton, think being made for the Congress.
am proud to be able to move forward because we don have to look at about, he said That why
the most important people for the African American community and we ve made good response
But the move will be only in fight against them, as well likely to prevent agreement to remain in the EU
The American Medical Association said that the militants had been arrested in connection with the murder of the same incident.
The two year old girl has been charged with a suspect who was in the vehicle to the police station. It is hard to buy on the Olympics. but we probably don t see a lot of it.
not going to be very proud of the other countries, he said. He said the U. N intelligence industry will not comment on the ground, which would be sensitive to the European Union take my work in the days. but I would have to go down on Wednesday night.
Table Samples from LeakGAN taken from Guo et (017))
17

---

Published as a conference paper at ICLR 2020
EMN1P2017 News
MLE
The UN Security Council is major concern for the U. S. government, well as NATO ally in the Syrian civil war A spokesman for the Met Office said the death toll was only slightly higher than the previous year, according to the report
But I hope that at the end of the day I going to give her the best chance go to the gym and go out and play.
The man, who cannot be named, said that he had never had sex with him, and he didn want to see him
And it just of those things that have to say conservative.
Democrat, and I The bank is now the fastest growing market in the world and it significant change in the economy.
The two men aged 20 and 22, were arrested and charged with the murder of man police officer.
The company will be able to provide post Brexit strategy which will be published in the coming weeks, She said she had been on the wrong side road and was finally caught a car accident and was taken to hospital.
I don t think he s even good player, he said, but he s got a good chance to win the game
don know what the future holds but I m sure will be good thing. It very important step forward, and we re going be able get the right results. The driver of the vehicle, who was inside the vehicle, was taken to hospital for treatment, but said he not aware of the incident.
The leak was obtained by a Freedom of Information request, which is based on the number of people claiming be vietim of fraud.
The former secretary of state has made a major speech in New York, where she running for president.
The US economy grew at record low of 1 6 percent in 2014, and the unemployment rate has fallen by 0 percent.
The new rules put into the hands of member of the police, and the public is not aware of the situation.
The World Health Organization said number of people were killed in the attack, according to the Pentagon.
The study also found that women who are not particularly vulnerable women health problems are more likely to commit suicide.
Table 9: Samples from our MLE with temperature to match BLEU scores reported in Guo et al. 2017)
18


Reviews:
[{"review_id": "rkx9Nl1zcH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper concerns the limitation of the quality-only evaluation metric for text generation models. Instead, a desirable evaluation metric should not only measure the sample quality, but also the sample diversity, to prevent the mode collapse problem in gan-based models generation. The author presents an interesting, but not too surprising finding that, tuning the temperature beam search sampling consistently outperform all other GAN/RL-based training method for text generation models. The idea of sweeping temperature during beam search decoding is not new in the NLP community, which limits the novelty of this paper. What’s more, some parts of the experiment results is also somehow not new, in the sense that the SBLEU vs Negative BLEU tradeoff curve is also shown in [1,2,3,4].\n\n[1] Jointly measuring diversity and quality in text generation models, 2019\n[2] Training language gans from scratch, 2019\n[3] On accurate evaluation of gans for language generation, 2018\n[4] Towards Text Generation with Adversarially Learned Neural Outlines, 2018\n\nI would love to increase my score if the author could address the following comments:\n(1) Are the comparing methods, say MLE models and other GAN-based models, have the similar number of model parameters? It is not clear from the paper. Otherwise, one can use a 12/24 layer Transformer-XL to have dominative performance?\n(2) Since this is an empirical study paper. It would be great if this paper can also present more SoTA models trained by MLE such as Transformer-XL on more challenging datasets, such as Wikitext-2 or Wikitext-103. In this kind of large vocabulary datasets, I think the RL/GAN-based training methods would easily breakdown, and far worse than MLE-based training.\n(3) To make the empirical study more comprehensive, the author could perhaps evaluate with the n-gram and FED metric.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HyeAqdnCFr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Recently many language GAN papers have been published to overcome the so called exposure bias, and demonstrated improvements  in natural language generation in terms of sample quality, some works propose to assess the generation in terms of diversity, however, quality and diversity are two conflicting measures that are hard to meet. This paper is a groundbreaking work that proposes receiver operating curve or Pareto optimality for quality and diversity measures, and shows that simple temperature sweeping in MLE generates the best quality-diversity curves than all language GAN models through comprehensive experiments. It points out a good target that language GANs should aims at. \n\nFor the experiments on long-text generation using EMNLP news 2017, it is not clear how the data is partitioned as training data, validation data and test data to get the results in Figures 4(a), moreover for the results for LeakGAN, RankGAN SeqGAN, MaliGAN, it seems that they are copied from other papers, but again how the data set is partitioned is not clear  for example in SeqGAN's paper, and most likely, the data is partitioned in a different way, so the results are not comparable. The authors should run the code and get the results on it own.\n\nAn important RL-free language GAN paper is missing, \nZhongliang Li, Tian Xia, Xingyu Lou, Kaihe Xu, Shaojun Wang, Jing Xiao: Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators. AISTATS 2019: 3089-3098.\nThis paper directly and adversarially train a language model without MLE pre-training and obtains good results, it is better to compare the results.\n\nTypo: page 1, the second line from bottom, as a computationally, not an", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Main content:

Blind review #1 summarizes it well:

Recently many language GAN papers have been published to overcome the so called exposure bias, and demonstrated improvements  in natural language generation in terms of sample quality, some works propose to assess the generation in terms of diversity, however, quality and diversity are two conflicting measures that are hard to meet. This paper is a groundbreaking work that proposes receiver operating curve or Pareto optimality for quality and diversity measures, and shows that simple temperature sweeping in MLE generates the best quality-diversity curves than all language GAN models through comprehensive experiments. It points out a good target that language GANs should aims at. 

--

Discussion:

The main reservation was the originality of the idea of using temperature sweep in the softmax. However, it turns out this idea came from the authors in the first place, which they have not been able to state directly due to the anonymity requirement. Per the program chair's instruction to direct this to the area chair, I think this has been handled correctly.

--

Recommendation and justification:

This paper should be accepted. It provides readers with insight in that it illuminates a misconception of how important exposure bias has been assumed to be, and provides a less expensive MLE based way to train than GAN counterparts.
Decision: Accept (Poster)