text: 479,116,1060211883112841216,885528008,William Fedus,"With the careful investigative work of @masscaccia and @LucasPCaccia, we find that NLP GAN models still aren't improving over a simple maximum-likelihood baseline with reduced softmax temperature as assessed on (local/global) quality-diversity spectrum!  <LINK> This was a funny paper for me as a past author of an NLP GAN paper (MaskGAN: <LINK>). In MaskGAN, we demonstrated that mode-collapse and loss of diversity was occurring Sec 5.4, appendix C.4). However, the recent *global* diversity advances by the Zurich Brain Group, S. Semeniuta, A. Severyn, S. Gelly <LINK> helped us make this comparison more rigorous. @sylvain_gelly",https://arxiv.org/abs/1811.02549,"Generating high-quality text with sufficient diversity is essential for a wide range of Natural Language Generation (NLG) tasks. Maximum-Likelihood (MLE) models trained with teacher forcing have consistently been reported as weak baselines, where poor performance is attributed to exposure bias (Bengio et al., 2015; Ranzato et al., 2015); at inference time, the model is fed its own prediction instead of a ground-truth token, which can lead to accumulating errors and poor samples. This line of reasoning has led to an outbreak of adversarial based approaches for NLG, on the account that GANs do not suffer from exposure bias. In this work, we make several surprising observations which contradict common beliefs. First, we revisit the canonical evaluation framework for NLG, and point out fundamental flaws with quality-only evaluation: we show that one can outperform such metrics using a simple, well-known temperature parameter to artificially reduce the entropy of the model's conditional distributions. Second, we leverage the control over the quality / diversity trade-off given by this parameter to evaluate models over the whole quality-diversity spectrum and find MLE models constantly outperform the proposed GAN variants over the whole quality-diversity space. Our results have several implications: 1) The impact of exposure bias on sample quality is less severe than previously thought, 2) temperature tuning provides a better quality / diversity trade-off than adversarial training while being easier to train, easier to cross-validate, and less computationally expensive. Code to reproduce the experiments is available at github.com/pclucas14/GansFallingShort ",Language GANs Falling Short,3,"[""With the careful investigative work of @masscaccia and @LucasPCaccia, we find that NLP GAN models still aren't improving over a simple maximum-likelihood baseline with reduced softmax temperature as assessed on (local/global) quality-diversity spectrum! \n\n<LINK>"", 'This was a funny paper for me as a past author of an NLP GAN paper (MaskGAN:  https://t.co/YhuUvXui5i).  In MaskGAN, we demonstrated that mode-collapse and loss of diversity was occurring Sec 5.4, appendix C.4).', 'However, the recent *global* diversity advances by the Zurich Brain Group, S. Semeniuta, A. Severyn, S. Gelly https://t.co/8kTzSDTJ4E helped us make this comparison more rigorous.  @sylvain_gelly']",18,11,632