Task: Summarization
Input (English): Despite considerable advances in neural language modeling, it remains an open question what the best decoding strategy is for text generation from a language model (e.g.  to generate a story).   The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, maximization-based decoding methods such as beam search lead to degeneration — output text that is bland, incoherent, or gets stuck in repetitive loops.

 To address this we propose Nucleus Sampling, a simple but effective method to draw considerably  higher  quality  text  out  of  neural  language  models. Our  approach avoids text degeneration by truncating the unreliable tail of the probability distribution, sampling from the dynamic  nucleus  of  tokens containing the vast majority of the probability mass.

 To properly examine current maximization-based and stochastic decoding methods, we compare generations from each of these methods to the distribution of human text along several axes such as likelihood, diversity, and repetition. Our results show that (1) maximization is an inappropriate decoding objective for open-ended text generation, (2) the probability distributions of the best current language models have an unreliable tail which needs to be truncated during generation and (3) Nucleus Sampling is the best decoding strategy for generating long-form text that is both high-quality — as measured by human evaluation — and as diverse as human-written text.
Output (English): Current language generation systems either aim for high likelihood and devolve into generic repetition or miscalibrate their stochasticity—we provide evidence of both and propose a solution: Nucleus Sampling.