Generating Text from Language Models (2023)

Abstract: An increasingly large percentage of natural language processing (NLP) tasks center around the generation of text from probabilistic language models. Despite this trend, techniques for improving or specifying preferences in these generated texts rely mostly on intuition-based heuristics. Further, there lacks a unified presentation of their motivations, practical implementation, successes and pitfalls. Practitioners must, therefore, choose somewhat blindly between generation algorithms—like top-p sampling or beam search—which can lead to wildly different results. At the same time, language generation research continues to criticize and improve the standard toolboxes, further adding entropy to the state of the field. In this tutorial, we will provide a centralized and cohesive discussion of critical considerations when choosing how to generate from a language model. We will cover a wide range of empirically-observed problems (like degradation, hallucination, repetition) and their corresponding proposed algorithmic solutions from recent research (like top-p sampling and its successors). We will then discuss a subset of these algorithms under a unified light; most stochastic generation strategies can be framed as locally adapting the probabilities of a model to avoid failure cases. Finally, we will then cover methods in controlled generation, that go beyond just ensuring coherence to ensure text exhibits specific desired properties. We aim for NLP practitioners and researchers to leave our tutorial with a unified framework which they can use to evaluate and contribute to the latest research in language generation.

## introduction and motivation with their widespread public availability, large pretrained language models have become a core part of many natural language processing (nlp) pipelines. this trend is particularly evident in language generation tasks, where prompt engineering and controlled generation techniques have shown that these models can essentially be used "out-of-the-box" for various language generation needs. yet, as has been observed repeatedly, how one chooses to generate text from these models can lead to vastly different results; make the wrong choice and a language model can fall into repetitive loops @xcite , generate gibberish @xcite , or spew out random (and possibly falsifiable) declarations @xcite . in the effort to circumnavigate these issues, one can make use of a variety of relatively straightforward methods: (i) sampling adapters, simple modifications to token-level distributions that help prevent the generation of incoherent text; (ii) controlled generation methods, techniques that guide these models to output strings with a set of desired attributes. while employing these methods often does not require domain expertise, many people do not have proper knowledge of the tools available-and much less how and when to apply them. hence, without years of experience in this subfield, both nlp researchers and practitioners may have difficulty using pretrained language models for text generation, as they will likely encounter the problematic behaviors mentioned above. in this cutting-edge tutorial, we aim to offer a comprehensive introduction to techniques for generating strings from language models, discussing both how to sample adeptly from and explicitly control them. this tutorial will be divided in four parts. first, we will present background knowledge on language modeling, discussing both its mathematical formulation, the empirically-observed successes and shortcomings of modern models when used to generate language, and the difficulty in evaluating these successes and failures. second, we will give a brief overview of the basics of language generation, framing generation as the combination of a choice of a decoding algorithm and objective. the final two parts of this tutorial, which focus on alleviating the previously discussed issues with using language models out-of-the-box for generation, will be discussed within this framing: we present heuristic modifications to the objective that have empirically-proven themselves effective at improving generation quality as well as new decoding algorithms that-when combined with the right objective-can be used to enforce constraints on the text output by models. we believe this will equip the nlp community with the knowledge of how to better employ these models for their downstream use-cases, thus making them more broadly accessible. ## presenters his research tackles basic problems in learning models from broad distributions over language, characterizing and understanding those models, and building smaller, simpler models.