A Plug-and-Play Method for Controlled Text Generation (2021)

Abstract: Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible directions. Current decoding methods with the goal of controlling generation, e.g., to ensure specific words are included, either require additional models or fine-tuning, or work poorly when the task at hand is semantically unconstrained, e.g., story generation. In this work, we present a plug-and-play decoding method for controlled language generation that is so simple and intuitive, it can be described in a single sentence: given a topic or keyword, we add a shift to the probability distribution over our vocabulary towards semantically similar words. We show how annealing this distribution can be used to impose hard constraints on language generation, something no other plug-and-play method is currently able to do with SOTA language generators. Despite the simplicity of this approach, we see it works incredibly well in practice: decoding from GPT-2 leads to diverse and fluent sentences while guaranteeing the appearance of given guide words. We perform two user studies, revealing that (1) our method outperforms competing methods in human evaluations; and (2) forcing the guide words to appear in the generated text has no impact on the fluency of the generated text.

## introduction having systems capable of automatically generating human-like text has been an objective pursued since the early days of artificial intelligence @xcite . the recent development of large pre-trained language models based on the transformer architecture has brought us closer to this goal. indeed, current state-ofthe-art models can produce impressively realistic text @xcite @xcite , facilitating their use across a wide range of language generation applications. yet, these models are probabilistic in nature, and in certain language generation tasks, i.e., those that place few semantic constraints on the output, it can be difficult to control the general theme or ensure the presence of specific words in generated text. we use story generation @xcite as a running example; even in the presence of a prompt, there are many continuations that may result in a good story. indeed, for this task, a generator that does not reflect this diversity is likely undesirable, since the number of stories it could produce would be limited. but what if you would like your story to fulfill certain criteria? under existing methods, one would have to train (or at least fine-tune) a new model for a limited set of use-cases or hope that the model places sufficient probability mass on text meeting this criteria such that it shows up during search foot_1 @xcite @xcite . in this work, we introduce a better strategy for controlled decoding-where generated text must contain certain words-for semantically unconstrained generation tasks. in short, we shift the output distribution of a language generation model towards the semantic space of a given guide word. while this proposal sounds abstract at first, its realization is simple and intuitive: at each generation step, we modify the log-probability of the words in our vocabulary according to their semantic similarity to our guide word, quantified as the cosine similarity between their vector representations, i.e., word embeddings. this method not only encourages the explicit appearance of a guide word, but also encourages the model to generate appropriate context for the guide word to appear, i.e., words in the same semantic neighborhood @xcite . by annealing the strength of the probability shift, we can guarantee that all guide words appear in our text. moreover, this method allows for guidance towards both ordered and unordered sets of words. our decoding strategy is plug-andplay, i.e., it can be combined with any autoregressive language model and decoding algorithm with no additional training, differentiating it from other self-claimed plug-and-play methods that both (1) require additional discriminators and (2) lack the ability to ensure the appearance of specific words @xcite . in particular, this means it is out-of-the-box compatible with large pre-trained transformer models @xcite . we run extensive experiments with gpt-2 as our language model and find that our method produces text containing all specified guide words without an impact on the fluency and diversity of generated text. that is, we can control generation without harming text quality. furthermore, we find that our strategy outperforms competing methods in human evaluations, and that generated text is close in fluency and overall quality to human text. ## keyword2text in this work, we propose keyword2text (k2t), a new and simple plug-and-play method for exerting hard control during text generation. by modifying the score function, we can incorporate a semantic shift at decoding time, without additional models or fine-tuning. this method is model agnostic-it works with any autoregressive language model, including pre-trained transformers, and can be combined with other decoding methods and objectives. further, we show that k2t (1) does not require a pre-defined ordering of constraints, and (2) can be used for guaranteeing hard constraints. ## experimental setup to evaluate our controlled decoding strategy, we run three sets of experiments: (1) analysis of hyperparameters; (2) comparison to competing methods; we use word stemming to check for the occurrence of the target words, i.e., if a generated word has the same stem as the keyword, it is counted as an occurrence; this way special cases such as plurals can be handled automatically, whilst keeping the text coherent by avoiding semantic redundancies, such as "protective protection". ## related work various other approaches exist for controlled language generation; here we review those that are not discussed in §2. hard control. hard control of autoregressive language models in unconstrained settings has remained elusive thus far. @xcite propose a framework to exert hard control over language generation which, nevertheless, requires training three large transformer models. pre-trained language model towards the target topic. similarly, @xcite use a contrastive strategy to soft-control language generation. yang and klein (2021) directly modify the output probabilities of a (pre-trained) language model using the output of a discriminator that determines whether future text will contain the desired attribute, e.g., formality. unfortunately, the use of external discriminators limits the applicability of these methods since they require training data for each of the target topics or attributes. our method elegantly dispenses with the need for a discriminator by using the geometric properties of semantic spaces. ## conclusion in this work, we present k2t, a simple and intuitive plug-and-play decoding method that can be used to impose controls on any autoregressive model for language generation, including large pre-trained transformers, like gpt-2. our method guarantees the appearance of guide words and requires neither re-training nor the use of external discriminators. our two user studies reveal that k2t is superior to competing hard control methods and that there is no statistical difference in perceived fluency, consistency and overall quality between news articles generated by our method and by gpt-2 without control. in future work, we plan to investigate in more detail the application of this decoding method to soft control tasks and to text detoxification.