Changing the Mind of Transformers for Topically-Controllable Language Generation (2021)

Abstract: Large Transformer-based language models can aid human authors by suggesting plausible continuations of text written so far. However, current interactive writing assistants do not allow authors to guide text generation in desired topical directions. To address this limitation, we design a framework that displays multiple candidate upcoming topics, of which a user can select a subset to guide the generation. Our framework consists of two components: (1) a method that produces a set of candidate topics by predicting the centers of word clusters in the possible continuations, and (2) a text generation model whose output adheres to the chosen topics. The training of both components is self-supervised, using only unlabeled text. Our experiments demonstrate that our topic options are better than those of standard clustering approaches, and our framework often generates fluent sentences related to the chosen topics, as judged by automated metrics and crowdsourced workers.

## introduction recently, transformer-based language models (lms) have achieved impressive performance in language generation tasks @xcite such as open-domain story generation @xcite . when writing with the lm, users often desire an intuitive and effective way to control what a lm is going to generate @xcite . to address this need, interactive writing assistants provide options to reveal possible developments of the story and generate continuations guided by the user-selected options. interactive writing assistants have wide applications in creative writing @xcite @xcite , education @xcite , and gaming @xcite . nevertheless, the existing systems' options usually do not provide fine-grained control and/or step 2: might say these topics step 1: let's see what language models would say step 3: please talk more about these topics 1 book books novels 2 essays perspectives perspective 3 university faculty undergraduate 4 reid sen. mcconnell 5 humanity life spirituality 6 2011 2010 2009 7 know sure want 8 insistence disdain dismissive 9 election elections democratic 10 u.s. states united input prompt: "barack obama writes a new book" output continuation: ": the future of a democratic election. the book tells the story of the 2008 election." transformer -based language models user step 4: let me try. what does this continuation sound? require substantial human labor. in some prior work @xcite , users choose among a static set of predefined attributes (e.g., sentiment) that only provide coarse-grained control. other work @xcite presents users with multiple generated continuations, which requires substantial reading effort and might not contain topics that users want to see. finally, options could be nodes in a plot graph that are handcrafted @xcite or derived from a collaboration between humans and machine @xcite , but such choices are usually limited due to the high cost of preparing the options. to address these limitations, we propose an interactive writing framework that provides a set of topics and guides the text generation by the userchosen topics. the topic options are generated dynamically based on the input prompt to pro- input prompt: barack obama writes a new book : the future of a democratic election. the book tells the story of the 2008 election. topic: election, elections, democratic topic: book, books, novels topic: humanity, life, spirituality on spirituality and the role of religion in society topic: god, christ, eternal , entitled my living with god , and writes that he will give the gift of grace . in it he describes why many americans believe in political parties. topic: understand, know, realize word: story word: zombie about the united states entitled i don't care...you bet i'm a zombie. topic: american, america, u.s. topic: political, ideology, politics . in the united states, many people know the story of the human race vide fine-grained control, and our models are selfsupervised without the need to define the attributes or collect annotations. as depicted in figure 1 , a user can peek at the most probable k topics (shown as bags of words) appearing after the input prompt and control the generation by choosing the topics. in figure 2 , we compare multiple generated sentences conditioned on different chosen topic(s) or specified word(s). for example, if the user chooses a topic about humanity, life, and spirituality, our system continues the input prompt "barack obama writes a new book" with "on spirituality and the roles of religion in society". then, we can use the generated text as the new input prompt and update the set of topics to include other more relevant topics such as god, christ, and eternal. the process can be repeated to create a plot tree. a user can also control the generation by specifying word(s) if the user wants to see the words that are not in the topic list or seeks a transition to a word that is not directly related to the input prompt. for example, a user can ask our system to generate a sentence about zombie. consequently, the continuation of "barack obama writes a new book" becomes "about the united states entitled i don't care...you bet i'm a zombie". the system is realized by two components: an option generator and a conditional text generator. given a prompt, the option generator suggests a set of k topics. after a user chooses a subset of the topics and specifies some words, the embedding of every word or topic will guide the conditional text generator to produce the continuation that is both consistent with the existing prompt and relevant to the chosen topics and words. both components are self-supervised and use pretrained gpt2 models @xcite to encode the input prompt. during training, the option generator predicts the cluster centers of future words, which are in the continuation of the prompt, based on the contextualized embeddings from gpt2. the conditional text generator finetunes gpt2 to predict the next words given the prompt and a few subsequent words. since both components' input and output only come from the prompt and its continuation, training the system only requires a raw corpus, word tokenizers, and a list of stop words. this makes the proposed method suitable for open-domain story generation and easily being fine-tuned for a specific domain. in experiments, we demonstrate that our system recommends high-quality topics and often generate sentences that follow the chosen topics. we compare our option generator with global topic models such as lda @xcite or local topic models such as clustering the words in the input prompt. the results show that the proposed method generates significantly more topics that are plausible and promote the narrative. moreover, we compare our conditional text generator with pplm (plug and play language models) @xcite and demonstrate that our generation is more fluent and relevant to the chosen topics. our code is available at https://github.com/iesl/interactive_lm . ## method the proposed framework consists of two components: option generator and conditional text generator. in figure 3 , we illustrate the two components and their interaction. first, given the prompt x 1 , ..., x i inputted by a user, the option generator at the bottom of the figure outputs k topics. after the user chooses two topics about book and election and specifies one extra word story, the topics gpt2 encoder book, books, novels t5 linear layer weighted average of glove t7 election, elections, democratic tw story sample based on probability glove gpt2 encoder transformer l1 l2 l3 l4 l5 l6 l7 l8 l9 l10 3. write t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 3. choose 2. show 1. write 4. show (b) option generator (a) conditional text generator user book : new barack … … book obama new barack writes a softmax p w 5 p w 6 … p w 1 p f 6 p f 6 p f 6 closest m words and word are passed to our text generator as the generation guidance. accordingly, the generator continues to write the next token y 1 . 1 in the following subsections, we introduce our model designs and the way to train each component. more implementation details are described in the appendix. ## experiments we evaluate two components separately, and both evaluations include automated metrics and human judgment. throughout the evaluation, the number of topics @xmath0. we find that fixing @xmath1. if the possible continuations cover more than 10 topics, our option generator tends to output the important topics. if they cover fewer topics, our option generator tends to output the related topics that are not explicitly mentioned in the prompt or the duplicated topics. more experiment setup details could be found in the appendix. ## related work different interactive writing assistants provide different forms of options to let users express their preferences. the options could be manually defined classes (e.g., sentiment) @xcite , semantic frames @xcite , or event structures such as (subject, verb, object, modifier) @xcite @xcite . the forms of options allow users to control the attributes of the generated text but require labels or classifiers that map the text to the attributes/options. the options could also be a single query word at the beginning @xcite , the article title @xcite , politeness @xcite or specificity @xcite of the text, or the length of the generated sentence @xcite . however, the options cannot provide fine-grained control on topical directions of the generated contents. a related research direction is the multi-stage story generation. to make a long story more coherent, recent work proposes to generate a skeleton and then generate the full text guided by the skeleton. the skeleton could be a sequence of srl frames @xcite , a sequence of event structure (subject, verb, object, preposition, modifier) @xcite , a story premise @xcite , or a story summary @xcite . users can revise the skeleton to control the generated text, but the approaches assume the existence of the skeleton extractor or labels in the training corpus. besides, the systems cannot suggest options given the partial text, which is one of the main focuses of our interactive writing assistant. the skeleton could also be multiple keyphrases. the keyphrases are extracted based on word frequency @xcite @xcite , an off-the-shelf keyword extraction method @xcite @xcite @xcite , a sentence compression dataset and reinforcement learning @xcite , or image caption datasets and conceptnet @xcite . most of the studies focus on modeling the longterm dependency among the keyphrases and/or forcing the generation to contain the keyphrases. instead, we focus on allowing users to determine the topical directions of the generation. compared with conditioning on keyphrases, our interactive writing assistant is especially helpful when users do not know the exact phrases they want to see or when the given keyphrase extractor does not detect the desired topics. ## conclusion we propose an interactive writing assistant that generates topic options given an input prompt and generates the continuation of the prompt given the topics chosen by a user. we decompose the framework into two components and propose a novel model for each component. the automated evaluation and human evaluation indicate that our system generates many topics that are related to but different from the prompt, and generates the sentences that are fluent and relevant to the chosen topics.