Sequentially Controlled Text Generation (2022)

Abstract: While GPT-2 generates sentences that are remarkably human-like, longer documents can ramble and do not follow human-like writing structure. We study the problem of imposing structure on long-range text. We propose a novel controlled text generation task, sequentially controlled text generation, and identify a dataset, NewsDiscourse as a starting point for this task. We develop a sequential controlled text generation pipeline with generation and editing. We test different degrees of structural awareness and show that, in general, more structural awareness results in higher control- accuracy, grammaticality, coherency and topicality, approaching human-level writing performance.

## introduction imagine that you are tasked with: write a "related works" section. would it help to know the past structure of the article (e.g. it is coming after the "discussion" section)? how about the full structure of the article (e.g. after the "introduction" but before the "problem statement")? the macro-structure of text (i.e. its discourse structure @xcite ) impacts both human and machine comprehension @xcite @xcite . although naive language models have made impressive advancements and generate fluent text @xcite @xcite , the text is structurally dissimilar to human-written text (figure 2 , section 7). even the well-known ovid's unicorn generation, which seems like a natural news article, exhibits unnatural structure (see appendix f). on the other hand, although numerous works have focused on content planning using keywords @xcite , plot-design @xcite and entity tracking @xcite , macrostructural control has been relatively understudied. figure 1 : we study the task of sequentially-controlled generation: generating documents exhibiting structure given by a sequence of local control codes. shown is a news article with it's van dijk structure @xcite and headline. our models take as input the headline and discourse tags and generate a sequence of sentences. we explore the degree of structural awareness (local, pastaware or full-sequence) for controlling each sentence in the document, with the goal of generating the most structurally faithful, coherent and topical text. so, in this work, we study (1) how to impose macro-structural control on narrative text generation and (2) how much structural awareness during generation contributes to well-structured and fluent text. we propose a novel task, sequentially controlled text generation. in this task, the user provides a prompt as well as a sequence of local control codes, each of which guides the generation of a single sentence. (in our experiments, we use headlines as prompts and van dijk (2013) discourse tags as control codes (figure 1 ).) we develop methods to address this task, expanding prior work focused on single control code generation @xcite @xcite . as in prior work, the controlled generation problem is decomposed into a discriminator and a generator. however, in this work, the discriminator learns to incorporate an entire sequence of control codes. we hypothesize that information about structural intention can positively impact generative output (intuition for this is given in the hypothetical at beginning of this introduction). we show that our methods improve structural cohesion and certain aspects of coherence over naive gpt-2 output. next, we hypothesize that more structural awareness improves generation. again, we refer to the introduction hypothetical: humans craft text according to how it fits into a document's full structure @xcite , so a generative model should similarly benefit from having such information. we test this hypothesis by varying the discriminator's conditional independence assumptions. we experiment with three different degrees of control: local-only (where the discriminator is only aware of the current sentences' control code), past-aware (where the discriminator is aware of the current sentences' control code and all previous control codes), and full-sequence (where the discriminator is aware of the entire document's sequence of control codes). we show that more structural awareness, especially of past structure, helps generate the highest-quality text. finally, we show how to re-introduce a degree of local control by combining structurally-aware generation methods with a local sentence-level editing technique. in summary, our novel contributions are: we see this work opening the door to a variety of follow-on directions: giving users control over the macro-structure of their generated output can allow users to quickly prototype different structures for their work. it can allow them to work in tandem with a generative algorithm to infill missing structural components in a piece of writing foot_0 . it might even allow them produce different versions of the same story for readers with different reading preferences. finally, we also see macrostructural control providing a natural complement to, and being used in tandem with, other forms of controlled generation, like fact-aware generation @xcite or creative generation @xcite @xcite to yield more engaging and useful generative content. ## problem statement we assume, as input, a headline sentence, x 0 , and a sequence of control codes ⃗ @xmath0, ..., c s of length s (i.e., one for each sentence we wish to generate in the document. adjacent codes can be of the same type.) we wish to produce, as output, a document x of length s as a sequence of sentences @xmath1, ..., x s , each composed of a sequence of words we define the sequentially controlled text generation objective as: we use bayes rule to factorize t 1 into: three approximations for t 3 are: in the local-only model, we assume each control code c k is conditionally independent of other control codes given x i . thus, our generator model t 1 is made aware only of local structure: the control code c k pertaining to the current sentence, k. because of this conditional independence assumption, local-only control is similar to prior work that used only single-control codes, where the goal was to generate a single sentence p(x|c) = n @xmath2. however, we show that we can remove these independence assumptions and study more complicated structural control which, as we show later, produces more coherent output. ## methodology as described in section 2, we can efficiently do generation by combining a naively-trained language model with a discriminator. hence, the discriminator is the main architectural component that allows us to incorporate inter-dependencies between control code sequences. we start by describing how our discriminator models different degrees of structural awareness (equations 3, 4 and 5) in section 3.1. we design a generation pipeline to balance structural and local awareness. the flow we use to accomplish this is depicted in figure 3 . the first step is generation. here, we sample each word, x i using techniques described in section 3.2 which allow us to leverage our discriminator to impose structural control. when we have completed a sentence, we move to editing. here, we edit the sentence to further impose local control on each sentence, updating x to optimize a variation of equation 1: p(x i |x -i , c k ), discussed in section 3.3. ## datasets and schema the form of sequential control we study is discourse: i.e. the functional role sentences play in a document's larger argumentative purpose. we use a news discourse schema proposed by @xcite . choubey et al. (2020) apply this schema and annotate a dataset, newsdiscourse, consisting of 802 articles from 3 outlets 10 , tagged on the sentence level. their schema consists of 9 classes: { main event, consequence, current context, pre-7 words that are not proper nouns, named entities (except the date class) or adjectives, as we find these categories are more likely to be topic words spuriously correlated with control-codes. e.g. label: background. text: the @xmath3. ## implementation details we fine-tune a gpt2-base model on a large news corpus with a max word-piece @xmath4. we use this to generate naive ptlm languagemodeling as well as sentence-embeddings in our discrimination model. further implementation details are discussed in appendix a. we discuss the discriminator results here briefly. as shown in figure 4 , the primary head, p, has a micro f1-score of .65, which approaches stateof-the-art on this dataset 14 . however, performance degrades rapidly for heads farther from p. for more results on discriminator performance, including experimental variations, see appendix a.1. ## experiments we sample 10 documents from the test set of our discourse dataset (@xmath5. the input to our models is a headline (as a prompt) and the full sequence of gold-truth discourse labels of that document. baselines we compare our experimental pipelines (section 3) with the following baselines: for (2), we directly train a class-conditional language model to generate text by including labels in the prompt, as in @xcite . local-only prompting is achieved by only including the local control code (and prior generated sentences) in the prompt, and updating the prompt to generate a new sentence. for past-aware prompting, we include all control codes prior to our current sentence in the prompt, and update on every new sentence. finally, for full-sequence prompting, we including the full sequence of control codes in the prompt. (see appendix c for more details and examples of prompt design.) for each of these baselines, we test with and without editing (with the human-written text being edited by our algorithm in human and with the generated text in all other trials being edited). evaluation for all pipelines, we select the best hyperparameter configurations based on perplexity and model-assigned class likelihood. then, we manually annotate each generated document for 4 metrics: accuracy (0-1) 15 grammar (1-5) 16 , logical flow (1-5) 17 and topicality (1-5) 18 . we recruit two expert annotators with journalism experience to perform annotations blindly without awareness to which generation pipeline was used, and find moderate agreement κ ∈ [.36, .55] across all categories. for more details, see appendix g. we record model-dependent and non-model automatic metrics used by @xcite , described further in appendix b. ## results best overall trial we show automatic and human metrics for the subset of pipelines with topperforming hyperparameters in we observe that dpc with past-aware control and editing has the highest class-label accuracy, nearly approaching the human trials. the top performing pipelines for grammar and topicality are dpc with full-sequence control and without editing. gpt-2 performed best only for logical flow, which was surprising but could perhaps be because the unconstrained nature of gpt-2's generation allowed it to hallucinate a flow that seemed consistent even if it was poorly structured. ## discussion we set out to answer two questions in this research: insight #1: some structural information improves all metrics of quality. our structural exploration suggests that, for the best-performing pipelines, past structural information (along with editing) boosts class accuracy the most, but knowledge of the full-sequence does not. in the analogy given in the introduction, this equates to: to write a "related works" section, it helps to know that it comes after the "introduction" vs. the "discussion", but not information of what sections come after. this is perhaps because enough signal is already given by the past sequence and the full sequence just adds more noise. however, fullsequence information does yield the best grammar and topicality. this might indicate a regularizing role played by the full-sequence. in general, we suspect that past-aware modeling and editing both push the model more towards the class label at the expense of topicality, flow and grammar, while full-sequence does the opposite. in practice, some combination of these pipeline components might be desired. insight #2: weak discriminators can still impose accurate control. at .61 macro f1, our discriminator is a relatively weak classifier. previous work in classifier-based controlled text generation used large training datasets and classifiers that routinely scored above .8 f1 @xcite . the weakness of our discriminator is one reason why hsc may have performed poorly. however, in other trials we see strong accuracy. thus, even with a weak classifier, we can control generation. this might be because even a weak discriminator can still give relative differ-ences between generation that does or does match the control code. insight #3: evaluating text candidates using multiple model's perplexity might result in better selections. just as surprisingly, editing also has an overall average positive effect on generation accuracy and generation quality (figure 6 ). we had hypothesized that, because the editor makes locally-aware infilling decisions, it would improve class-accuracy but hurt other metrics of document quality, like topicality and flow. indeed, for the topperforming trials, like dpc and human, editing only improves class accuracy. however, grammar and flow improves in other trials. this could be because, as mentioned in section 3.3, we selected candidates based on how well they make sense in the document. this also suggests that using multiple ptlms to select for better quality combines different virtues of each model. ## conclusion we have formalized a novel direction in controlled text generation: sequentially controlled text generation. we extended different techniques in controlled text generation to fit this direction, and have shown how a news discourse dataset can be used to produce news articles exhibiting human-like structure. we have explored what degrees of structural awareness yield the most human-like output: more structural control yields higher-quality output. and, we shown how to combine structural control with local editing. we have probed different parts of our pipeline to show the effects of each part.