PAIR : Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation (2020)

Abstract: Pre-trained Transformers have enabled impressive breakthroughs in generating long and fluent text, yet their outputs are often “rambling” without coherently arranged content. In this work, we present a novel content-controlled text generation framework, PAIR, with planning and iterative refinement, which is built upon a large model, BART. We first adapt the BERT model to automatically construct the content plans, consisting of keyphrase assignments and their corresponding sentence-level positions. The BART model is employed for generation without modifying its structure. We then propose a refinement algorithm to gradually enhance the generation quality within the sequence-to-sequence framework. Evaluation with automatic metrics shows that adding planning consistently improves the generation quality on three distinct domains, with an average of 20 BLEU points and 12 METEOR points improvements. In addition, human judges rate our system outputs to be more relevant and coherent than comparisons without planning.

## introduction large pre-trained language models are the cornerstone of many state-of-the-art models in various natural language understanding and generation tasks @xcite @xcite ), yet they are far from perfect. in generation tasks, although models like gpt-2 @xcite are able to produce plausible text, their spontaneous nature limits their utility in actual applications, e.g., users cannot specify what contents to include, and in what order. to make large models more useful in practice, and to improve their generation quality, we believe it is critical to inform them of when to say what, which is addressed as content planning in traditional generation systems @xcite mckeown, content plan (output by planning model): (1) a communist 3 ▷ begin with 8 ▷ coherent ideology 15 ▷ [sen] 21 (2) [sen] 4 (3) no evidence 2 ▷ any coherent 8 ▷ held beliefs 12 ▷ any topic 15 ▷ [sen] 18 prompt: cmv. donald trump is a communist. template: (1) __ 0 __ 1 __ 2 a communist __ 5 __ 6 __ 7 begin with __ 10 __ 11 __ 12 __ 13 __ 14 coherent ideology__ 17 __ 18 __ 19 __ 20 (2) __ 0 __ 1 __ 2 __ 3 (3) __ 0 __ 1 no evidence __ 4 __ 5 __ 6 __ 7 any coherent __ 10 __ 11 held beliefs __ 14 any topic __ 17 draft (initial generation): (1) well call him a communist, you must begin with that donald trump has some kind of coherent ideology to begin with. (2) which is unlikely. (3) there is no evidence to suggest donald trump has any coherent or commonly held beliefs on any topic. refined (final generation): (1) to call him a communist, you must begin with that he has some kind of coherent ideology in the first place. (2) he does not. (3) there is no evidence whatsoever that trump has any coherent, commonly held beliefs on any topic. i: template construction iii: refinement ii: generation with content plan 2001; @xcite . specially designed control codes and auxiliary planning modules have been integrated into neural models @xcite @xcite , yet those solutions require model architecture modification or retraining, making text generation with large models a very costly endeavor. to this end, this work aims to bring new insights into how to effectively incorporate content plans into large models to generate more rele-vant and coherent text. we first study a planning model trained from bert @xcite to produce the initial content plan, which assigns keyphrases to different sentences and predicts their positions. next, we propose a contentcontrolled text generation framework, built upon the pre-trained sequence-to-sequence (seq2seq) transformer model bart @xcite . as shown in figure 1 , our generation model takes in a content plan consisting of keyphrase assignments and their corresponding positions for each sentence. the plan is encoded as a template, with [mask] tokens added at positions where no content is specified. our model then outputs a fluent and coherent multi-sentence text (draft) to reflect the plan. this is done by fine-tuning bart without modifying its architecture. furthermore, we present an iterative refinement algorithm to improve the generation in multiple passes, within the seq2seq framework. at each iteration, tokens with low generation confidence are replaced with @xcite to compose a new template, from which a new output is produced. unlike prior refinement algorithms that only permit editing in place, our solution offers more flexibility. figure 1 exemplifies the refinement outcome. we call our system pair (planning and iterative refinement). 1 it is experimented on three distinct domains: counter-argument generation with reddit changemyview data, opinion article writing with the new york times (nyt) corpus 2 @xcite , and news report production on nyt. automatic evaluation with bleu, rouge, and meteor shows that, by informing the generation model with sentence-level content plans, our model significantly outperforms a bart model fine-tuned with the same set of keyphrases as input ( § 5.1). human judges also rate our system outputs as more relevant and coherent ( § 5.2). additionally, our iterative refinement strategy consistently improves the generation quality according to both automatic scores and human evaluation. finally, our model achieves better content control by reflecting the specified keyphrases in the content plan, whose outputs are preferred by human to another version with weaker control. to summarize, our major contributions include: bert to facilitate long-form text generation. ## further discussions on discourse prior work's evaluation mainly focuses on fluency and content relevance, and largely ignores the discourse structure exposed by the generated text. however, unnatural discourse and lack of focus are indeed perceived as major problems of longform neural generations, as identified by human experts. 4 here, we aim to investigate whether contentcontrolled generation with ground-truth content plans resembles human-written text by studying discourse phenomena. are pair generations similar to humanwritten text in discourse structure? we utilize dplp @xcite , an off-theshelf rhetorical structure theory (rst) discourse parser. dplp converts a given text into a binary tree, with elementary discourse units (edus, usually clauses) as nucleus and satellite nodes. for instance, a relation ns-elaboration indicates the second node as a satellite (s) elaborating on the first nucleus (n) node. dplp achieves f1 scores of 81.6 for edu detection and 71.0 for relation prediction on news articles from the annotated rst discourse treebank @xcite . we run this trained model on our data for both human references and model generations. first, we analyze the depth of rst parse trees, which exhibits whether the text is more locally or globally connected. for all trees, we truncate at a maximum number of edus based on the 90 percentile of edu count for human references. distributions of tree depth are displayed in figure 6 . as can be seen, generations by pair full show similar patterns to human-written arguments and articles. we also find that trees by pair light tend to have a more "linear" structure, highlighting the dominance of local relations between adjacent edus, compared with pair full which uses knowledge of keyphrases positions. this implies that content positioning helps with structure at a more global level. we further look into the ratios of ns, nn, sn relations, and observe that most model outputs have similar trends as human-written texts, except for kpseq2seq which has more sn relations, e.g., it produces twice as many sns than others on arguments. can pair correctly generate discourse markers? since discourse markers are crucial for coherence @xcite and have received dedicated research efforts in rulebased systems @xcite , we examine if pair full can properly generate them. for each sample, we construct sentence pairs based on content word overlaps between system generation and human reference. we manually select a set of unambiguous discourse markers from appendix a of the penn discourse treebank manual @xcite . when a marker is present in the first three words in a reference sentence, we check if the corresponding system output does the same. figure 7 displays the numbers of generated sentences with markers produced as the same in human references (correct) or not (wrong). the markers are grouped into three senses: contingency, comparison, and expansion. the charts indicates that pair full does better at reproducing markers for contingency, followed by comparison and expansion. manual inspections show that certain missed cases are in fact plausible replacements, such as using at the same time for in addition, or also for further, while in other cases the markers tend to be omitted. overall, we believe that content control alone is still insufficient to capture discourse relations, motivating future work on discourse planning. ## ethics statement we recognize that the proposed system can generate fabricated and inaccurate information due to the systematic biases introduced during model pretraining based on web corpora. we urge the users to cautiously examine the ethical implications of the generated output in real world applications. ## conclusion we present a novel content-controlled generation framework that adds content planning to large pretrained transformers without modifying model architecture. a bert-based planning model is first designed to assign and position keyphrases into different sentences. we then investigate an iterative refinement algorithm that works with the sequenceto-sequence models to improve generation quality with flexible editing. both automatic evaluation and human judgments show that our model with planning and refinement enhances the relevance and coherence of the generated content.