DYPLOC : Dynamic Planning of Content Using Mixed Language Models for Text Generation (2021)

Abstract: We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times’ Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.

## introduction opinion articles serve as an important media to convey the authors' values, beliefs, and stances on important societal issues. automatically generating long-form opinion articles has the potential of facilitating various tasks, such as essay writing and speech drafting, and it is the focus of this work. though opinion generation has been investigated for constructing arguments @xcite , writing reviews @xcite , and producing emotional dialogue responses @xcite , those outputs are relatively short. while impressive progress in generation has been achieved by using large pre-trained transformers @xcite , directly adopting united_states, intelligence knowledge, attack america was never prepared and had a bad intelligence system. president_of_the_u.s., bill_clinton, 9/11_attacks make, happen, mistake, administration george_w._bush, 9/11_attacks, iraq existence 1) -> 2 ) -> 3) -> figure 1 : sample counter-argument on reddit change-myview. our generator considers an input containing (1) a title and (2) an unordered set of content items. each content item consists of elements of an entity set @xcite , a concept set @xcite , and an optional onesentence claim @xcite . each output token is generated by conditioning on all content items, and the best aligned ones (learned by our model) are highlighted in corresponding colors. we also underline words that reflect the input concepts and entities. first, large models still fall short of producing coherent text due to the lack of efficient content control and planning @xcite @xcite . a common solution is to use concatenated phrases or semantic representations to guide the generation process @xcite @xcite , where content planning, including both content selection and ordering, is expected to be learned by attention mechanisms. however, attentions have only achieved limited improvements. recent work also explores training a separate planning module to produce sorted content, which is then fed into a generator @xcite @xcite . nonetheless, this strategy results in a disconnection between planning and realization, and the output is not guaranteed to respect the planning results @xcite . the second challenge for opinion generation resides in the diversity of information that is needed to produce an output with consistent stances and supported by pertinent facts. though large models memorize significant amounts of knowledge, they cannot retrieve and operate with them precisely @xcite . due to the argumentative nature of opinion text, simply including knowledge bases @xcite is insufficient to uphold the desired quality, as it requires the combination of subjective claims and objective evidence as supports. to this end, we propose a novel generation framework, dyploc (dynamic planning of content), to conduct content selection and ordering as text is produced. 1 concretely, given a set of unordered content items, as displayed in figure 1 , we design mixed language models, with each implemented as a sequence-to-sequence model to encode one item and the input statement. at each decoding step, our system selects which items to reflect, and predicts a word based on probabilities marginalized over all language models. crucially, our end-to-end trained framework (1) enables the generator to access multiple content items at all times and select content based on what has been generated so far, (2) can be directly built on large pre-trained transformers, e.g., bart @xcite , with planning and generation modules jointly trained, and (3) outputs learned content selection scores to provide an interface for system decision interpretation. furthermore, to ensure that our framework can be applied to a broad range of generation tasks, we design content items to cover three critical elements: entities and concepts that are central to many generation applications, and claims that are building blocks for opinion text. we show an example for counter-argument generation in figure 1 . importantly, we employ bart to predict additional relevant concepts, derived from concept- net @xcite , and generate claims, as central propositions, to enrich the generated text with both objective and subjective content. for experiments, we collect two datasets: (1) posts from reddit changemyview for argument generation, and (2) articles from the new york times opinion section @xcite for opinion article writing. our proposed framework outperforms competitive comparisons, such as finetuning bart with the same content items, based on automatic metrics of bleu, rouge, and me-teor. human assessment further confirms that our system outputs have richer content and are more coherent in both tasks. our main contributions are summarized as below: ## related work neural generation with planning. text planning is seen as a crucial step to guide the generation of high-quality, well-organized natural language text @xcite . incorporating planning modules to neural text generator has attracted significant research interests @xcite @xcite , which proves to be especially beneficial for long-form output @xcite . more recently, large pre-trained transformers have established new state-of-the-arts for a wide range of text generation tasks @xcite @xcite . but it is non-trivial to integrate planning modules into them. existing approaches resort to decoupling planning and decoding stages @xcite , which inevitably increases system complexities and potentially introduces cascading errors. we take inspiration from the retrieval-augmented generation framework @xcite , which is designed to incorporate relevant documents for v content item encoding (in parallel) plan scoring mixed conditional language model content item-conditioned @xmath0. i believe 9/11 would not have happened if al gore were elected president." 0.80 x 0.10 x 0.05 x 0.05 x [left] for each input content item (a title t, an entity set e i , and a core concept set c i ), we first expand it with more relevant concepts, i.e., c + i . for sentences to be realized as claims, we employ a separate generator to produce one draft claim, m i . [right] the augmented content items, denoted as {x i }, are encoded in parallel. at each decoding step, a plan scoring network estimates a distribution d(x i |@xmath1. a word is predicted based on probabilities marginalized over all content item-conditioned language models, i.e., p(y t |@xmath2, x i ) for the i-th model. question answering. our adaptation uses a trainable plan scoring module to reflect content selection and ordering, which is more suitable for long text generation and offers better interpretability. concurrent work by @xcite presents a mixtureof-expert decoder to tackle knowledge-grounded generation. however, their score distribution for language models is fixed across all decoding steps, whereas ours is updated as generation progresses and can better reflect the dynamic nature of content planning. ## experiment setups we experiment with the tasks of argument generation and opinion article writing ( § 4.1). both tasks require generating multi-sentence output, and contain a substantial amount of opinions and factual content. we describe the construction of initial content items and the training data for generating expanded concepts and claims in § 4.2. we present models for comparison in § 4.3. finally, we provide implementation details in § 4.4. ## further discussions evaluation results on generation quality have shown the effectiveness of our mixed language models. in this section, we aim to further understand the behavior of the plan scoring network, d(x|@xmath3, such as how it affects the usage of content items for generation. specifically, we adopt the following procedure to construct alignment between each sentence in the output and content items: for each token y t , we establish a mapping y @xmath4, i.e., x @xmath5, and d(x i |@xmath6.5. if all tokens in an entire sentence are mapped to the same x i , we consider this sentence is aligned to that content item. based on this rule, we show sample output and corresponding alignments in how many content items are used by the output? human judges have rated our model output to contain more relevant information ( how are claim content items realized? claims are the central elements for opinion text construction. as mentioned in § 4.2, a subset of the content items are supplied with claim sentences. in order to examine whether they are realized as claim sentences in the outputs, we leverage the fine-tuned bert classifier ( § 4.2) to label all output sentences. 90.96% of the sentences that are aligned to a claim element in the input are also labeled as claim on cmv. the percentage is only 69.41% for nyt, though, likely because the nyt opinion articles still contain more objective information. furthermore, we conduct a human evaluation study to assess the semantic relations between claim input and its aligned generated sentence. we randomly sample 50 outputs from test sets, and ask four human judges to read each. for each sample, we highlight one output sentence that is aligned to a content item with claim element. the judges determine a three-way (entail, neutral, contra-dictory) entailment relation between the input claim (premise) and the output (hypothesis). results show that entail accounts for 49.06% of all instances, while only 3.77% are deemed contra-dictory. upon inspection, the contradictory pairs are usually disagreements with regard to implicit sentiments, e.g., "journalist is the most responsible for the problem" vs. "media coverage is a good thing.". this suggests that while our conditional language model achieves reasonable semantic control in most cases, it is still not guaranteed to capture more nuanced semantics encoded in opinions and arguments. future work includes designing representations that can better model stances in opinions as well as argumentative structures. ## conclusion we present a novel text generation framework that enables dynamic content planning based on mixed conditional language models. we further employ large models to augment system inputs with diverse content that covers both objective and subjective information. the experiments on two distinct opinion text generation tasks show that our proposed model compares favorably against strong comparisons based on fine-tuned bart models with the same input. human evaluation further confirms that our model generations have richer information and better content organization.