Substance over S tyle: D ocument- L evel T argeted C ontent T ransfer (2020)

Abstract: Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an existing document to fit a set of constraints. Although sentence-level rewriting has been fairly well-studied, little work has addressed the challenge of rewriting an entire document coherently. In this work, we introduce the task of document-level targeted content transfer and address it in the recipe domain, with a recipe as the document and a dietary restriction (such as vegan or dairy-free) as the targeted constraint. We propose a novel model for this task based on the generative pre-trained language model (GPT-2) and train on a large number of roughly-aligned recipe pairs. Both automatic and human evaluations show that our model out-performs existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. Finally, we analyze our model’s rewrites to assess progress toward the goal of making language generation more attuned to constraints that are substantive rather than stylistic.

## introduction we often think that writing starts from a blank page, but in practice, writing often involves adapting an existing document to fit a new context. this might involve rewriting documentation written for a mac so that it will apply to a pc, rewriting a lesson plan for a different grade level, or rewriting a product description to appeal to customers in multiple regions. automating such rewriting is valuable but challenging, since it requires learning to make coordinated changes spanning an entire document while adhering to constraints that apply not to the style but to the substance of the document. we introduce the novel task of document-level targeted content transfer, defined as rewriting a document to obey a user-provided constraint resulting in some systematic alteration of the document's content. success at this task involves both transfer and controlled generation at the document level. prior work on controlled generation guides the output of a model using attribute classifiers @xcite or control codes @xcite , but we find that these models do not perform well on our transfer task ( §4.1.2). in contrast, models built for the transfer task are generally trained at the sentence level @xcite @xcite . document-level transfer has typically found success by rewriting each sentence independently @xcite . however, many real-world rewriting scenarios require interdependent changes across multiple sentences. a clear example is cooking, where rewriting a hot cocoa recipe to make it vegan requires more than just substituting "coconut milk" for "milk" in a single step-it may also require changing the cooking times and techniques, adjusting ingredient amounts, or replacing other ingredients like toppings or spices (figure 1 ). such mix together cocoa, sugar, and salt in a small saucepan. <@xmath0 the recipe domain, constrained by dietary restrictions, is particularly well-suited to our task since recipes are commonly rewritten according to dietary constraints in real-world scenarios 2 , and this process often requires multiple related changes across the recipe. to construct our dataset, we use three steps: collect recipes spanning a range of dietary constraints ( § 2.1), tag recipes with dietary constraints using a rule-based method ( §2.2), and align recipes into pairs with similar content but opposite dietary tags ( §2.3). although our model relies on large amounts of parallel data, we obtain this parallel data automatically by running an unsupervised alignment algorithm @xcite on non-parallel data. large collections of non-parallel data are readily available on the web for many other domains, such as lesson plans for different grade levels or technical documentation for different operating systems. with the methods outlined in this section, non-parallel data can be aligned and transformed into a parallel dataset for transfer tasks in other domains. ## model description we propose two model variants for documentlevel targeted content transfer in the recipe domain. given a recipe and a dietary constraint, the goal is to rewrite the recipe one step at a time to fit the dietary constraint. ## experimental results we aim to answer the following research questions: ## analysis simple substitution is not adequate for the task of document-level targeted content transfer. in a recipe that contains a single violating ingredient "meat", the rule-based method makes the minimal edit of substituting "imitation meat", but ignores the other parts of the recipe that must change as a result. although on automatic metrics our model does only marginally better, qualitatively we found many cases where the rule-based method fails: it always suggests the same substitutions independent of the type of recipe leading to awkward food combinations, it misses a long tail of uncommon ingredients, and it does not make contextual changes to ingredient amounts, cooking times, or techniques. these flaws lead to the rule-based method performing worse than our model according to human judges ( the larger the number of invalid ingredients for a dietary constraint, the more difficult it was for our model to follow that constraint. vegan, the most restrictive constraint we studied, had the lowest dietary adherence accuracy across all models (93.6%). the alcohol-free constraint, which is dominated by one common ingredient (wine), had the highest accuracy (99.5%) despite the models seeing fewer training examples for that constraint. 21 the contextual rewriter + rule prompt falls short in its understanding of the physical entities involved in cooking. some of the steps it generates are not physically possible, such as "dip the cheese into the bread". the model can also suggest unrealistic or illogical cooking times (e.g. "bake for 10-10 minutes"), or change oven temperature mid-recipe. while these results are uncommon, they highlight that the model has not learned the physical rules governing the use of ingredients and cooking techniques. ## conclusion we introduce the novel task of document-level targeted content transfer and address it in the recipe domain, where our documents are recipes and our targeted constraints are dietary restrictions. we propose a novel model for rewriting a source recipe one step at time by making use of document-level context. further, we find that conditioning the model with step-level constraints allows the rewritten recipes to stay closer to the source recipe while successfully obeying the dietary restriction. we show that our proposed rewriter is able to outperform several existing techniques, as judged both by automatic metrics and human evaluators. although we focus on the recipe domain, our method naturally generalizes to other domains where procedural tasks can be substantively rewrit-ten. for example, one could rewrite technical documentation by constraining on the target operating system, rewrite lesson plans by constraining on the target grade level, or rewrite furniture assembly instructions by constraining on the tools used. more broadly, this approach makes it possible to customize existing content to better fit a user's physical reality, whether that entails accommodating their dietary needs, updating their schedule based on the weather forecast, or providing information on a dashboard based on what's in their field of view. as language generation becomes more grounded in signals outside of language, work in the area of substantive transfer becomes increasingly relevant.