Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks (2022)

Abstract: Iterative text revision improves text quality by fixing grammatical errors, rephrasing for better readability or contextual appropriateness, or reorganizing sentence structures throughout a document.Most recent research has focused on understanding and classifying different types of edits in the iterative revision process from human-written text instead of building accurate and robust systems for iterative text revision.In this work, we aim to build an end-to-end text revision system that can iteratively generate helpful edits by explicitly detecting editable spans (where-to-edit) with their corresponding edit intents and then instructing a revision model to revise the detected edit spans.Leveraging datasets from other related text editing NLP tasks, combined with the specification of editable spans, leads our system to more accurately model the process of iterative text refinement, as evidenced by empirical results and human evaluations.Our system significantly outperforms previous baselines on our text revision tasks and other standard text revision tasks, including grammatical error correction, text simplification, sentence fusion, and style transfer.Through extensive qualitative and quantitative analysis, we make vital connections between edit intentions and writing quality, and better computational modeling of iterative text revisions.

## related work our work is most closely related to @xcite , who formally introduced the task of iterative text revision by releasing an annotated dataset of iteratively revised texts, and also used it to provide edit suggestions in a human-in-the-loop iterative editing setting @xcite . in both their works, they computationally model the iterative text revision process, leveraging edit intent information by simply appending it to the input text. however, we improve on their modeling formulation, as evidenced by our experimental results in a significant way: instead of simply appending edit intentions at the beginning of any sentence, we provide more fine-grained edit intention information to our text revision model by first detecting the exact spans which require an edit. moreover, by incorporating edit-intention-specific knowledge from external task-specific datasets, we are able to push the performance further. ## deliterater we follow the iterative text revision task as introduced by @xcite : given a source document d t-1 , at each revision depth t, a text revision system will apply a set of edits to get the revised document d t . the system will continue iterating revision until the revised document d t satisfies a set of predefined stopping criteria, such as reaching a predefined maximum revision depth t max , or making no edits between d t-1 and d t . in this section, we describe iterater+, the augmented version of the iterative text revision dataset @xcite and the system pipeline for deliterater. ## quantitative results while the recent previous studies on text revision include felix @xcite , lasertagger @xcite , seq2edits ( stahlberg and kumar, 2020), and iterater du et al. (2022b) , we mainly compare our system against the latest work, iterater, as it had shown consistent improvements over the other aforementioned systems in revision quality @xcite . we evaluate our system, deliterater, on the test splits of iterater+ dataset which consists of the original iterater set and newly augmented task-specific datasets from four different text editing nlp tasks: text simplification, sentence fusion & splitting, gec, and formality style transfer, as described in section 3.1.2. tim wanted to go to sarah's birthday @xmath0. @xmath1. ## qualitative results in this section, we analyze the behavior of our deliterater system in greater detail by looking at the development of edit intent trajectories as we run the system iteratively using its previous revision d t-1 to generate the revision at depth t (section 5.1). in section 5.2, we try to probe our text revision model by modifying the outputs of the span-based model to see if we can generate diverse revisions by placing (1) different intents on the same edit spans, and (2) same intent on different edit spans. ## human evaluation to better understand how our system affects the text quality and the iterative revision process, we conducted human evaluations to investigate how do text editing models affect document quality. we hired a group of proficient linguists to evaluate the quality of the documents being edited across multiple (up to 3) revisions, where each revision was annotated by 3 linguists. for each revision, we randomly shuffle the original and revised texts, and ask the evaluators to select which one was better in terms of fluency, coherence, readability, meaning preservation, and overall quality. they could choose one of the two texts, or neither. then, we calculated the score for the quality of the human revisions as follows: 1 means the revised text is worse compared to the original text; 2 means the revised text does not show a better quality than the original text, or there was no agreement among the 3 annotators; 3 means that the revised text was better than the original text. figure 4 shows the results of the human evaluation on the aforementioned criteria. we choose our best-performing model (deliterater-multi) trained on iterater+ using the delineate-edititerate approach to generate revisions by first identifying editable spans, and compare with human revisions and the text revision model from @xcite . we see that our system produces the best overall results, outperforming the human edits, as well as iterater system in overall quality. this is a major improvement relative to @xcite where model revisions were significantly underperformed by human edits in overall quality. ## conclusion and discussions we propose deliterater: an improved system for iterative text revision, using a delineate-edititerate framework, consisting of an intent span detection model, and a text revision generation model, based on the iterater framework of @xcite . the edit intent detection model is a tokenlevel edit-intention classification model which detects editable spans: spans of text that require an edit along with the type of edit needed. the text revision model is a generative model, which makes re-visions to the detected editable spans, conditioned on their corresponding edit intentions. we also create iterater+: an expanded version of @xcite iterater dataset by incorporating data from other text editing nlp tasks. leveraging this dataset and the delineate-edit-iterate framework, our system supervises the revision generation model to reflect both the location, and intentions behind the desired revision, leading to superior performance on the iterative text revision task, compared to other baselines and related works. experiments on the standard iterater test dataset, as well as standard nlp text editing datasets demonstrate the effectiveness of our framework for the task. moreover, human evaluations indicate that our system produces the best overall results, outperforming the human edits, as well as iterater system across all revision depths. additionally, we provide insights into our models by probing them, and into the progression of iterative text revision by analyzing the edit intent trajectories across both our test datasets as well as a dataset of english essays from the ets corpus, hinting the possibility of learning optimal revision paths possibly thorough reinforcement learning. in the future, we plan to investigate in this direction as well as improving the general robustness of the system by task-specific data augmentation with induced noise.