Leveraging Prefix Transfer for Multi-Intent Text Revision (2023)

Abstract: Text revision is a necessary process to improve text quality. During this process, writers constantly edit texts out of different edit intentions. Identifying edit intention for a raw text is always an ambiguous work, and most previous work on revision systems mainly focuses on editing texts according to one specific edit intention. In this work, we aim to build a multi-intent text revision system that could revise texts without explicit intent annotation. Our system is based on prefix-tuning, which first gets prefixes for every edit intent, and then trains a prefix transfer module, enabling the system to selectively leverage the knowledge from various prefixes according to the input text. We conduct experiments on the IteraTeR dataset, and the results show that our system outperforms baselines. The system can significantly improve the SARI score with more than 3% improvements, which thrives on the learned editing intention prefixes.

## introduction revision is an essential process to improve the text quality @xcite . during this process, writers perform various editing operations on the text with different editing intentions. as shown in figure 1 , the writer corrects misspelled words to improve text fluency, deletes redundant words to improve text clarity, adds connective words to improve text coherence, inserts adverbs to convey the writer's writing preferences (style) and modifies data to update text information (meaning-changed). lots of recent studies have focused on a text revision task corresponding to a specific edit intention, such as grammatical error correction @xcite she went to the markt the changes made the paper better than before. ## method the revision task can be defined as the following process: given a source sentence @xmath0, . . . , x m ] and an optional edit intent @xmath1, . . . , y n ], where e is the set of all edit intentions. note that e is optional because it can be inferred from the input x. our method is depicted in figure 2 . it includes two stages: the multi-prefix tuning stage and the prefix transfer stage. ## experimental setup we choose bart-large as the plm for our system and use adapter-transformers @xcite to implement prefix-tuning. more implementation details are in appendix a. ## conclusion in this paper, we introduce a new method for multiintent text revision. the system is based on prefixtuning, which first obtains a prefix for every edit intention and then learns to transfer the knowledge in prefixes for every input instance by training a prefix transfer module. this prefix transfer module is configured as two attention units that act respectively on the key states and the value states at each attention layer of the plm. in this way, our method can make full use of the knowledge of various edit intentions and does not need to anno-tate the intentions of the input. the experimental results show that our method significantly outperforms baselines, and both multi-prefix and prefix transfer settings could improve the performance.