Leveraging Prefix Transfer for Multi-Intent Text Revision
Year: 2023

Abstract:
Text revision is a necessary process to improve text quality. During this process, writers constantly edit texts out of different edit intentions. Identifying edit intention for a raw text is always an ambiguous work, and most previous work on revision systems mainly focuses on editing texts according to one specific edit intention. In this work, we aim to build a multiintent text revision system that could revise texts without explicit intent annotation. Our system is based on prefix-tuning, which first gets prefixes for every edit intent, and then trains a prefix transfer module, enabling the system to selectively leverage the knowledge from various prefixes according to the input text. We conduct experiments on the ITERATER dataset, and the results show that our system outperforms baselines. The system can significantly improve the SARI score with more than 3% improvements, which thrives on the learned editing intention prefixes.

heading: 1 Introduction
text: Revision is an essential process to improve the text quality (Vaughan and McDonald, 1986). During this process, writers perform various editing operations on the text with different editing intentions. As shown in Figure 1, the writer corrects misspelled words to improve text fluency, deletes redundant words to improve text clarity, adds connective words to improve text coherence, inserts adverbs to convey the writer’s writing preferences (style) and modifies data to update text information (meaning-changed).
Lots of recent studies have focused on a text revision task corresponding to a specific edit intention, such as grammatical error correction (Omelianchuk
*Equal contribution †Corresponding author: Liner Yang
et al., 2020; Kaneko et al., 2020; Liu et al., 2021; Yang et al., 2022), text simplification (Dong et al., 2019; Jiang et al., 2020; Omelianchuk et al., 2021; Martin et al., 2022), and text style transfer (Malmi et al., 2020; Reid and Zhong, 2021). The work divides text revision into several independent problems. While some methods with strong universality can be applied to multiple tasks (Malmi et al., 2019; Stahlberg and Kumar, 2020; Mallinson et al., 2020), they train different models on various data sets. Real-world scenarios require addressing multiple types of editing errors at the same time, such as grammatical errors, spelling errors, etc. But these methods failed to integrate knowledge from these tasks into a unified model.
To solve the problem, Du et al. (2022) attempted to train one model using data with multiple editing intentions and leveraged edit intent information by simply appending it to the input. However, when adding a new intent, the entire model must be re-
1219
trained. A more lightweight and scalable approach to multi-intent text revision is still required.
Li and Liang (2021) proposed a new kind of prompt tuning method to quickly adapt a pretrained model to new tasks, which is called prefixtuning. Prompt tuning can help the pre-trained language model to locate the task learned in pretraining and enable the related knowledge to model text revision with different edit intentions (Reynolds and McDonell, 2021). This method enables a model to handle multiple edit intentions in a lightweight and scalable way.
In this paper, we present our method: a prefixtuning-based model which adapts to text revision with multiple edit intentions. This method involves a two-step training process. In the first step, we initialize a pre-trained language model (PLM) and train multiple prefixes on it. Each edit intention corresponds to a prefix. In the second step, a prefix transfer module is trained at each attention layer of the PLM. The prefix transfer module is configured as two attention units that act respectively on this layer’s key states and value states. It enables our model to learn a tailored prefix for the given input with the help of prefix embeddings from the predefined tasks.
We conduct experiments on ITERATER (Du et al., 2022), an iterative text revision dataset. It mainly contains parallel sentences with five edit intentions: fluency, coherence, clarity, style, and meaning-changed. The results show that our approach performs better than the fully fine-tuned BART (Lewis et al., 2020) and PEGASUS (Zhang et al., 2020) baselines reported in Du et al. (2022) with fewer training parameters.

heading: 2 Related Work

heading: 2.1 Iterative Text Revision
text: For the first time, Du et al. (2022) systematically studied the iterative revision phenomenon in human writing. They presented the ITERATER, an annotated dataset across multiple domains of formally human-written text, which includes Wikipedia, ArXiv, and Wikinews. And they trained several types of text revision models using ITERATER. Dwivedi-Yu et al. (2022) presented EDITEVAL, an instruction-based benchmark, to evaluate the editing capabilities of models and they also included the test set of ITERATER in it. Based on Du et al. (2022), our work further explores the method of text revision.

heading: 2.2 Transfer Learning of Prompt Tuning
text: Transfer learning is a common and powerful technique in NLP (Raffel et al., 2020). Some recent studies have tried to improve prompt tuning performance by leveraging the knowledge of multiple related or unrelated tasks. Asai et al. (2022) used an attention module to make use of the knowledge in exiting soft prompts (Lester et al., 2021) while learning a new task. Chen et al. (2022) improved the few-shot text summarization by multi-task pretraining and prefix-tuning. Specifically, they pretrained a summarization model on a set of popular summarization datasets and then conducted prefixtuning for it on an unseen summarization task. Different from their modeling of a new task through existing tasks, our work aims to achieve the mutual utilization of knowledge between different edit intents in text revision.

heading: 3 Method
text: The revision task can be defined as the following process: given a source sentence x = [x1, . . . ,xm] and an optional edit intent e ∈ E to generate a revised sentence y = [y1, . . . ,yn], where E is the set of all edit intentions. Note that e is optional because it can be inferred from the input x.
Our method is depicted in Figure 2. It includes two stages: the multi-prefix tuning stage and the prefix transfer stage.

heading: 3.1 Multi-Prefix Tuning Stage
text: The prefix is a set of parameters on every attention layer of PLM. For an edit intention e, at each attention layer, the prefix can be described as Pe = {PKe , P Ve }, where PKe and P Ve are parameters added before the key states and value states in this attention layer. After adding these parameters, the calculation of the attention head in this layer becomes:
H = Attention(Q, [PKe ;K], [P V e ;V ]) (1)
where H is the output vector sequence; Q,K, V are query states, key states, and value states, respectively; Attention means scaled dot-product attention. Only PKe and P V e are updated during the training process. Note that we ignore the layer number information because the operation for each layer is the same.
As shown in the left part of Figure 2, for every edit intention e, we train a prefix Pe accordingly.
In this way, the model could revise an intentionannotated text by activating the corresponding prefix at inference.

heading: 3.2 Prefix Transfer Stage
text: Identifying edit intention is always an ambiguous work. At the prefix transfer stage, we aim to build a new prefix for an unannotated input instance by transferring existing prefixes. The new prefix Pnew is instance-specific.
The prefix transfer stage is described in the right part of Figure 2. At each layer, we rearrange the prefixes {Pe | e ∈ E} obtained in the last stage as PK = {PKe | e ∈ E} and P V = {P Ve | e ∈ E} according to whether they are configured before the key states or before the value states. Then a pair of attention units GK and GV are trained for PK and P V .
Take GK as an example. It calculates the similarity between the key states K and every PKe in PK to get attention scores.
The similarity can’t be calculated directly, because K and PKe have different lengths. So we perform the max-pool operation for length dimension on K and PKe . After that, we obtain K̂ ∈ Rd and P̂Ke ∈ Rd, where d is the dimension of the hidden states in the PLM.
To get attention scores, we train a fully connected layer to extract features from K̂:
H = NonLinear(W⊤(K̂)) (2)
where W ∈ Rd×d is a transfer matrix updated during training. Following Asai et al. (2022), we use SiLU (Elfwing et al., 2018) for the non-linear layer and add a Layer Norm (Ba et al., 2016) layer:
Hnorm = LayerNorm(H) (3)
Then, we calculate the attention scores for intent e as follows:
ae = exp(P̂Ke ·Hnorm)/T∑ i∈E exp(P̂ K i ·Hnorm)/T
(4)
where T is the softmax temperature (Radford et al., 2021) which could avoid making the attention unit over-confident.
Finally we use them to build PKnew as follows:
PKnew = ∑
e∈E aeP
K e (5)
In the same way, we get P Vnew by GV . Using the new prefix Pnew = [PKnew, P V new], our system could revise the unannotated input instance with the knowledge from existing prefixes.

heading: 4 Experimental Setup
text: We choose BART-large as the PLM for our system and use adapter-transformers (Pfeiffer et al., 2020) to implement prefix-tuning. More implementation details are in Appendix A.

heading: 4.1 Datasets
text: We conduct our experiments on the iterative text revision dataset: ITERATER (Du et al., 2022). We remove the Other class of the data as it essentially contains a variety of unrecognized edit intentions and accounts for a small proportion (1.44%). The entire dataset consists of two parts: ITERATERHUMAN and ITERATER-FULL. The former is a smaller dataset with manual annotation of edit intentions, while the latter is a large dataset annotated by a classification model trained on ITERATERHUMAN. We train our model on both of them. Following Du et al. (2022), we report the results on the test set of ITERATER-HUMAN in Section 5,
which is completely a human-created dataset and is reliable for evaluation. We show more details of the datasets in Appendix B.

heading: 4.2 Evaluation Metrics
text: Following previous work, we report three metrics: SARI (Xu et al., 2016), Rouge-L (Lin, 2004), and BLEU (Papineni et al., 2002). Among them, SARI is considered an important metric in situations where input text and output text have a large overlap in words. It also indicates the positive impact of revisions on document quality.
The setting of evaluation metrics is the same as Du et al. (2022). We use the metrics package from Huggingface transformers (Wolf et al., 2020) to calculate the SARI, BLEU, and Rouge-L scores.

heading: 4.3 Models Setup and Baselines
text: Using our method, we train the models in two ways: the model that only trains the multi-prefix tuning stage and that trains both the multi-prefix tuning stage and the prefix transfer stage.
We compare our method with three baselines: full fine-tuning BART (BART-FineTune), full finetuning PEGASUS (PEGASUS-FineTune), and prefixtuning of BART with a single prefix (BARTSinglePrefix). Both BART and PEGASUS are generative models based on the transformer architecture. Compared to the edit-based model FELIX, they perform better. We use the results reported by Du et al. (2022) for these two models. Furthermore, we compare BART-SinglePrefix as a possible technical solution as we choose BART as our backbone model. BART-SinglePrefix trains only one prefix on the entire dataset.
All three baselines are trained with two config-
urations. The first configuration is using the pure sentence pairs without edit intention annotations to train the model. The second configuration is appending an edit intent token at the beginning of the input text during the training process, which is the same as the approach of Du et al. (2022).

heading: 5 Results and Analysis

heading: 5.1 Main Results
text: The main results are shown in Table 1. Compared to training with a single prefix, the setting of multiple prefixes can improve the results, especially training on ITERATER-HUMAN. Meanwhile, with fewer training parameters, the multi-prefix setting could achieve a comparable SARI score and better average score than the fully fine-tuned BART and PEGASUS baselines.
Moreover, prefix transfer could further improve the model’s performance. Training on ITERATERHUMAN, prefix transfer significantly improves the SARI score from 33.12 to 36.01 and gets the highest average score of 67.91. Training on ITERATERFULL, prefix transfer can also improve the average score from 67.23 to 68.36.
An interesting phenomenon is that training on different datasets results in different gains for prefix transfer in evaluation metrics. On ITERATERHUMAN, prefix transfer improves the SARI score significantly. While on ITERATER-FULL, prefix transfer mainly improves the BLEU score and Rouge-L score. One possible explanation is that in situations when the training data is small, prefix transfer tends to learn more editing operations to improve text quality. In this way, the SARI score related to editing operations will be improved significantly. When the training data is sufficient, pre-
fix transfer will model the gold reference in more detail. So the BLEU score and the Rouge-L score will be improved.

heading: 5.2 Analysis
text: We further tried to use different training data at different stages of training to conduct experiments. The results are shown in Table 2.
We find that the best practice is to train the model on ITERATER-FULL in the multi-prefix tuning stage and on ITERATER-HUMAN in the prefix transfer stage, which gets the highest SARI score and average score. This may be because of the different distributions of manually annotated edit intent and automatically annotated edit intent. The auto-annotated dataset ITERATER-FULL contains many incorrectly classified sentences, which may cause mismatched knowledge in prefixes. In the prefix transfer stage, due to the existence of mismatched knowledge and incorrectly classified sentences, the continued use of the same training data may finally cause a certain degree of negative transfer. However, if we use ITERATER-HUMAN in the prefix transfer stage, the impact of negative transfer will be mitigated, because ITERATER-HUMAN only contains correctly classified sentences.
In Appendix C, we separately provide the performance results on different edit intentions of the best-performing model.

heading: 6 Conclusion
text: In this paper, we introduce a new method for multiintent text revision. The system is based on prefixtuning, which first obtains a prefix for every edit intention and then learns to transfer the knowledge in prefixes for every input instance by training a prefix transfer module. This prefix transfer module is configured as two attention units that act respectively on the key states and the value states at each attention layer of the PLM. In this way, our method can make full use of the knowledge of various edit intentions and does not need to anno-
tate the intentions of the input. The experimental results show that our method significantly outperforms baselines, and both multi-prefix and prefix transfer settings could improve the performance.
Limitations
Due to the lack of multi-intent text revision datasets, we only conduct experiments on ITERATER. Although it is a multi-domain dataset, we only use its sentence-level data, and each sentence pair only contains one editing operation. The robustness of our method is still to be verified by evaluating it on more types of datasets in future work.
Another limitation of our work is that we only made improvements at the model level. We have noticed that Kim et al. (2022) recently improved text revision by leveraging extra data from other text editing tasks and performing editable span detection before revising. Similar methods can also be applied to our model and will be tried in our future work.
Ethics Statement
The PrefixTransfer method mainly aims at fusing multiple prefixes to obtain a unified model that can perform multi-intent text revision. The experiments are based on the ITERATER dataset, which is unlikely to include harmful content.

heading: Acknowledgments
text: This work was supported by the funds of the Research Project of the National Language Commission ( No. ZDI145-24), Natural Science Foundation of China (No. 62206042), Fundamental Research Funds for the Central Universities (No. 21PT04, No. N2216013) and China Postdoctoral Science Foundation (No. 2022M710022). We would like to thank all anonymous reviewers for their valuable comments and suggestions on this work.

heading: A Details on Computational Experiments
text: Our system is established based on BART-large (with 400 million parameters). We use the AdamW optimizer with weight decay and adopt Noam Optimizer. We initial the learning rate of 5e−5 in the multi-prefix tuning stage and 1e−5 in the prefix transfer stage. We set the warm-up steps to 4000. Regarding batch size, we use max-token configuration and set the maximum of tokens to 1024. The maximum epoch is set to 100. And we set an early stop strategy in the patience of 20 epochs.
The length of the prefix (with 40 million parameters) is 12 and the prefix vectors are not optimized directly but reparameterized via a bottleneck MLP which has a middle dimension of 512. We set the prefix dropout to 0.2.
We do validation every epoch while training the model on ITERATER-HUMAN and every 200 steps while training the model on ITERATER-FULL.
We report descriptive statistics with a single run. We deploy all our experiments on a slurm cluster. We train the prefixes on 4 Tesla V100-SXM2 (16GB) GPUs and train prefix transfer modules on an NVIDIA TITAN RTX (24GB) GPU.

heading: B Details of Dataset
text: B.1 Taxonomy
The taxonomy of edit intentions in ITERATER after removing Other:
• Fluency Fix grammatical errors in the text.
• Coherence Make the text more cohesive, logically linked, and consistent as a whole.
• Clarity Make the text more formal, concise, readable, and understandable.
• Style Convey the writer’s writing preferences, including emotions, tone, voice, etc.
• Meaning-changed Update or add new information to the text.
B.2 Data split ITERATER dataset is splited as in Table 3 after romoving the Other class.
B.3 License The ITERATER dataset uses Apache License, and it allows the data for academic usage.

heading: C Model Performance of Different Edit Intentions
text: ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: The Limitation section after conclusion and before the references

heading: 3 A2. Did you discuss any potential risks of your work?
text: The Ethics Statement section after Limitation

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section Abstract and Section 1

heading: 3 A4. Have you used AI writing assistants when working on this paper?
text: We use grammarly to to correct the full text
B 3 Did you use or create scientific artifacts?

heading: In Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: In Section 4

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: In Appendix B.3
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Our use of existing artifacts was consistent with their intended use. All for research purposes
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? The dataset we use is open source and does not involve such problems

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? In Section 1 Section 4 and Appendix B
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. In Appendix B
C 3 Did you run computational experiments?

heading: In Section 4 and 5

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? In Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? In Appendix A

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? In Appendix A

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? In Section 4
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: The robustness of our method is still to be verified by evaluating it on more types of datasets in future work. Another limitation of our work is that we only made improvements at the model level. We have noticed that Kim et al. (2022) recently improved text revision by leveraging extra data from other text editing tasks and performing editable span detection before revising. Similar methods can also be applied to our model and will be tried in our future work