On Improving Summarization Factual Consistency from Natural Language Feedback
Year: 2023

Abstract:
Despite the recent progress in language generation models, their outputs may not always meet user expectations. In this work, we study whether informational feedback in natural language can be leveraged to improve generation quality and user preference alignment. To this end, we consider factual consistency in summarization, the quality that the summary should only contain information supported by the input documents, as the user-expected preference. We collect a high-quality dataset, DeFacto, containing human demonstrations and informational natural language feedback consisting of corrective instructions, edited summaries, and explanations with respect to the factual consistency of the summary. Using our dataset, we study three natural language generation tasks: (1) editing a summary by following the human feedback, (2) generating human feedback for editing the original summary, and (3) revising the initial summary to correct factual errors by generating both the human feedback and edited summary. We show that DeFacto can provide factually consistent human-edited summaries and further insights into summarization factual consistency thanks to its informational natural language feedback. We further demonstrate that fine-tuned language models can leverage our dataset to improve the summary factual consistency, while large language models lack the zero-shot learning ability in our proposed tasks that require controllable text generation.

heading: 1 Introduction
text: While recent natural language generation (NLG) models (Radford et al., 2019; Lewis et al., 2020; Raffel et al., 2020; Brown et al., 2020) have made significant progress on the generation quality, they cannot always generate outputs that meet the user needs. For example, while state-of-the-art summarization systems can generate fluent and relevant
∗Most of the work was done while the first author was an intern at Microsoft Research.
summaries, recent work (Goyal and Durrett, 2021; Tang et al., 2022) have shown that they still make errors on fine-grained qualities such as factual consistency.1 These errors can lead to serious risks to the intended users and make it difficult for them to trust the systems for their decision-making.
Such failures in satisfying the user needs have an intrinsic reason – the large benchmark datasets that are used to train NLG models are usually not collected according to pre-defined user needs, which results in a discrepancy between model behaviors and user expectations. For example, XSum (Narayan et al., 2018), one of the most commonly used summarization datasets, contains a large portion of reference summaries with hallucinations.2 As a result, summarization models trained on XSum dataset generate many non-factual
1Following Goyal and Durrett (2021), we define factual consistency as the summary quality that all the information of the summary can be supported by the source document.
2Maynez et al. (2020) reports that around 76.9% reference summaries on the XSum dataset contains hallucinated contents that are not supported by the source documents.
15144
Chelsea weren’t awarded a penalty.
David Ospina clashed with Oscar.
David Ospina clattered Oscar.
David Ospina plays for Arsenal.
David Ospina is a goalkeeper.
The clash occurred inside the box.
Oscar is Brazilian.
Oscar was taken off at half time.
Didier Drogba replaced Oscar.
Atomic Content Units (ACUs)
ACU Writing
ACU Matching
1. Detect Errors
2. Categorize Errors
3. Give Explanation
4. Provide Evidence
5. Write Corrective Instructions
6. Correct Summary
contents, more than models trained on datasets such as CNN/DailyMa l (Herma n et al., 2015) dataset (Goyal and Durrett, 2021). Unfortunately, it can be prohibitively expensive to collect new, large-enough datasets to train NLG models according to user needs, as they ca be diverse, personal, and ever-changing over time.
Instead of aligning an existing NLG model to a specific user need, we explore adjusting model outputs according to the user needs through human demonstrations and feedback. Specifically, we investigate three scenarios (Fig. 1): (1) an Editing model that aligns initial system outputs to human demonstrations based on the user feedback; (2) a Critic model that predicts user feedback of initial system outputs according to the user requirements; (3) an Editor model that automatically aligns the initial system outputs to user needs by predicting both the user feedback and edited summary.
We choose factual consistency of systemgenerated summaries as the user-required quality to study the aforementioned application scenarios. To this end, we collect a high-quality, informational dataset containing human demonstrations and feedback. Specifically, the annotators are presented with initial system-generated summaries and asked to make changes to the summaries to make them factually consistent if they find errors in them. Apart from the human-edited, factually consistent summaries, the annotators are also required to provide instructions on how to change the initial summaries (i.e., if they find errors in them) and explanation on why the initial summaries are factually consistent or not. An example of our dataset is shown in Fig. 2. Using the collected dataset, we show that (1) the Editing model can effectively leverage human feedback to adjust the initial sys-
tem outputs towards human demonstrations; (2) the Critic model can learn to generate meaningful feedback that can be used by the Editing model; (3) the Editor model can utomatically correct factuality errors without explicit human intervention. Moreover, we find that the Editor model achieves better performance than the baseline model that only generates the edited summary, which indicates that natural language feedback can be beneficial for training models for the corresponding task.
Our contributions can be briefly summarized as: (1) we collect DeFacto,3 a high-quality dataset containing human Demonstrations and Feedback for improving factual consistency of text summarization; (2) we conduct comprehensive analyses on the collected dataset, which provides further insights about factual consistency in text summarization, such as the relation between the type of factual errors and the type of editing operations; (3) we provide strong baseline models for the proposed three NLG tasks – summary editing (Editing model), feedback generation (Critic model), and automatic factuality error correction with feedback prediction (Editor model), which illustrates methods of leveraging natural language feedback for aligning model outputs with user expectations. (4) we present two case studies with large language models (LLMs) such as GPT-3.5 (Ouyang et al., 2022b), showing that LLMs still lack the controllable text generation ability in our proposed tasks.

heading: 2 The DEFACTO Dataset
text: Our dataset, DEFACTO, contains human demonstrations and feedback w.r.t. the factual consistency of system-generated summaries. We choose XSum
3We make the DeFacto dataset publicly available at https: //github.com/microsoft/DeFacto.
dataset as the target dataset to conduct the data collection because it is the most commonly studied dataset for summarization factual consistency. For the system-generated summaries, we select PEGASUS (Zhang et al., 2020), a top-performing summarization model to generate summaries on both the validation and test set of the XSum dataset.

heading: 2.1 Annotation Process
text: Our annotation process follows the following steps: (1) Detect errors: The annotator is required to evaluate a summary given the source document and decide if the summary is factually consistent. (2) Categorize errors: If the annotator decides the summary is not factually consistent, they are required to categorize the factual errors in the summary as either intrinsic or extrinsic.4 We note that both error detection and categorization are defined at the summary level. (3) Give explanation: The annotator is required to provide a natural language explanation on why the summary is factually consistent or not. (4) Provide evidence: The annotator is required to select a sentence from the source document as evidence to support their claims described in (3). (5) Write corrective instruction: The annotator is required to provide instructions of how to correct the original summary if they think it is not factually consistent. To enforce uniformity and reduce the noise in the instructions, we provide six templates for the annotators corresponding to different operations: Remove, Add, Replace, Modify, Rewrite, and Others. The annotators need to fill in the templates to generate the instructions. The details of the templates are in Appendix A.1. (6) Correct summary: Following the instruction in (5), the annotator is required to edit the initial summary to make it factually consistent with minimal, necessary modifications. We provide annotated examples in Appendix A.2.

heading: 2.2 Data Collection
text: We conduct our data collection on Amazon Mechanical Turk5 (MTurk) platform. The MTurk annotators need to pass a qualification test to be able to accept our assignments. The qualification test
4Following Goyal and Durrett (2021), we define intrinsic errors as errors that arise as a result of misinterpreting information from the source article and extrinsic errors as errors that hallucinate new information or facts not present in the source article.
5https://www.mturk.com/
includes three actual annotation tasks, and we manually checked the correctness of the answers of the annotators and assigned them scores accordingly.
For the actual tasks, we collected one annotation for each example (i.e., a document-summary pair), and collected around 1000 examples on the test set and 1500 examples on the validation set. To estimate the inter-annotator agreement, we additionally collect two more annotations for 100 examples on the test set. We require the annotators to be located in the United States. Depending on the difficulty of the assignments, the annotators are compensated with 1.2 - 2.0 US dollars per assignment accordingly based on a $12/hour pay rate.
To check the inter-annotator agreement on steps (1) Detect Errors and (2) Categorize Errors in §2.1, we calculated the Krippendorff’s alpha (Krippendorff, 2011), and found that the agreement score is 0.5552, 0.1899, 0.5260 for if the summary contains extrinsic factual errors, intrinsic factual errors and any factual errors respectively. For humanwritten explanation, instructions, and edited summary in step (3), (5), (6) in §2.1, we calculated the ROUGE (Lin, 2004) score among the answers provided by different annotators, and found the average ROUGE-1 F-score to be 30.52, 50.96, 71.77, respectively. Lastly, for the evidence in step (4), we consider two sentences as equivalent if the ROUGE1 score between them is above 90. We found the match rate among different annotators to be 0.4403.

heading: 3 DEFACTO Analysis
text: With the collected annotations, we further split the data collected on the validation set of XSum dataset into a training set and a validation set for the following experiments. We perform data analyses with different aspects of the collected dataset. The basic dataset statistics are in Tab. 1. Out of all the examples, 71.1% of them contain at least one factual error, 58.8% of them contain extrinsic errors, 22.0% of them contain intrinsic errors, and 9.63% of them contain both types of errors.

heading: 3.1 Edited Summary
text: For the edited summaries written by the annotators, we evaluate (1) their factual consistency; (2) their textual similarity with either the reference summaries or the initial outputs; (3) other aspects of their intrinsic quality (Grusky et al., 2018; Bommasani and Cardie, 2020). Factual Consistency To evaluate the factual consistency, we use two automatic metrics, DAE (Goyal and Durrett, 2020) and QAFactEval (Fabbri et al., 2022a), which achieve strong performance on the XSum dataset (Tang et al., 2022).6 The results are in Tab. 2, showing that the human-edited summaries are more factually consistent than both the reference summaries and initial system outputs.
Text Similarity For textual similarity, we compare human-edited summaries against both the reference summaries and initial system outputs in Tab. 3. We note two observations: (1) There is a high-degree similarity between the initial system outputs and human-edited summaries, indicating that the annotators only made small changes to the initial outputs. (2) Compared with the initial system outputs, the human-edited summaries have lower similarity with the reference summaries, which suggests that the reference summaries and initial system outputs may share similar factual errors, leading to higher textual similarity.
6Automatic metric setting details are in Appendix B.
Intrinsic Evaluation We evaluate two intrinsic summary qualities: the compression rate (Grusky et al., 2018) and the abstractiveness (Bommasani and Cardie, 2020). In particular, compression rate measures the length difference between the input text and the summary. And to evaluate abstractiveness we use two features, (1) Extractive Fragment Coverage (Grusky et al., 2018), which measures the extent to which the summary can be “copied” from the input text, (2) Novelty, which measures the ratio of words in the summary that are not in the input text.7 The statistics in Tab. 4 suggest that the human-edited summaries are less abstractive than the initial system outputs and reference summaries. This finding is coherent with Xiao et al. (2022) which found that there exists a tradeoff between faithfulness and abstractiveness. However, we note that the decrease of abstractiveness can result from removing non-factual information from the summary, which is the most common operation for correct extrinsic errors, as we will show next.
7More details can be found in Appendix C.1.

heading: 3.2 Instructions
text: The annotators need to provide instructions on how to make changes to the initial system outputs to correct factual errors. We find that the editing can take more than one instruction and the average number of instructions is 1.52. We show the distribution of the number of instructions in Appendix C.2. As for the distribution of instruction types (Fig. 3), we found that removing information and replacing information to be the most frequent operations. Interestingly, fine-grained analysis in Fig. 3 shows that extrinsic errors are more likely to be corrected by the replacing operation while intrinsic errors can require more diverse types of operations.

heading: 4 Summary Editing for Improving Factual Consistency
text: With DEFACTO, we propose a new NLG task: editing the initial summary based on human feedback.

heading: 4.1 Methods
text: We formulate the summary editing task as a sequence-to-sequence (Seq2Seq) (Sutskever et al., 2014) problem. Specifically, a Seq2Seq model g learns a mapping from an input sequence X to a target output sequence Y : Y ← g(X).
For this specific task, the input X has three components: input document, initial system-generated summary and human feedback, while the target output is the human-edited summary (Fig. 1). The human feedback consists of the instructions and explanation. To concatenate the different components of the input sequence, a short “prompt” is appended at the beginning of each component, then the entire input sequence becomes: “Article: input document; Candidate: initial system-generated summary; Instruction: human instructions; Explanation: human explanation”. While recent work (Sanh et al., 2022; Bach et al., 2022) has shown that prompt design can affect the model performance, for simplicity we use simple text snippets for the baseline models.
We instantiate the Seq2Seq model using a family of pre-trained Encoder-Decoder models, T5 (Raffel et al., 2020) and T0 (Sanh et al., 2022), which are widely used for transfer learning and few-shot learning where the data is scarce. To achieve better performance, the model is fine-tuned on the training set of DEFACTO using Maximum Likelihood Estimation (MLE) under the training paradigm of teacher forcing (Williams and Zipser, 1989). We note that we only used the subset of data in which
the initial system output contains factual errors.

heading: 4.2 Experiments
text: Implementation Details To initialize the Editing model, we use T5-3B and two variants of T0 models, T0-3B and T0pp.8 We compare the model performance with different variants of input (e.g., with or without the human-written explanation). To evaluate the quality of the model output, we focus on two aspects: textual similarity with the human-edited summary, as evaluated by ROUGE (Lin, 2004), and factual consistency with the input document, as evaluated by DAE (Goyal and Durrett, 2020) and QAFactEval (QFE) (Fabbri et al., 2022a). The checkpoints are selected based on their performance on the validation set. Experimental Results Tab. 5 shows the performance of fine-tuned T0pp with different input variants. We note the following observations: (1) Compared with the initial system-generated summaries, the Editing model is able to generate summaries more similar to the human-edited summaries and more factually consistent with the input document. (2) Both the human-written instructions and explanation can provide meaningful guidance to the Editing model, and the model with both of them as input (the D+S+I+E variant) achieves the best performance. (3) Without the input document, the Editing model (the S+I variant) can still improve
8T5-3B (https://huggingface.co/t5-3b), T0-3B (https://huggingface.co/bigscience/T0_3B), and T0pp (https://huggingface.co/bigscience/T0pp) have around 3, 3, and 11 billion parameters respectively.
the initial system-generated summaries by following the instructions. However, taking the input document as part of the input helps the model (the D+S+I variant) to achieve better performance, especially for better factual consistency.
In Tab. 6, we compare the performance of T0pp, T0-3B and T5-3B with different kinds of inputs. We found that (1) the findings on T0pp (Tab. 5) are generalizable to T0-3B and T5-3B with few exceptions. (2) T0pp outperforms T0-3B across different input variants according to different automatic metrics except for the QAFactEval metric. (3) T0-3B generally outperforms T5-3B, likely thanks to the pre-training of T0 which is designed for performing zero-shot learning with instructions. Human Evaluation We conduct a human evaluation on the quality of model-edited summaries. We ask the annotators two questions: (1) Are the generated summaries more factually consistent than the original summaries (yes/no); (2) Do the generated summaries follow the instructions (yes/partly/no). We randomly sampled 100 examples from the test set, and have each generated summary annotated by three MTurk annotators. The generated summaries are from the trained checkpoint of T0pp with the input containing input document, initial systemgenerated summaries, and human-written instructions. Under major voting (with ties ignored), we found that 97% of model-edited summaries are more factually consistent than the original system outputs, and 91% of them follow the provided human-written instructions.

heading: 4.3 Case Study of LLM Summary Editing
text: As a case study, we evaluate the zero-shot learning ability of GPT-3.59 for summary editing. We apply it to two settings, (1) editing without instructions and (2) editing by following instructions, in a zero-
9OpenAI’s gpt-3.5-turbo-0301: https://platform. openai.com/docs/models/gpt-3-5.
shot learning manner.10 The results in Tab. 7 show that (1) GPT-3.5 is able to leverage the editing instructions; (2) Compared with the fine-tuned model (T0pp), GPT-3.5 can generate edited summaries with higher factual consistency but it is worse at maintaining the content similarity with the original summary, which suggests that it still struggles with controllable text generation.

heading: 5 Generating Feedback for Improving Factual Consistency
text: We investigate if it is possible to train a model to generate feedback from a given document and summary pair to correct factual errors, and we name the subsequent model as a Critic model.

heading: 5.1 Methods
text: Similarly to §4.1, we formulate the Critic model as a Seq2Seq model. The input sequence is a concatenation of the input document and the initial system-generated summary while the target output is the human-written instructions (Fig. 1).11 We use
10The prompts we used can be found in Appendix D.2. 11While it is also possible to require the Critic model to generate the explanation, we choose human-written instructions as the target output as it works better at helping the Editing model to improve the initial outputs (§4.2).
T0 as the startpoint to fine-tune the Critic model with MLE training on the subset of DEFACTO in which the initial summary contains factual errors.

heading: 5.2 Experiments
text: Experimental Results Tab. 8 shows the textual similarity between the instructions generated by the Critic model and the human-written instructions. To have a more intuitive understanding of the model performance, in Tab. 9 we evaluate the performance of the Editing model with the instructions generated by the Critic model. We found that (1) While the generated instructions cannot work as well as the human-written instructions, they are helpful to the Editing model to improve the factual consistency of the initial system-generated summaries. (2) Compared with the Editing model that only takes the input document and initial summary as input (D+S), the Editing model (D+S+I*) that also uses the generated instructions achieves better performance with respect to factual consistency, but its outputs have lower textual similarity with the human-edited summaries. It indicates that the Critic model can generate useful instructions. Meanwhile, the lower textual similarity may result from the fact that there can be more than one way
to correct the factual errors,12 and the Critic model can generate instructions for a way of correction different from the human-edited summary. Human Evaluation To further evaluate the quality of model-generated instructions, we ask human annotators two questions: (1) Are the generated instructions equivalent to human-written instructions (yes/partly/no); (2) Are the generated instructions useful for correcting the factual errors (yes/partly/no). Similar to §4.2, we randomly sampled 100 examples from the test set, and have each generated instruction annotated by three MTurk annotators. For the first question, we found that the annotators think 24% of the generated instructions are exactly equivalent to the human-written instructions while 45% of them are partly equivalent. For the second question, we found that 39% of the generated instructions are useful for correcting factual errors while 31% of them are partly useful. As a result, we found that it is easier for the Critic model to generate useful instructions than generating instructions that are equivalent to human-written instructions. We hypothesize this is because there can be more than one way to edit the initial summary therefore the human-written instructions represent only one acceptable solution.

heading: 5.3 Case Study of LLM Critic
text: As a case study, we evaluate the zero-shot learning ability of GPT-3.5 and GPT-413 for instruction generation. The results in Tab. 10 show that, compared with fine-tuned models, instructions generated by both GPT-3.5 and GPT-4 lead the editing model to generate summaries that are more factual but less similar to the original summaries. This finding shows a similar trend as in §4.3, that LLMs in a zero-shot learning setting lack the ability of con-
12For example, one may choose to remove a factual error or replace the error with factual information when appropriate.
13OpenAI’s gpt-4-0314: https://platform.openai. com/docs/models/gpt-4.
trollable text generation. For example, GPT-3.5 responded with “No editing instructions needed” 23.9% of the time, despite being directly instructed to edit a factually inconsistent summary.14

heading: 6 Summary Editor with Feedback Generation and Editing
text: We define the third NLG task as to predict both the human feedback and the edited summary given the input document and initial system-generated summary. We name this model the Editor model because it needs to both evaluate the initial summary and make edits according to its own assessments.

heading: 6.1 Correcting Known Factual Errors
text: Similar to §4.1 and §5.1, we fine-tuned the pretrained T0 and T5 models for our experiments. The two parts of the target output, the human feedback, and the edited summary are indicated by textual tags as specified in §4.1. We investigate two specific scenarios: (1) generating both the instructions and the edited summary; (2) generating both the explanation and the edited summary.
We present the experimental results in Tab. 11. Compared with the Editing model that takes only the input document and the initial system-generated summary as the input, the Editor models have better performance in textual similarity, and the one that generates the explanations also achieves higher
14Prompts and more details are in Appendix D.3.
factual consistency. The results suggest that learning to predict related information of a target generation task can be beneficial to the performance of language generation models, echoing the recent findings in chain-of-thought prompting (Wei et al., 2022; Huang et al., 2022; Jung et al., 2022).

heading: 6.2 Detecting and Correcting Factual Errors
text: While in the previous experiments the models are trained to edit the initial system outputs with known factual errors, the Editor model can also be used on arbitrary system outputs where it is required to edit the initial output only when it identifies factual errors in it. To this end, we use the entire DEFACTO in this experiment with the following modifications to the target output: (1) the target summary is set to the original system output when it contains no factual errors, and to the human-edited summary otherwise; (2) only explanations are used as part of the target output because it is always available.
We fine-tune T0-3B in this experiment and compare its results with several recently introduced summarization systems that are specifically trained to improve the summary factual consistency: (1) CCGS (Chen et al., 2021), (2) CLIFF (Cao and Wang, 2021), (3) ReDRESS (Adams et al., 2022), (4) FactPegasus (Wan and Bansal, 2022), (5) CompEdit (Fabbri et al., 2022b). More details about these systems can be found in Appendix D.1.
The results in Tab. 12 show that the Editor can achieve competitive performance compared with the baseline systems and yield a balanced performance between the content similarity with the reference summary and the factuality consistency. Since the Editor model is trained on much fewer data than
the other systems, its strong performance indicates the effectiveness of utilizing human demonstrations and feedback for improving factual consistency.

heading: 7 Related Work
text: Factual Consistency in Text Summarization Factual consistency is an important quality of text summarization systems (Kryscinski et al., 2020; Maynez et al., 2020; Fabbri et al., 2021). Related work has proposed various methods of improving the factual consistency of summaries by (1) training abstractive summarization models with factuality metrics (Goyal and Durrett, 2021; Cao et al., 2022), (2) introducing new training objectives and multi-task learning for model training (Cao and Wang, 2021; Zhu et al., 2021; Aralikatte et al., 2021; Zhang et al., 2022; Xu and Zhao, 2022), (3) post-editing or re-ranking the initially generated summaries to improve the factual consistency (Cao et al., 2020; Chen et al., 2021; Balachandran et al., 2022; Fabbri et al., 2022b), (4) designing factualityaware pre-training (Wan and Bansal, 2022).
To facilitate the evaluation of summarization models and automatic factuality metrics that evaluate the factual consistency of summaries, various benchmark datasets have been collected by the related work (Kryscinski et al., 2020; Wang et al., 2020; Huang et al., 2020; Fabbri et al., 2021; Goyal and Durrett, 2021; Pagnoni et al., 2021). In these benchmarks, system-generated summaries are evaluated by human annotators with either numerical quality scores, binary labels, or binary labels with fine-grained error taxonomies. In contrast, our dataset contains more detailed human feedback with natural language descriptions and provides error-free, human-edited summaries. Neural Text Editing Neural text editing models (Malmi et al., 2022) are suitable for application scenarios where there is a significant textual overlap between the input and output sequences (Awasthi et al., 2019; Malmi et al., 2019; Stahlberg and Kumar, 2020; Mallinson et al., 2020; Reid and Zhong, 2021; Mallinson et al., 2022), such as grammatical error correction, text simplification, and text style transfer. Instead of autoregressive generation, text editing can also be achieved by predicting and performing edit operations (Stahlberg and Kumar, 2020; Mallinson et al., 2020) or through non-autoregressive text generation (Gu et al., 2019; Agrawal and Carpuat, 2022). Unlike most of the related work, we propose a
text editing task that requires the editing models to follow the editing instructions. Faltings et al. (2021) introduces a similar dataset as ours containing single-sentence edits and the associated natural language commands crawled from Wikipedia. However, our dataset is different from theirs as we define a specific target quality, summary factual consistency, for the text edits and instructions. Improving Neural Models through Human Feedback Leveraging human feedback to improve neural models has become a recent research focus. InstructGPT3 (Ouyang et al., 2022a) use human feedback to improve initial predictions from a GPT3 model for better user preference alignments. Madaan et al. (2021) propose the interactive MERCURIE system, where users interactively correct the explanations generated by a reasoning system. In Xu et al. (2022), a generic chatbot is continuously trained using various forms of human feedback including natural language comments. Schick et al. (2022) propose a collaborative language model, PEER, which imitates a draft writing process and interactively refines a language generation task through human feedback. For text summarization, prior works (Stiennon et al., 2020; Wu et al., 2021; Nguyen et al., 2022; Scheurer et al., 2022) have studied training summarization models through human feedback in the form of numerical scores of summary quality and thus different from natural language feedback used in our work.

heading: 8 Conclusions
text: Using summary factual consistency as a target quality, we study improving text generation with human demonstrations and feedback. We demonstrate the usages of human feedback in three proposed NLG tasks using the collected dataset, DEFACTO, and show that human feedback can be used to improve summary factual consistency. We believe that our proposed tasks can be extended to other important text qualities beyond factual consistency, and utilizing natural language feedback for improving text generation can be a promising path for future work.

heading: Acknowledgements
text: We thank the anonymous reviewers for their valuable feedback and helpful suggestions.
Limitations
The annotation task we proposed in this work, i.e., detecting factual errors in summaries and providing human demonstrations and feedback for correcting the identified errors, can be complicated and timeconsuming. During our recruiting phase for MTurk annotators, we found that the ratio of annotators who were qualified after finishing the qualification test was relatively low. Therefore, it can be difficult to scale up the annotated dataset given the time and budget limitations. As a result, our dataset is of a relatively small scale and we only used one summarization dataset (XSum) and one base summarization model (Pegasus).
In this work, we view summary factual consistency as an example of user-expected quality to study leveraging natural language feedback for aligning system outputs with user preferences. However, user preferences can be diverse and personal and some user-expected output quality will be less well-defined and objective than summary factual consistency, which further increases the difficulty and ambiguity of data annotation and model evaluation. Therefore, it can be challenging to directly apply the methods we proposed in this work to such subjective quality aspects, and we leave it for future work to explore generalizing our methods to more diverse user expectations and preferences.

heading: A Data Collection Details
text: We used the XSum dataset for our data collection. It is released under the Apache 2 license and contains news articles written in English.
A.1 Instruction Templates We provide six templates for the annotators corresponding to different operations: Remove, Add, Replace, Modify, Rewrite, Others: (1) Remove the information about __ from the summary. (2) Add the information about __ to the summary. (3) Replace the information about __ with the information about __. (4) Modify the information about __ in the summary. (5) Rewrite the summary entirely by __. (6) Other instructions: __.
We note that sometimes it takes more than one instruction to edit the original summary.
A.2 Annotated Examples
We provide the following annotated examples. Example 1
Original Summary: A Wirral biscuit factory is to close with the loss of 342 jobs.
Explanation: Location is in Moreton (Morton?), not Wirral, and 342 families will be affected but that technically doesn’t translate to 342 jobs.
Instruction: Replace the information about Wirral with the information about Moreton. Replace the information about loss of 342 jobs with the information about affect 342 families.
Edited Summary: A Moreton biscuit factory is to close, affecting 342 families. Example 2
Original Summary: Two teenage girls have appeared at Teesside Crown Court accused of murdering a woman in Middlesbrough.
Explanation: The Teesside Crown Court was not mentioned by name, only the Youth court. The woman was found in Stephen Street and not Midlesbrough.
Instruction: Replace the information about Middlesbrough with the information about Stephen Street. Replace the information about Teesside Crown Court with the information about Teesside Youth Court.
Edited Summary: Two teenage girls have appeared at Teesside Youth Court accused of murdering a woman in Stephen Street. Example 3
Original Summary: Michael O’Halloran believes St Johnstone manager Tommy Wright will get the best out of him following his release by Rangers.
Explanation: the first name info in summary is not found in the source text. St Johnstone info is also not mentioned in the source.
Instruction: Remove the information about the first names of both people from the summary. Remove the information about Wright being the St Johnstone manager from the summary.
Edited Summary: O’Halloran believes Wright will get the best out of him following his release by Rangers. Example 4
Original Summary: Aberdeen’s Royal Concert Hall is to sell off hundreds of items of memorabilia as part of building work.
Explanation: The source text doesn’t state the name of the concert hall.
Instruction: Replace the information about Aberdeen’s Royal Concert Hall with the information about Aberdeen Performing Arts.
Edited Summary: Aberdeen Performing Arts is to sell off hundreds of items of memorabilia as part of building work. Example 5
Original Summary: Lancashire County Council’s decision to stop composting waste has been criticised as “catastrophic”.
Explanation: The original summary strongly implies that the decision to stop composting was “catastrophic” but the original text strongly implies that the composting program itself was a catastrophic failure versus stopping the program.
Instruction: Replace the information about the stoppage of the composting program being catastrophic with the information about how the composting program was a catastrophic failure.
Edited Summary: Lancashire County Council’s decision to stop composting waste shows the program has been a “catastrophic” failure. Example 6
Original Summary: Alex Goode and Ollie Devoto have been called up to England’s Six Nations training squad.
Explanation: The source text does not mention the name of England’s training squad.
Instruction: Remove the information about the name of England’s training squad from the summary.
Edited Summary: Alex Goode and Ollie Devoto have been called up to England’s training squad.

heading: B Factuality Metrics Setting
text: We use two factuality metrics, DAE (Goyal and Durrett, 2020) and QAFactEval (Fabbri et al., 2022a), in our experiments. For DAE, we transfer its dependency-level predictions into summarylevel scores. Specifically, following the notation of Goyal and Durrett (2021), we use d(S) to denote the dependency-parse of a summary S. For each arc a in d(S), the DAE metric predicts a label ya representing if this dependency is entailed by the input document (ya = 1 means entailment.) Then, we define a summary-level factuality score fS using the predictions:
fS =
∑ a ya |d(S)| . (1)
For QAFactEval, we directly use its prediction scores since they are summary-level scores.

heading: C DEFACTO Analysis
text: C.1 Intrinsic Summary Evaluation In §3.1, we use three metrics to evaluate the summary’s intrinsic quality. (1) Compression rate (Grusky et al., 2018) is defined as the ratio of the number of words in the input document D and in the summary S:
COMPRESSION(D,S) = |D| |S| . (2)
(2) Extractive Fragment Coverage (Grusky et al., 2018) is defined with extractive fragments (Grusky et al., 2018), F (D,S), which is a set of word sequences shared between the input document D and the summary S. Then, the Extractive Fragment Coverage is defined as
COVERAGE(D,S) = 1 |S| ∑
f∈F (D,S) |f |. (3)
(3) We define a word-level novelty between the input document D and the summary S as
NOVELTY = 1− |D ∩ S||S| . (4)
C.2 Human-written Instructions We find that it can take more than one instruction to perform the summary editing. Fig.4 shows the distribution of the number of instructions.

heading: D Experimental Details
text: We use T5-3B and two variants of T0 models, T03B and T0pp in our experiments.15 For the 3B
15T5-3B (https://huggingface.co/t5-3b), T0-3B (https://huggingface.co/bigscience/T0_3B), and T0pp (https://huggingface.co/bigscience/T0pp) have around 3, 3, and 11 billion parameters respectively.
models, it takes one 40GB GPU to train the model, and the training time is around 8 hours. For the 11B models, it takes eight 32GB GPUs to train the model, and the training time is around 20 hours. All the experiments converged in 50 epochs.
D.1 Baseline Summarization Systems In §6.2, we compare the performance of Editor model with the following summarization systems: (1) CCGS (Chen et al., 2021), which is based on contrastive candidate generation and selection. (2) CLIFF (Cao and Wang, 2021), which is trained with contrasting learning and synthetically generated contrastive examples. (3) ReDRESS (Adams et al., 2022), which is a summary post-editor that learns to remove factual errors through contrastive learning. (4) FactPegasus (Wan and Bansal, 2022), which is pre-trained and fine-tuned with factual-consistencyaware training objectives. (5) CompEdit (Fabbri et al., 2022b), which is a compression-based post-editing model that removes the non-factual entities from the original summary by performing summary compression.
D.2 Setting of LLM Case Study for Summary Editing
We use GPT-3.5 for the summary editing experiment with LLMs. To ensure stable results, we set the sampling temperature to 0. The prompt for summary editing without instructions is as follows:
You will be given an article and a summary of the article, which is not factually consistent with the article. That is, the summary contains information that is not supported by the article.
Your task is to edit the summary to make it factually consistent with the article. The correction should preserve most of the summary and only adapt it. Please only make the necessary changes to the summary. However, if you find all the information in the summary is not correct, please write a new summary of the entire article instead.
The edit operations are: 1. Remove Information, 2. Add Information, 3. Replace Information, 4. Modify Information, 5. Rewrite Summary 6. Others.
The summary should contain only one sentence. Please keep the style of the
summary unchanged, and the length of the summary should be similar before and after your edits.
Article: {{Article}}
Summary: {{Summary}}
Please edit the summary accordingly:
The prompt for summary editing with instructions is as follows:
You will be given an article and a summary of the article, which is not factually consistent with the article. That is, the summary contains information that is not supported by the article.
You will be given instructions about how to edit the summary to make it factually consistent with the article. Your task is to follow the instructions and edit the summary accordingly. The correction should preserve most of the summary and only adapt it. Please only make necessary changes to the summary, and keep the summary length close to the original length. The summary should contain only one sentence.
Article: {{Article}}
Summary: {{Summary}}
Instructions: {{Instruction}}
Please edit the summary accordingly:
D.3 Setting of LLM Case Study for Instruction Generation
The prompt we used for instruction generation is as follows:
You will be given an article and a summary of the article, which is not factually consistent with the article. That is, the summary contains information that is not supported by the article.
Your task is to generate instructions for editing the summary to make it factually consistent with the article. The correction should preserve most of the summary and only adapt it. Please only make the necessary changes to the summary.
The edit operations are: 1. Remove Information, 2. Add Information, 3. Replace Information, 4. Modify Information, 5. Rewrite Summary 6. Others.
Please note that “Remove Information” and “Replace Information” should be the majority of the operations. “Add Information” comes next.
The summary should contain only one sentence. Please keep the style of the summary unchanged, and the length of the summary should be similar before and after the editing.
Example:
Summary: A coalition of US civil rights groups has called on Facebook to do more to protect black people from racist abuse on the social network, saying the site is "racially biased".
The summary is not factually consistent because the source text does not contain the information called on Facebook to do more to protect black people, which is stated in the summary.
Instructions: Replace the information about the claim of the coalition with information about better moderation on the platform.
More instruction examples (summaries omitted):
- “Instruction 1”
- “Instruction 2”
- ...
Input:
Article: {{Article}}
Summary: {{Summary}}
Please generate the editing instructions for the summary to make it factually consistent with the article:
As discussed in §5.3, LLMs still lack the ability of controllable instruction generation. For example, GPT-3.5 responded with “No editing instructions needed” 23.9% of the time, despite being directly instructed to edit a factually inconsistent summary. Conversely, GPT-4 only made such mistakes in 1.8% of examples.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 9

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 9

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 2-6.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 1

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Yes. We described the license of XSum dataset in the Appendix. We will add descriptions of the license for the dataset we create upon acceptance.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No, due to time constraints, we will specify the data release license upon acceptance. Our use case is research-based and consistent with the underlying licenses.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? We use standard benchmark datasets that have been widely used so we expect the risk of their containing offensive or personal information is relatively low.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? In Appendix A.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 2.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? Section 4, 5, 6.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 4 and Appendix D

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? In Section 4, 5, 6, Appendix D, and we provide the hyperparameter settings in the training scripts, which is included in the supplementary material.
7 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? We only report the result one time since we only report the baseline model results for our collected dataset.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix B.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 2.

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Appendix A and supplementary materials.

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 2.
7 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? The annotators had signed consent before they accepted the annotation asks. Due to anonymity concerns, we will release the details of the consent upon acceptance.
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? We use standard benchmark datasets that have been widely used so we expect the risk of their containing offensive or personal information is relatively low.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Section 2.

heading: Future Work
text: Therefore, it can be challenging to directly apply the methods we proposed in this work to such subjective quality aspects, and we leave it for future work to explore generalizing our methods to more diverse user expectations and preferences