Venue: NAACL 2022

# FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for

Abstractive Summarization

David Wan & Mohit Bansal

University of North Carolina at Chapel Hill

{davidwan,mbansal}@cs.unc.edu

###### Abstract

We present FactPEGASUS, an abstractive summarization model that addresses the problem of factuality during pre-training and fine-tuning: (1) We augment the sentence selection strategy of PEGASUS's Zhang et al. (2020) pre-training objective to create pseudo-summaries that are both important and factual; (2) We introduce three complementary components for fine-tuning. The _corrector_ removes hallucinations present in the reference summary, the _contractor_ uses contrastive learning to better differentiate nonfactual summaries from factual ones, and the _connector_ bridges the gap between the pre-training and fine-tuning for better transfer of knowledge. Experiments on three downstream tasks demonstrate that FactPEGASUS substantially improves factuality evaluated by multiple automatic metrics and humans. Our thorough analysis suggests that FactPEGASUS is more factual than using the original pre-training objective in zero-shot and few-shot settings, retains factual behavior more robustly than strong baselines, and does not rely entirely on becoming more extractive to improve factuality.1

Footnote 1: Our code and data are publicly available at: [https://github.com/meetdavidwan/factpegasus](https://github.com/meetdavidwan/factpegasus).

+
Footnote †: _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1010 - 1028

## 1 Introduction

Abstractive summarization aims at generating short summaries that capture the essentials of a long document. Research in this challenging task has made significant progress with the help of large pre-trained models Lewis et al. (2020); Raffel et al. (2020); Zhang et al. (2020). However, current models suffer from the crucial problem of hallucinations Maynez et al. (2020), where a summary contains facts or entities not present in the original document. Such unfaithful generation raises the question of whether the models can be trustworthy and used safely for real-world applications. To tackle this problem, many approaches propose postprocessing models Chen et al. (2021); Dong et al. (2020); Liu and Liu (2021), but such methods are often constrained by external resources to train additional correction or selection models. An alternative line of works focuses on learning factuality directly during fine-tuning by filtering nonfactual training data Goyal and Durrett (2021); Nan et al. (2021) or, most recently, incorporating contrastive learning Cao and Wang (2021) to encourage generating faithful summaries.

In this work, we propose FactPEGASUS, a model that addresses the problem of hallucinations for abstractive summarization holistically, by incorporating factuality into the whole training pipeline: We tackle the lack of factuality objective in pre-training and the presence of hallucinations in the downstream dataset during finentuning. Current pre-training objectives focus on improving the quality of the generated output in the downstream tasks but often overlook the factuality aspect. Thus, we explore incorporating **factuality into the pre-training objective** of PEGASUS Zhang et al. (2020) (a state-of-the-art abstractive summarization model). The original objective, gap sentence generation (GSG), transforms any text into a pseudo-summarization dataset by selecting important sentences using ROUGE Lin (2004) as output summaries. We explore strategies for combining ROUGE and the factuality metric FactCC Kryscinski et al. (2020) as the selection criteria, so that the model learns to generate sentences that cover the most important information of the input document as well as remain faithful to it.

Next, we propose three complementary modules that further address factuality problems during fine-tuning: (1) **Corrector** that removes hallucinations existing in reference summaries, allowing training on the full training set without learning unit faithful behaviors; (2) **Contrastor** that encourages the model to better differentiate factual summaries from nonfactual ones by paying attention to the document using contrastive learning; (3) **Connec

tor**, a special mask-token fine-tuning technique enabled by the GSG-style objective, that simulates the pre-training task during fine-tuning by inserting the mask token into the input document so that the pre-trained model can adapt its knowledge of generating factual summaries directly to the downstream tasks. The connector, corrector, and contractor address the input, output, and training objective of the downstream task, respectively, and the combination of the components reduces potential confounding problems that cannot be addressed by a single module. We show that the full model improves three factuality metrics, the token and sentence error of DEP Entail (Goyal and Durrett, 2021) and FactCC, on the downstream datasets of XSum (Narayan et al., 2018), WikiHow (Koupaee and Wang, 2018), and Gigaword (Rush et al., 2015). Most notably, FactPEGASUS outperforms existing factuality-aware summarization models by more than \(40\%\) and \(34\%\) on XSum for token error and FactCC, respectively. Ablation studies show the usefulness of each of our fine-tuning components as well as the additive gain of combining our complementary modules, and human evaluation confirms that FactPEGASUS generates significantly more factual summaries over strong baselines.

Finally, we perform a detailed analysis of FactPEGASUS, which points to several important observations regarding learning and maintaining factuality: (1) **Zero-shot** setting demonstrates the utility of our actuality-aware pre-training objective, as our model outperforms PEGASUS (which uses the original objective) on all three factuality metrics when evaluated directly on the downstream task without any supervised training data. **Few-shot** experiment indicates that even a small number of nonfactual examples can have a strong negative impact on factuality and can nullify much of the gain from factuality pre-training, highlighting the importance of ensuring factuality during fine-tuning. (2) **Factuality dynamics**(Goyal et al., 2022) further shows that FactPEGASUS exhibits a lesser degree of factuality degradation than what is observed for BART-base. (3) **Factuality vs abstractiveness tradeoff curve** reveals that FactPEGASUS effectively improves factuality by not simply relying on the increase in extractiveness.

To summarize, our contributions are as follows:

1. We propose a factuality-aware pre-training objective for abstractive summarization and study the effect of different sentence selection strategies on downstream fractuality.
2. We introduce three complementary components for improving factuality during fine-tuning that correct hallucinations present in the training set, discourage unfaithful generation during training, and bridge the gap between pre-training and fine-tuning. The full model consistently achieves better factuality scores than strong baselines on three downstream abstractive summarization tasks, confirmed by human evaluation.
3. We conduct thorough factuality analysis and show that FactPEGASUS generates more factual summaries with no or little supervision, slows down factuality degradation observed for current models, and improves factuality not by becoming more extractive.

## 2 Related Work

#### Pre-training Objective for Generation Tasks.

Transformer-based models have achieved state-of-the-art performance for abstractive summarization Devlin et al. (2019); Lewis et al. (2020); Raffel et al. (2020); Zhang et al. (2020). Many such pre-trained models study the effect of useful pre-training objectives, often in the form of masking certain parts of the input. BART Lewis et al. (2020) randomly masks spans of tokens in the text as input and asks the model to reconstruct the original text. Our work builds on the success of PEGASUS's Zhang et al. (2020) pre-training objective that closely resembles the downstream summarization task. Their objective selects sentences that best represent the document as the output summary, and masks out the selected sentences in the original text as the input document. We explore various sentence selection strategies to encourage the model to generate summaries that cover the most important information of the document and also remain faithful to it.

#### Improving Factuality for Summarization.

Recent models can achieve highly fluent and coherent abstractive summaries, yet the generated summaries often contain factual errors Falke et al. (2019); Maynez et al. (2020). Several approaches have addressed this problem, which can be roughly categorized into two types. The first approach proposes post-processing models, that either removes hallucinations in the generated summaries Cao et al. (2020); Dong et al. (2020), or selects the most factual candidate during beam search Chen et al. (2021). This approach often requires training addi 

tional models and external resources. In an attempt to improve factuality in an end-to-end fashion, Nan et al. (2021) and Goyal and Durrett (2021) explore a useful method of removing nonfactual examples during training, but this only allows the model to be trained on a small portion of the training data.

Recently, contrastive learning Chopra et al. (2005), CL) has started to gain traction for improving factuality. Popular for representation learning, CL has had great success for vision tasks Chen et al. (2020) and has also been successfully applied to summarization, where Liu and Liu (2021) improves summary quality by differentiating high-quality summaries from the lower-quality ones. Cao and Wang (2021) extend this idea to improve factuality with various approaches to generate hallucinated summaries as negative examples, showing consistent improvement over existing methods. We similarly incorporate CL as an additional training objective, but we differ from previous works in the choice of anchor and positive sample. Inspired by Lee et al. (2021), who use encoder and decoder output as candidates for CL across multiple text generation tasks, we extend this idea to factuality, i.e., instead of performing CL only between summaries, we perform CL between the document and the summary. This setup encourages the model to generate a faithful summary that pays attention to the document, i.e., the definition of faithfulness.

## 3 FactPEGASUS

We describe our training procedure consisting of pre-training with a factuality-aware objective Section 3.1) and fine-tuning with three complementary modules for improving factuality (Section 3.2).

### Factuality-Aware Pre-training

Recent exploration of good pre-training objectives for abstractive summarization aims at achieving high quality on downstream tasks, often in terms of ROUGE. However, few have analyzed the effect of pre-training objective on factuality. We focus on incorporating this aspect into the pre-training objective of PEGASUS, gap sentence generation (**GSG**), since PEGASUS achieves state-of-the-art performance on the downstream abstractive summarization tasks. The GSG objective transforms text documents into a pseudo-summarization dataset by selecting important sentences as the output summary, which are subsequently masked out in the original text. The best strategy determines the importance by calculating ROUGE-1 between each chosen sentence and the rest of the document. While the original strategy selects sentences that contain the most unigram overlap, there is no guarantee that the selected sentences are faithful to the rest of the document. We provide an illustrative example in Figure 0(a), where the original objective selects sentence C due to its high ROUGE-1 score. However, this sentence is not a faithful summary to the rest of the document as the other sentences concern with the fire in Seattle while only sentence C talks about the fire in Denver.

To address this problem, we extend this objective, which we call factual GSG (**factGSG**), where we additionally measure the importance of the sentences according to factuality. We use FactCC Kryscinski et al. (2020) as the factality criteria when selecting the summary sentences, as it correlates highly with human factuality judgment Pagnoni et al. (2021) and is relatively fast to compute. FactCC produces a binary prediction where a score of 1 indicates that the selected sentence is consistent with the rest of the document. Another change in factGSG is the choice of gap sentence ratio, which determines the percentage of sentences in the text that will be selected as the summary. Instead of selecting \(30\%\) of the text document as output summary, we only select one sentence, as selecting more sentences will inevitably increase the possibility of hallucinations.

Formally, given a document \(D\) of \(n\) sentences, \(D=\{x_{1},x_{2},...,x_{n}\}\), we select the top-scoring sentence as the output summary, where the score of each sentence \(x_{i}\) is calculated by:

\[s_{i}=rouge(x_{i},D\backslash\{x_{i}\})+FactCC(x_{i},D\backslash\{x_{i}\})\]

Going back to the example in Figure 0(a), FactCC assigns a score of 0 to the nonfactual sentence C because the fire in Denver is not entailed by the other sentences. This results in sentence A scoring higher than the nonfactual sentence, and thus overcomes the problem in the original objective.

### Factuality-Aware Fine-tuning

Although the typical approach of updating all the model's parameters during fine-tuning adapts well to the downstream task, the model suffers from imitative falsehood Lin et al. (2021): The model learns to generate similar hallucinations present in the downstream dataset, and even completely forgets its factual behaviors learned during pre-training. This is especially problematic for datasets 

like XSum that contains hallucinations on \(70\%\) of the summaries Maynez et al. (2020).

To this end, we present three complementary fine-tuning modules, illustrated in Figure1b. Each component addresses different parts of the downstream task and collaboratively ensures factuality throughout the fine-tuning stage.

#### 3.2.1 Connector

The GSG objective enables faster and better adaptation during fine-tuning by simulating the downstream task Zhang et al. (2020). However, there still exists a gap between pre-training and fine-tuning: GSG is a masked sentence prediction task, but downstream summarization does not make use of the mask token. Thus, we simply insert the mask token into the input document of the downstream dataset, so as to simulate what the model expects during pre-training. This can be seen as a form of prompting, which helps us to elicit the factuality knowledge of the pre-trained models. We insert the mask token between sentences, and the best position is determined by evaluating the summarization performance on the validation set. We report the best position of the mask token and discuss the similarity to prompting in AppendixC.

#### 3.2.2 Corrector

The corrector removes hallucinations in the reference summaries so that such examples can be used during training without contributing to the problem of imitative falsehood. We consider summary entities as hallucinating if the text cannot be matched to one of the document entities. We propose three approaches with varying degrees of aggressiveness w.r.t. the removal of hallucinations and the possibility of generating ungrammatical sentences.

**Replace:** Upon qualitative analysis, we discover that some hallucinated entities in the summary are partially present in the documents. The most prominent example is the use of names, where the summary contains the full name of the person while only the first or last name is mentioned in the document, as shown in Figure2. Given such observation, we propose a method to find a similar entity with the same NER label in the document and use that to replace the original hallucinated entity in the summary. Although this approach cannot correct hallucinations where similar entities are missing in the document, grammaticality is ensured.

**Remove:** A more aggressive approach is to remove the hallucinated entities in the training examples. The intuition is that it is often better to not say anything than to say something wrong. We mitigate the problem of creating ungrammatical sentences by removing related words to the removed entities determined by dependency arcs.

**Combined:** As a middle ground that ensures no hallucinations are present in the reference summaries while being grammatical when possible, we first replace all possible entities and then apply the remove strategy on the remaining ones.

We refer the readers to AppendixB.1 for the details about hallucination detection, as well as the algorithm and discussion of grammatically for the remove method.

Figure 1: Illustration of FactPEGASUS. For pre-training (a), we use the factGSG objective introduced in Section3.1 that transforms a text document into a pseudo-summarization dataset. We select the pseudo-summary using the combination of ROUGE and FactCC. Here, sentence A is selected as the pseudo-summary, and we mask this sentence in the original text to create the pseudo-document. During fine-tuning (b), the connector (i) simulates the factGSG task by appending the same mask token used in (a) to the input document, so that we have the same setup in both training stages. Then, corrector (ii) removes hallucinations (highlighted in red) from the summary. Finally, contrastive learning in (iii) encourages the model to prefer the corrected summary over the perturbed summary.

 



#### 3.2.3 Contrastor

To better distinguish factual summaries from nonfactual ones, we next introduce a contrastive learning objective that encourages the model to prefer factual summaries given the context of the document. We use the document \(D_{i}\) as the anchor and only consider the reference summary \(S_{i}\) as the positive sample. Then, we create a set of nonfactual summaries \(N_{i}\) to form negative pairs following Kryscinski et al. (2020), where we replace factual entities with random entities of the same named entity labels. We experiment with two variants simulating either extrinsic and intrinsic hallucinations. As formulated in Maynez et al. (2020), extrinsic hallucinations refer to entities that are present in the summary but not in the document, whereas intrinsic hallucinations are those that are present in the document but contain inaccurate information or are misplaced. See Appendix B.2 for more details.

We stress that we perform contrastive learning between the document and the summary, similar to Lee et al. (2021), instead of between summaries Cao and Wang (2021), as it follows closer to the definition of faithfulness - the summary should be generated within the context of the document.

We use the NT-Xent loss Chen et al. (2020):

\[l_{D_{i},S_{i}}=-\log\frac{\exp\text{sim}(z_{D_{i}},z_{S_{i}})/\tau}{\sum_{S_{ j}\in N_{i}\cup\{S_{i}\}}\exp\text{sim}(z_{D_{i}},z_{S_{j}})/\tau}\]

where \(z_{D_{i}}\), \(z_{S_{i}}\) and \(z_{S_{j}}\) are representation for \(D_{i}\), \(S_{i}\) and \(S_{j}\), respectively. We generate \(z_{D}\) and \(z_{S}\) by performing mean pooling over the last hidden layer of the encoder and decoder output, respectively. \(\text{sim}(\cdot,\cdot)\) is the cosine similarity between the representations, and \(\tau\) is the temperature parameter.

The final loss is calculated by the sum of the cross-entropy loss \(L_{CE}\) and the contrastive loss: \(L=L_{CE}+\lambda L_{CL}\), where \(\lambda\) is a scalar.

## 4 Experimental Setup

We describe our experimental setup, and refer to Appendix A for more details.

### Datasets and Evaluation Metrics

We pre-train on the C4 dataset Raffel et al. (2020), and evaluate our pre-trained model on three downstream abstractive summarization datasets: XSum Narayan et al. (2018), WikiHow Koupaee and Wang (2018), and Gigaword Rush et al. (2015). XSum is the primary dataset for analysis unless otherwise stated, as most of the factuality works for abstractive summarization evaluate on this dataset. Dataset details are presented in Appendix A.1.

We report ROUGE-L Lin (2004) to evaluate our generated summaries against the reference. However, we note that this method is not ideal given the presence of hallucinations in the reference summaries Chen et al. (2021); Maynez et al. (2020): If a more factual model does not produce such hallucinations, the output is scored lower than those

Figure 3: Relative effect using different sentence selection criteria on XSum. Adding FactCC to criteria consistently improves factuality. Full result in Table 10.

Figure 2: Example output using different strategies of corrector and contractor. The first two rows show the original document and summary with highlighted entities and their respective labels (date, number, ent). We mark hallucinated entities in the summaries with red, factual entities in document and summary with green and underlined, and removed entities by the corrector with a strikethrough. Perturbed entities by the contrastor are _italicized_.



that contain the same hallucinations found in the reference.

To evaluate factuality, there have been many proposed automatic metrics Durmus et al. (2020); Wang et al. (2020); Scialom et al. (2021). We report FactCC Kryscinski et al. (2020) and DEP-Entail Goyal and Durrett (2021), as they are highly correlated with human judgment of factuality Pagnoni et al. (2021). For DEP-Entail, we report the token-level and sentence-level error. For FactCC, since the model has been trained to evaluate on single sentences, we calculate the average score across all sentences for each summary.

To confirm our observation, we conduct human evaluation asking Amazon Mechanical Turk2 (AMT) to judge the factuality and informativeness of the summaries. We randomly select 100 documents and ask the annotators to check whether each of the generated summaries is factual and informative. Appendix E provides more details.

Footnote 2: [https://www.mturk.com/](https://www.mturk.com/)

### Pre-training and Fine-tuning Setup

For pre-training, we use BART-base's architecture with PEGASUS's SentencePiece Kudo (2018) unigram model tokenizer. We first determine the best sentence selection criteria by experimenting with selection criteria that use ROUGE-1, ROUGE-2, and ROUGE-L, as well as combining each with FactCC. To save computation Lewis et al. (2020); Zhang et al. (2020); Raffel et al. (2020), we pre-train these models on a smaller dataset and fewer training steps. We report the effect of the selection criteria using the normalized ROUGE score and factuality scores over the model that uses ROUGE-1 as the selection criteria. We take the complement of token error and sentence error as token accuracy and sentence accuracy, respectively, to present all metrics where higher is better. Details of pre-training are shown in Appendix A.4.

Finally, We evaluate our pre-trained model on the three downstream tasks. As baselines, we compare our model to BART-base and PEGASUS*, our variant of the PEGASUS-base as there is no publicly available checkpoint. We train PEGASUS* by using the original sentence selection metric (ROUGE-1), and observe higher ROUGE scores on XSum and WikiHow than the ones reported in the original paper. We also compare FactPEGAGSUS to two summarization models optimized for factuality. DAE Goyal and Durrett (2021) uses DEP-Entail to mask out the nonfactual tokens during training, and CLIFF Cao and Wang (2021) uses contrastive learning between the reference summaries and automatically generated nonfactual summaries. We apply both methods to BART-base. Details are described in Appendix A.5.

## 5 Result

### Pre-training Sentence Selection Results

Figure 3 shows the effect of different sentence selection criteria. Adding FactCC to all three ROUGE-only criteria consistently improves all factuality metrics at the cost of a small decrease in quality. Overall, the selection strategy of combining ROUGE-1 and FactCC achieves the highest FactCC score out of all strategies while maintaining the smallest relative drop in ROUGE.

\begin{table}
\begin{tabular}{l c r|r r l} \hline \hline Dataset & Model & RL & tok errj & sent errj & FactCC \\ \hline \multirow{6}{*}{XS} & BART-base & **33.78** & 12.38 & 60.70 & 23.99 \\  & PEGASUS* & 33.17 & 12.33 & 60.01 & 24.14 \\ \cline{2-6}  & DAE & 31.78 & 4.794 & 35.52* & 25.43 \\  & CLIFF & 31.40 & 10.36 & 53.14 & 23.77 \\  & FactPEGASUS & 31.17 & **6.07** & **38.66** & **34.32** \\ \hline \multirow{6}{*}{WH} & BART-base & 31.81 & 8.99 & 45.77 & 99.09 \\  & PEGASUS* & 30.30 & 9.77 & 47.28 & 98.83 \\ \cline{2-6}  & DAG & 31.66 & 4.91* & 34.45* & 98.87 \\  & CLIFF & **33.82** & 13.74 & 57.42 & 99.18 \\  & FactPEGASUS & 29.33 & **7.86** & **42.40** & **99.41** \\ \hline \multirow{6}{*}{GW} & BART-base & **35.11** & 2.29 & 19.68 & 55.66 \\  & PEGASUS* & 34.74 & 2.84 & 22.66 & 56.43 \\ \cline{1-1} \cline{2-6}  & DAE & 35.57 & 0.58* & 7.54* & 59.61 \\ \cline{1-1}  & CLIFF & 34.89 & **1.72** & **18.45** & 58.53 \\ \cline{1-1}  & FactPEGASUS & 34.23 & 2.30 & 19.32 & **60.02** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Fine-tuning results on the XSum (XS), WikiHow (WH), and Gigaword (GW) dataset. FactPEGASUS consistently improves factuality metrics for all datasets over the two baseline models, and outperforms existing factuality models on FactCC. The token error and sentence error achieved by DAE (marked with *) is not a fair comparison, because the model optimizes the metric during training.

\begin{table}
\begin{tabular}{l c c} \hline \hline Model & Factuality & Informativeness \\ \hline BART-base & 24.67 & 61.33 \\ PEGASUS* & 27.33 & 58.33 \\ DAE & 31.99 & 61.66 \\ CLIFF & 29.33 & **62.99** \\ FactPEGASUS & **39.66** & 58.67 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Human evaluation results on XSum. Our model is statistically significantly better (\(p<0.05\)) than BART-base, PEGASUS*, and CLIFF, and moderately significantly better than DAE (\(p=0.055\)). There is no statistical significance between the informativeness of FactPEGASUS and other models (\(p>0.15\)).

 



### Fine-tuning Results

We present our full result on the three downstream tasks in Table 1. While the two baseline models achieve similar factuality scores, FactPEGASUS consistently improves factuality over the two baselines on all three datasets. The largest improvement can be seen for the XSum dataset, where FactPEGASUS, compared to BART-base, lowers the token error and sentence error by \(51\%\) and \(36\%\), respectively, and increases FactCC by \(43\%\)3 . The same trend but to a lesser degree can also be observed for WikiHow and Gigaword, most notably a 3-point decrease in sentence error for WikiHow and a 2-point increase in FactCC for Gigaword.

Footnote 3: We also experimented with a more aggressive corrector that can achieve more than \(50\%\) increase in FactCC and \(59\%\) improvement on sentence error on XSum, but this variant can hurt informativeness. Hence, the results can be tuned depending on the desired tradeoff between factuality and informativeness on the downstream task at hand.

Compared to factuality-aware models, FactPEGASUS achieves the highest FactCC on all tasks. Notably, FactpEGASUS outperforms DAE by \(34\%\) on XSum. In terms of DEP-Entail, FactPEGASUS outperforms CLIFF on XSum and WikiHow. We note that DAE is trained using the DEP-Entail metric and thus is not a fair comparison.

We note that the ROUGE-L scores for FactPEGASUS are lower than both baseline models by about 2 points, but we stress that our increase in FactCC is substantially larger than the decrease in ROUGE-L for XSum and Gigaword. The negative relationship between factuality metrics and ROUGE is also reported in prior works Chen et al. (2021); Kryscinski et al. (2019). For example, fine-tuning BART on a subset of XSum Goyal and Durrett (2021) improves fractuality at the cost of a 6-point drop in ROUGE-L4, which is triple the amount of decrease observed for our model.

Footnote 4: The result is reported in Cao and Wang (2021).

**Human Evaluation** results are shown in Table 2. The result agrees with our observation on automatic factuality metrics, as FactPEGASUS produces significantly more fractal summaries than the BART-base, and PEGASUS*, and CLIFF. We achieve moderately significantly better summaries (\(p=0.055\)) than DAE. Although, FactPEGASUS achieves low informativeness, we find no statistical significant difference between our model and other models (\(p>0.15\)).

### Fine-tuning Ablation Studies

We present ablation studies of our proposed methods in Table 3. We first compare the performance of different strategies for the corrector and contractor. For corrector, the level of aggressiveness in correcting hallucinations has a positive relationship with factuality metrics but a negative relationship with ROUGE-L. Although the remove method achieves the highest FactCC score, the combined method further lowers the token and sentence error while achieving relatively high ROUGE-L and FactCC. For contractor, simulating intrinsic errors, which creates more challenging negative samples, provides better factuality results than simulating extrinsic ones. Finally, we show the additive gain in combining the best corrector and contractor, as well as adding the connector to form the final model.

We report the same ablation studies for Gigaword and Wikihow in Appendix D.3, and that for PEGASUS* in Appendix D.4.

### Zero-shot and Few-shot Results

With the help of connector proposed in Section 3.2.1, we can explore how knowledge about factuality is transferred to fine-tuning, especially in

\begin{table}
\begin{tabular}{l c c c c} \hline \hline Model & RL & tok err\(\downarrow\) & sent err\(\downarrow\) & FactCC \\ \hline GSG+mask & 23.49 & 9.04 & 43.62 & 24.49 \\ factGSG+mask & **24.23** & **7.69** & **38.88** & **35.14** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Zero-shot results when applying the connector to our pre-trained model (factGGG+mask) and PEAGSUS*(GGG+mask). FactGSG+mask outperforms GSG+mask on all metrics.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline Model & RL & tok err\(\downarrow\) & sent err\(\downarrow\) & FactCC \\ \hline factdSG & **32.99** & 12.31 & 59.30 & 24.94 \\ \hline + corrector replace & 32.48 & 10.57 & 55.05 & 25.06 \\ + corrector remove & 30.37 & 6.44 & 39.89 & **35.77** \\ + corrector combined & 31.19 & 6.10 & 38.96 & 33.79 \\ \hline + contractor intrinsic & 32.14 & 11.46 & 57.61 & 25.26 \\ + contractor extrinsic & 32.54 & 11.95 & 59.10 & 25.07 \\ \hline + contractor + corrector & 31.17 & 6.08 & 38.92 & 34.17 \\ \hline \hline FactpegCASUS & 31.17 & **6.07** & **38.66** & 34.32 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Fine-tuning ablation on XSum. We present our pre-trained model factGSG fine-tuned without any of our proposed components, and adding different strategies of corrector and contractor. We then combine the best of the two modules (corrector combined and contractor intrinsic), and finally add the connector to form the final model, which we copy from Table 1.



the zero-shot and few-shot settings5.

Footnote 5: Strictly speaking, typical zero-shot and few-shot settings do not allow using the full validation set. However, we use validation results to decide the position of the mask token.

Zero-Shot.We apply the mask token to the best position and directly analyze the performance of the models on the test set. To better understand the effectiveness in transferring knowledge about summarization and factuality from the pre-training objective, we apply the connector to our pre-trained model (factGSG+mask) and PEGASUS* (GSG+mask), so that the two models differ only in their pre-training objective. We report the result in Table 4. FactGSG+mask outperforms GSG+mask on all metrics, especially for actuality metrics. Specifically, factGSG+mask lowers the sentence error by 5 points and increases FactCC by about 10 points. This observation confirms that the factGSG objective is more effective at capturing factuality than the original GSG objective.

Few-Shot.We follow a similar setup in Zhang et al. (2020), where we limit the number of training data to 1, 10, 100, and 1,000, and then fine-tune the model up to 2,000 steps with the patience of 10 epochs for early stopping. We select the checkpoint with the best validation performance.

We conduct this experiment by comparing FactPEGASUS to PEGASUS*, which has been shown for its ability to transfer with as little as 100 training examples (Zhang et al., 2020). In addition, we report the performance of factGSG+mask to understand how the the model is affected without explicitly ensuring factuality (i.e. without corrector and contractor). As shown in Figure 4, connector allows the model to better make use of the knowledge of pre-training and produces high-quality summaries, as both FactPEGASUS and factGSG+mask produces a ROUGE-L score comparable to PEGASUS* trained with 1000 examples.

In terms of factuality, we notice that with just 10 examples, PEGASUS* starts to degrade in factuality, which also applies to the factGSG+mask model. However, FactPEGASUS demonstrates an opposite trajectory: Sentence error decreases with more training data, and FactCC remains about the same score. This indicates that factual behavior is prone to be overwritten when factuality is not ensured explicitly, and thus calls for the importance of the corrector and contrastor.

### Factuality Dynamics during Fine-tuning

To see whether the factuality degradation observed in few-shot experiment also applies to the full fine

Figure 4: Zero-shot and few-shot results. The lines represent each models’s performance when fine-tuned on 0 (zero-shot), 1, 10, 100, and 1000 examples. FactPEGASUS consistently improves sentence error with more training data. Without the corrector and contrractor, factuality decreases with just 10 examples.

Figure 5: Factuality dynamics result. We show token error, sentence error, and FactCC as training progresses. FactPEGASUS slows down factuality degradation for all metrics compared to BART-base.

 

tuning process, we extend our analysis by studying the factuality dynamics, similar to Goyal et al. (2022). The authors observe an increase in sentence errors with the BART model during fine-tuning, and we analyze whether similar factuality degradation occurs for FactPEGASUS. We save checkpoints of our models every \(10\%\) of the total training steps, and evaluate the models on all three factuality metrics. Figure 5 shows the factuality dynamics during fine-tuning. We notice that the degradation occurs for both models but at a different degree. The token and sentence error for BART-base increase by 2 and 8 points, respectively. However, factuality for FactPEGASUS remains similar, with only an increase of 1 point for token error and 4.8 points for sentence error. The degradation is only about half of what is observed with BART-base, indicating that FactPEGASUS is better at avoiding learning nonfactual behaviors.

### Factuality vs Abstractiveness Tradeoff

Lastly, we wish to understand whether our proposed method is effectively improving factuality without relying on the increase in extractiveness. To this end, Ladhak et al. (2021) introduces a faithfulness-abstractiveness trade-off curve to measure the faithfulness given the model's extractiveness. The authors kindly provided the same set of examples for Gigaword and AMT template for calculating the faithfulness score.

We show our result on Gigaword in Figure 6. We include the result of their proposed Selector and previous works, including Loss Truncation Kang and Hashimoto (2020) and DAE Goyal and Durrett (2021). We note that the baseline models increase factuality but mostly due to an increase in extractiveness and thus fall below the curve. In contrast, FactPEGASUS lies above the line, indicating that we are effectively increasing factuality without relying much on becoming more extractive.

## 6 Conclusion

In this work, we proposed FactPEGASUS, a model for abstractive summarization consisting of factuality-aware pre-training and modules for ensuring factuality during fine-tuning. We demonstrated the effectiveness of our model at improving factuality on three downstream abstractive summarization datasets, confirmed by our human evaluation. Our analysis showed that our proposed factuality-aware pre-training objective is effective at capturing knowledge of factuality compared to the original objective and that our fine-tuning modules reduce the factuality degradation observed with current models. We finally showed that improvement in factuality is not solely explained by the increase of extractiveness.

## 7 Ethical Impact

Our work aims at reducing the risk of generating hallucinations, and even possibly misinformation, for abstractive summarization models so that such models can be used safely for real-world applications. While we demonstrate that we can alleviate this problem, we stress that there is still a long way to go for improving factuality. Thus, we stress that such models should be used with caution for real-world applications.

## Acknowledgment

We thank the reviewers for their helpful comments. We also thank Shiyue Zhang and Xiang Zhou for useful discussions and comments on the paper. This work was supported by NSF-CAREER Award 1846185 and NSF-AI Engage Institute DRL-211263.

## References

* Cao et al. (2020) Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020. Factual error correction for abstractive summarization models. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 6251-6258, Online. Association for Computational Linguistics.

Figure 6: Faithfulness-abstractness trade-off curve, shown as the dashed red line, on Gigaword dataset. We plot each model’s average faithfulness score evaluated by AMT against its extractiveness level. Our model lies above the graph, performing better than MLE-baseline, DAE Goyal and Durrett (2021), and Loss Truncation Kang and Hashimoto (2020).

 

Shuyang Cao and Lu Wang. 2021. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 6633-6649, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Chen et al. (2021) Sihao Chen, Fan Zhang, Kazoo Sone, and Dan Roth. 2021. Improving faithfulness in abstractive summarization with contrast candidate generation and selection. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5935-5941, Online. Association for Computational Linguistics.
* Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In _Proceedings of the 37th International Conference on Machine Learning_, volume 119 of _Proceedings of Machine Learning Research_, pages 1597-1607. PMLR.
* Chopra et al. (2005) S. Chopra, R. Hadsell, and Y. LeCun. 2005. Learning a similarity metric discriminatively, with application to face verification. In _2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)_, volume 1, pages 539-546 vol. 1.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Dong et al. (2020) Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, and Jingjing Liu. 2020. Multi-fact correction in abstractive text summarization. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 9320-9331, Online. Association for Computational Linguistics.
* Dreyer et al. (2021) Markus Dreyer, Mengwen Liu, Feng Nan, Sandeep Atluri, and Sujith Ravi. 2021. Analyzing the abstractiveness-factuality tradeoff with nonlinear abstractiveness constraints. _CoRR_, abs/2108.02859.
* Durmus et al. (2020) Esin Durmus, He He, and Mona Diab. 2020. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 5055-5070, Online. Association for Computational Linguistics.
* Efron and Tibshirani (1993) Bradley Efron and Robert J. Tibshirani. 1993. _An Introduction to the Bootstrap_. Number 57 in Monographs on Statistics and Applied Probability. Chapman & Hall/CRC, Boca Raton, Florida, USA.
* Falke et al. (2019) Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 2214-2220, Florence, Italy. Association for Computational Linguistics.
* Goyal and Durrett (2021) Tanya Goyal and Greg Durrett. 2021. Annotating and modeling fine-grained factuality in summarization. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1449-1462, Online. Association for Computational Linguistics.
* Goyal et al. (2022) Tanya Goyal, Jiacheng Xu, Junyi Jessy Li, and Greg Durrett. 2022. Training dynamics for text summarization models. In _Proceedings of ACL_.
* Kang and Hashimoto (2020) Daniel Kang and Tatsunori B. Hashimoto. 2020. Improved natural language generation via loss truncation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 718-731, Online. Association for Computational Linguistics.
* Koupaee and Wang (2018) Mahnaz Koupaee and William Yang Wang. 2018. Wikihow: A large scale text summarization dataset.
* Kryscinski et al. (2019) Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Neural text summarization: A critical evaluation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 540-551, Hong Kong, China. Association for Computational Linguistics.
* Kryscinski et al. (2020) Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 9332-9346, Online. Association for Computational Linguistics.
* Kudo (2018) Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 66-75, Melbourne, Australia. Association for Computational Linguistics.
* Ladhak et al. (2021) Faisal Ladhak, Esin Durmus, He He, Claire Cardie, and Kathleen McKeown. 2021. Faithful or extractive? on mitigating the faithfulness-abstractiveness trade-off in abstractive summarization.
* Lee et al. (2021) Seanie Lee, Dong Bak Lee, and Sung Ju Hwang. 2021. Contrastive learning with adversarial perturbations for conditional text generation. In _International Conference on Learning Representations_ 

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. Association for Computational Linguistics.
* Lhoest et al. (2021) Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Sasko, Gunjan Chhablani, Bhavitya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Party, Angelina McMillan-Major, Philipp Schmid, Sylvain Gugger, Clement Delangue, Theo Matsuisere, Lysandre Debut, Stas Beknem, Pierric Cistac, Thibault Goehringer, Victor Mustar, Francois Lagunas, Alexander Rush, and Thomas Wolf. 2021. Datasets: A community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online. External Links: Link Cited by: SS4.3.
* C. Lin (2004)ROUGE: a package for automatic evaluation of summaries. In Text Summarization Branches Out, Barcelona, Spain. External Links: ISBN 978-3-642-4865-3 Cited by: SS4.3.
* S. Lin, J. Hilton, and O. Evans (2021)Truthfulqa: measuring how models mimic human falsehoods. Cited by: SS4.3.
* Y. Liu and P. Liu (2021)SimCLS: a simple framework for contrastive learning of abstractive summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Online. External Links: Document, Link Cited by: SS4.3.
* R. L. Logan, I. Balazevic, E. Wallace, F. Petroni, S. Singh, and Sebastian Riedel (2021)Cutting down on prompts and parameters: simple few-shot learning with language models. External Links: Link Cited by: SS4.3.
* J. Maynez, S. Narayan, B. Bohnet, and R. McDonald (2020)On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. External Links: Link Cited by: SS4.3.
* F. Nan, R. Nallapati, Z. Wang, C. Nogueira dos Santos, H. Zhu, D. Zhang, K. McKeown, and B. Xiang (2021)Entity-level textual consistency of abstractive text summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Online. External Links: Link Cited by: SS4.3.
* S. Narayan, S. B. Cohen, and M. Lapata (2018)Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium. External Links: Link Cited by: SS4.3.
* A. Pagnoni, V. Balachandran, and Y. Tsvetkov (2021)Understanding factuality in abstractive summarization with FRANK: a benchmark for factuality metrics. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online. External Links: Link Cited by: SS4.3.
* C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020)Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research21 (140), pp. 1-67. External Links: ISSN 1536-490X, Link, Document Cited by: SS4.3.
* N. Reimers and I. Gurevych (2019)Sentence-BERT: sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 3982-3992. External Links: Link, Document Cited by: SS4.3.
* A. M. Rush, S. Chopra, and J. Weston (2015)A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 379-389. External Links: Link, Document Cited by: SS4.3.
* T. Sciulom, P. Dray, S. Lamprier, B. Piwowarski, J. Staiano, A. Wang, and P. Gallinari (2021)QuestEval: summarization asks for fact-based evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, Association for Computational Linguistics, Cited by: SS4.3.
* A. Wang, K. Cho, and M. Lewis (2020)Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. External Links: Link Cited by: SS4.3.
* T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Fantowicz, J. Davison, S. Shleifer, P. von Platen, 

Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 38-45, Online. Association for Computational Linguistics.
* Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In _Proceedings of the 37th International Conference on Machine Learning_, ICML'20. JMLR.org.

## Appendix A More Details on Experimental Setup

### Datasets

Following PEGASUS, we pre-train on the C4 dataset, a large collection of documents from Common Crawl. We evaluate our pre-trained model on three downstream abstractive summarization datasets: XSum, WikiHow, and Gigaword. XSum is a collection of articles from the British Broadcasting Corporation, Gigaword is a large collection of news articles headlines, and WikiHow consists of how-to articles.

We show the dataset statistics for pre-training and fine-tuning in Table 5, where we present the number of examples in the training, validation, and test splits. We also show the number of examples corrected using the replace and remove method. All datasets are from _datasets_Lioest et al. (2021).

### Evaluation Metrics

We use the ROUGE package provided by _transformers_Wolf et al. (2020). We follow the instructions provided by the authors of the factuality metrics to set up and run their code. We report all scores of our models from single runs.

### Training Details

We use _ transformers_ library for the training script and the checkpoints of the pre-trained models. We use the default setting, including the AdamW optimizer and the linear rate scheduler. We also use mixed precision for both pre-training and fine-tuning the models. We conduct our experiments on the RTX A6000 GPU with 48GB memory and the A100 GPU with 40GB memory. BART-base model has 139M parameters, and PEGASUS* and FactPEGASUS have 175M parameters.

### Pre-training Setup

Model Architecture.We use the same architecture as BART-base. Specifically, the model has \(L=6\), \(H=768\), \(F=3072\), \(A=12\), where \(L\) is the number of layers, \(H\) is the hidden size, \(F\) is the dimension for feed-forward layer, and \(A\) is the number of self-attention heads. We use the SentencePiece Kudo (2018) unigram model tokenizer from PEGASUS with a vocabulary size of 96,103.

Sentence Selection Criteria.Before pre-training the full model, we first determine the best sentence selection criteria that produces more factual summaries with comparable quality. We experiment with sentence selection criteria that use ROUGE-1, ROUGE-2, and ROUGE-L, as well as combining each with FactCC. To understand the effect of the pre-training objective on actuality directly, we evaluate the performance on the XSum dataset without applying any of our proposed fine-tuning modules. Following Zhang et al. (2020), we report the models' relative performance to the base model, which only uses ROUGE-1 as the selection criteria. We use the normalized ROUGE F1 scores \(\frac{1}{3}(\frac{RL}{RL_{base}}+\frac{R2}{RL_{base}}+\frac{RL}{RL_{base}})\), where \(RL_{base}\), \(RL_{base}\), and \(RL_{base}\) are the ROUGE F1 scores of the base model. We similarly report the factuality metrics by normalizing each score by that of the base model. We take the complement of token error and sentence error as token accuracy and sentence accuracy, respectively, to present all metrics where higher is better.

Similar to previous works Lewis et al. (2020); Zhang et al. (2020); Raffel et al. (2020) that save computational resources when selecting strategies for pre-training, we pre-train these model on the _realnewslike_ subset of the C4 dataset with less steps.

Pre-training Details.We use a learning rate of 1e-4, a weight decay of 0.01, and set the maximum number of input tokens to be 512 and a maximum number of output tokens to be 256. We use a batch size of 256. We pre-train the full model for 750,000 steps with a warm-up of 20,000 steps, and only pre-train the smaller models for the sentence selection criteria experiment for 250,000 steps. Pre-training the smaller models takes 30 hours, and pre-training the full model takes 90 hours.

Calculating FactCC Score.In practice, running FactCC on each sentence-document pair of the pre-training data is expensive. Thus, we opt to only calculate the FactCC score for the top 5 sentences 

according to the ROUGE score between the sentence and the rest of the document.

### Fine-tuning Setup

For all datasets, we use a label smoothing of 0.1. For decoding, we use a beam size of 6 for all datasets. Task-specific hyper-parameters are shown in Table 6. Fine-tuning on XSum and WikiHow takes 8 hours, and fine-tuning on Gigaword takes 11 hours. Decoding on XSum and Gigaword takes half an hour, while decoding WikiHow takes an hour. We use 5 negative examples for the contractor and set \(\lambda\) to 5 when calculating the combined loss. We set the temperature \(\tau\) to 0.05.

For fine-tuning DAE and CLIFF, we follow the authors' instructions and fine-tune BART-base with their respective code and hyper-parameters. For WikiHow and Gigaword, we use the same hyperparameters as above.

## Appendix B Implementation Details for Corrector and Contractor

### Corrector

We use spaCy's NER model6 to find entities in the document and summary. Entities in the summery sentence are considered nonfactual if no matching document entities with the same string are found. We have previously experimented with the additional requirement of matching entity type similar to Kryscinski et al. (2020), but we find that this constraint unintentionally causes some correct entities to be considered hallucinating, leading to unnecessarily less informative summaries when removed.

Footnote 6: We use the en_core_web_trf model.

Given hallucinated entities, we can perform either replace or remove operations. For replace, we find document entities whose words are all contained in the selected entity.

For the remove method, we need to make sure to also remove any related words. We use spaCy's dependency parser to systematically remove those. The algorithm is as follows: We first add all the tokens in the selected hallucinated entity to the list of tokens to remove. Then, we recursively find all parents that contain the dependency relation of _pobj_ and _prep_ without any other children and add those to the tokens to remove. Finally, we add all children that do not have the label _compound_, _relcl_, and _fixed_. The final set of words will then be removed in the summary sentence.

We qualitatively observe that this approach can cover most of the edge cases that would otherwise result in ungrammatical sentences. Nevertheless, this method is not perfect. We include some sample output with the remove method in Figure 7. The algorithm is good at removing entities and related words, such as prepositions, as illustrated in example 1, 3, and 5. However, we observe that it will create ungrammatical sentences when the hallucinated entity is the subject (example 2), or the object of a transitive verb (example 6).

We leave exploration with the best systematic correction algorithm or models for future work.

### Contrastor

Similar to Kryscinski et al. (2020), we generate hallucinated summaries by performing entity perturbation on the original summaries. We find entity candidates using the NER labels and sort them into three categories: We include MONEY, QUANTITY, and CARDINAL as _number_, DATE and TIME as _date_, and all other labels as _namedentities_. We randomly select a factual entity in the summary and replace it with an entity belonging to the same category.

For extrinsic hallucinations, we sample candidates of the same category from the training corpus but exclude those present in the document. For the intrinsic case, we select to consider the entities from the document. The number of negative examples for all tasks is 5.

\begin{table}
\begin{tabular}{r r c c c c} \hline \hline Dataset & Train & Validation & Test & \multicolumn{2}{c}{Corrector} \\  & & & Replace & Remove \\ \hline C4 & 364,868,892 & 364,608 & - & - & - \\ _realnewslike_ & 13,799,838 & 13,863 & - & - & - \\ \hline XSum & 204,045 & 11,332 & 11,334 & 54,036 & 152,716 \\ WikiHow & 157,252 & 5,559 & 5,577 & 8,077 & 71,936 \\ Gigaword & 3,803,957 & 178,651 & 1,951 & 115,896 & 1,296,168 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Dataset Statistics. We show the number of examples in each split, as well as the number of training examples changed using the replace and remove strategy of the corrector.

 



## Appendix C Connector Result

This mask-token fine-tuning technique can be seen as a form of prompting, where we elicit our desired faithful abstractive summarization behavior from the pre-trained model directly. Specifically, we consider this as null-prompting Logan et al. (2021), where using the mask token as the prompt can achieve competitive results with manually engineered prompts. Conveniently, since the mask token during pre-training already serves as a placeholder of where the summary sentence should be generated, it naturally serves as a valid prompt. Figure 0(b) shows an example of adding the mask token before the first sentence and thus creating a similar setup for pre-training.

We first need to determine the best position of mask token, as discussed in Section 3.2.1, where we insert the mask token before the \(i\)th sentence of the document, where \(i=1,2,...,6\), and select the best position that achieves the highest ROUGE score on the dev collection. We report ROUGE score of all positions in Table 7 for the three datasets. Interestingly, we observe that the best mask token position for all datasets is before the first sentence. This agrees with the dataset generation of XSum: the summary is taken from the first sentence of the original article. For Gigaword, there is not a change after the first sentence, since the document only consists of a single sentence.

## Appendix D Additional Results

### Sentence Selection Criteria Result

We report the full result for the sentence selection criteria in Table 10. Surprisingly, each sentence selection criteria that uses FactCC excels in one specific factuality metric: R1+FactCC is best at FactCC, R2+FactCC is best at sentence error, and RL+FactCC is best for token error.

### Zero-shot and Few-shot

We present additional results of the zero-shot and few-shot experiments here.

Zero-shotWe first report the reference-based result of the two baseline models and FactPEGA

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline Dataset & Learning rate & Num Steps & Warmup & Batch size & Max Input tokens & Max Target tokens \\ \hline XSum & 3e-5 & 15k & 500 & 256 & 512 & 64 \\ WikiHow & 3e-5 & 15k & 500 & 256 & 512 & 256 \\ Gigaword & 3e-5 & 50k & 2000 & 256 & 128 & 32 \\ \hline \hline  & & & & & & \\ \end{tabular}
\end{table}
Table 6: Hyperparametrs for fine-tuning on the three tasks.

\begin{table}
\begin{tabular}{l r r|r l} \hline \hline Model & R1 & R2 & RL \\ \hline BART-base & 19.75 & 2.61 & 12.81 \\ PEGASUS* & 18.03 & 2.65 & 13.02 \\ FactPEGASUS & **32.97** & **11.42** & **25.41** \\ \hline \hline \end{tabular}
\end{table}
Table 8: ROUGE score in zero-shot setting on XSum. We apply the connector to our model. FactPEGASCSU outperforms BART base and PEGASUS* on all metrics.

\begin{table}
\begin{tabular}{c c c|r l} \hline \hline Num Examples & RL & tok err. & sent err. & FactCC \\ \hline
0 & 25.44 & 7.69 & 38.88 & 35.14 \\
1 & 22.36 & 7.69 & 38.85 & 35.15 \\
10 & 23.44 & 7.69 & 38.85 & 37.78 \\
100 & 25.44 & **5.15** & 36.48 & **37.95** \\
1,000 & **27.03** & 5.67 & **32.97** & 34.38 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Full Result of zero-shot and few-shot experiments.



SUS in Table 8. Due to the mismatch of pre-training and fine-tuning, we observe that both baseline models perform much worse than their result when fully trained. However, with the help of the connector, we observe 11.5 ROUGE-1 points increase for our model compared to the baseline models, and almost four times and double the score for ROUGE-2 and ROUGE-L, respectively.

Few-shotWe show FactPEGASUS's full result of the few-shot experiment in Table 9.

### Fine-tuning ablation on Gigaword and WikiHow

We report ablation of each fine-tuning components on Gigaword and Wikihow. The result can be found in Table 11. We observe similar trend as Table 3, where each component improves the performance. For WikiHow and Gigaword, the extrinsic method for contrastive learning perform the best. We think that this is due to the fact that the two tasks do not contain rich entities in the document, and thus require introduction of additional entities from the training corpus.

### Fine-tuning ablation using PEGASUS*

We similarly perform the same ablation using the PEGASUS* model, which we present in Table 12. We observe similar trend as Table 3. We note that using our pre-trained model factGSG achieves better factuality than PEGASUS* in each setting.

## Appendix E Human Evaluation Detail

To ensure high-quality annotations, we select the workers from the United States and have more than 10,000 number of HITS approved as well as an approval rate greater than 98%. In addition, we also create a qualification test where we rate the factuality of the selected generated summaries. Such examples include cases where some summaries hallucinate the first name of a person, which the workers should mark them as not factual. Only workers with the correct annotation can perform the actual task.

To avoid giving too much text to the workers, we select the most important sentences and replace the less relevant sentences with an ellipsis. For each of the summaries, we select the ten most relevant sentences from the document by cosine similarity of the sentence embedding using SentenceTransformer7Reimers and Gurevych (2019). We combine and show all the selected relevant sentences from each summary. Since the summaries are similar, we see a large overlap of the relevant sentences.

Footnote 7: We use the all-mpnet-base-v2 model.

We give the following prompt, which we modify from Dreyer et al. (2021):

* _consistency/factuality_: Please avoid using general knowledge, and only consider it in the context of the provided document. Select not consistent if facts in the summary are not supported by the document, such as cases like these: 1. The summary contradicts the information in the document. The summary might say "A fire broke out in Seattle", but a document says it broke out in Portland. Or the summary might say "the Republicans won the election", but the document indicates the Democrats won instead 2. The summary adds (hallucinates) a fact that is not mentioned anywhere in the document. For example, the summary might say that "A fire broke out at 2 am", but the document doesn't mention the time when the fire broke out.
* _Informativeness_: Please select informative if the summary expresses the main points of the document. Summary should contain relevant and important information and few unimportant details. If you select the summary to be not consistent with the document, please only consider the consistent information when evaluating this category.

The order of the summary is randomly shuffled. Each task consists of three unique workers, where we take the mean as the scores for this document. The final score is the mean factuality score across all documents. The average time for each task is

\begin{table}
\begin{tabular}{c c c c c} \hline \hline Model & RL & tok err↓ & sent err↓ & FactCC \\ \hline R1 & 29.04 & 12.31 & 60.65 & 23.93 \\ R1+FC & 28.99 & 12.13 & 59.93 & **24.81** \\ R2 & 29.08 & 12.12 & 59.59 & 23.67 \\ R2+FC & 28.65 & 12.13 & **59.48** & 24.37 \\ RL & **29.23** & 12.17 & 60.08 & 23.06 \\ RL+FC & 28.62 & **12.10** & 59.63 & 24.58 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Full result of pre-trained models with different sentence selection criteria shown in Figure 3. We denote the criteria with FactCC with (+FC).

 

around 3 minutes and we pay 0.6 USD per task, hence an hourly rate of \(\geq\$12\) per hour.

We use boostrap test [5] to determine statistical significance between two summaries with \(p<0.05\).

### Sample Outputs

We include some sample outputs in Figure 8 and Figure 9.

 

Example 1 (XSum)

**Summary:** Tap water in 80,000 homes in Lancashire has been declared safe to drink, after the discovery of a parasite at a treatment works left residents boiling water for three weeks.

**Remove:** Tap water in homes has been declared safe to drink, after the discovery of a parasite at a treatment works left residents boiling water .

Example 2 (XSum)

**Summary:** Great Britain's Becky James won her second Olympic silver of Rio 2016 by finishing second in the women's sprint.

**Remove:** Won her second Olympic silver of Rio 2016 by finishing second in the women's sprint.

 Example 3 (WikiHow)

**Summary:** Use a "magic eraser.", Try a regular eraser. Make an alcohol solution. Apply a dry erase marker over the permanent marker. Try nail polish remover or acetone on plastic.Acetone and nail polish removers containing acetone can strip away most paints and dyes, and the dye in Sharpies is usually weak against it.

**Remove:** Use a "magic eraser.", Try a regular eraser. Make an alcohol solution. Apply a dry erase marker over the permanent marker. Try nail polish remover or acetone on plastic.Acetone and nail polish removers containing acetone can strip away most paints and dyes, and the dye is usually weak against it.

Example 4 (WikiHow)

**Summary:** Call 911 if you think you are in danger. Cut through your denial. Do not confront your stalker. Tell other people. Keep a record of stalking activities. Get a restraining order. Be prepared to defend yourself. Change your daily routine. Change your passwords. Use anti-virus software. Clean up your social network. Make your online materials private. Notice signs of control in your relationship. Make your break up clear. Cut off contact. Get back your keys.

**Remove:** Call if you think you are in danger. Cut through your denial. Do not confront your stalker. Tell other people. Keep a record of stalking activities. Get a restraining order. Be prepared to defend yourself. Change your routine. Change your passwords. Use anti-virus software. Clean up your social network. Make your online materials private. Notice signs of control in your relationship. Make your break up clear. Cut off contact. Get back your keys.

Example 5 (Gigaword)

**Summary:** xinhua summary of asia-pacific stocks news on tuesday feburary ##

**Remove:** summary of stocks news on tuesday ##

Example 6 (Gigaword)

**Summary:** cuba urges eu to drop its common position

**Remove:** cuba urges to drop its common position

Figure 7: Summaries changed using the corrector. We mark hallucinated entities in the summaries with red.

 



## Chapter 7 Conclusion

In this thesis, we have presented a new method for constructing the most common tasks in order to find the best way of using the word-level information. We also present an approach that is based on the word-level information and the word-level information are used as input to the word-level information. The word-level information can be extracted from the word-level information by using the word-level information. This work has been done with the word-level information, which is not available at the word level. In particular, we will use word-level information to extract words from word-level information.

The word-level information is then extracted from the word-level information. It is shown how it works well when using word-level information. We show that word-level information is extracted from word-level information. We also provide a more efficient way to extract words from word-level information.

Figure 8: Example summaries from XSum and Gigaword. Nonfactual components are marked with red.



* **Article**: Slide the seats all the way forward and then back to thoroughly vacuum the carpet underneath. Start from the top and work your way down. Dust or dirt that has accumulated up top could fall down; dust or dirt that has accumulated at the bottom will rarely fall up:, , Allow it to sit for a few minutes before blotting dry with a towel. If the stain doesn't come out, repeat. After your final cleaner application, wash the area with a damp sponge and do a final blotting. Make sure to try to get as much dampness from the fabric as possible. Any excessive dampness may promote molding and/or mildew, which does not fall in the definition of detailing a car. Replace with a piece you've cut from a hidden place, such as underneath the seat. Use a water-resistant adhesive to keep it down... Warning: Always ask the owner of the car for permission before doing this step. If you want, have a sample repair that you can show the owner of the car of what the process will look like. If it's done well, this sample will be reassuring. Apply a non-slip dressing so that the driver's feet don't slip and slide while they're trying to do important things like braking. , Use an interior dressing like Armor All to finish it off. If you're not going to use liquids afterward, your detailing brushes should be a super-absorbant material like microfiber cloth, which picks up dust and dirt efficiently. Lightly mist some spray-on vinyl dressing onto the vent grilles to make them look brand new. Cleaning the seats is essential for a good detail. But different seats require different methods. Note that after cleaning, you may have to vacuum out the seats or surrounding area again, as dirt will be loosened up by this process. Cloth interiors: Interiors with nylon or other cloth can be shampooed with a wet-vac extraction machine. Cloth should be adequately dried after extraction has taken place. Leather or vinyl interiors: Interiors with leather or can be cleaned with a leather or vinyl cleaner and then hit softly with a leather brush. Cleaner can be wiped away with a microfiber cloth afterward. If you've cleaned leather seats with product, now's the time to condition them so that the leather looks visually appealing and doesn't dry out or begin to crack. For stronger build-up removal, use 4-ought steel wool on windows. Use plastic cleaner if the gauge cover is made of plastic...
* **Bart-base**: Vacuum the carpet underneath the seats. Blot the stain with a damp sponge. Replace a piece of carpet. Apply a non-slip dressing to the back of the seat. Wipe the seat down with a soft cloth.
* **PEGASUS***: Vacuum the seats. Blot the area with a damp sponge. Remove any molding and/or mildew. Apply a non-slip dressing to the area. Use a spray-on vinyl dressing to clean the vent grilles. Clean the interior of the car with a leather or vinyl cleaner.
* **CLIFF**: Vacuum the seats and surrounding area. Blot the stain with a damp sponge. Remove a piece of tape from a hidden place. Cover the tape with a non-slip dressing. , Apply an interior dressing to the back of the seat. Apply a vinyl dressing on the vent grilles. Clean the interior of the car with a microfiber cloth. Wash the seats. Condition the leather seats. Use steel wool or plastic cleaner on windows. Brush the wheel rims. Apply tire dressing. Spray degreaser on everything. Apply protectant to the tinted windows. Apply tilting to the glass. Wash and dry the car. Rinse the car thoroughly.
* **DAE**: Vacuum the seats and surrounding area. Blot the area with a damp sponge. Remove a piece of wood from a hidden place. Dress the seats. Wear a non-slip dressing. , Wipe down the vent grilles. Cover the vents.
* **FactPEGASUS**: Vacuum the carpet under the seats. Wipe the area with a damp sponge. Apply a non-slip dressing to the seat. Wipe down the interior with a spray-on vinyl dressing. Use a leather brush to clean the interior of the car. Use steel wool on windows. Clean the interior

Figure 9: Example summaries from WikiHow. The article is truncated to fit the page. Nonfactual information are marked with red.