Reducing named entity hallucination risk to ensure faithful summary generation (2023)

Abstract: The faithfulness of abstractive text summarization at the named entities level is the focus of this study. We propose to add a new criterion to the summary selection method based on the “risk” of generating entities that do not belong to the source document. This method is based on the assumption that Out-Of-Document entities are more likely to be hallucinations. This assumption was verified by a manual annotation of the entities occurring in a set of generated summaries on the CNN/DM corpus. This study showed that only 29% of the entities outside the source document were inferrable by the annotators, leading to 71% of hallucinations among OOD entities. We test our selection method on the CNN/DM corpus and show that it significantly reduces the hallucination risk on named entities while maintaining competitive results with respect to automatic evaluation metrics like ROUGE.

## introduction abstractive text summarization methods aim at generating shorter versions of documents, possibly containing new words with respect to original documents. recent pre-trained language models @xcite @xcite allow to obtain fluent generated text. despite their remarkable performance, those models tend to generate information that cannot be inferred from the source document. according to a study by @xcite , 30% of the summaries generated by various systems have inconsistencies, qualified as "hallucination" by @xcite . current metrics used to assess automatic text summarization systems, such as rouge @xcite and bertscore @xcite , do not account for these issues. these hallucinations led to several studies on evaluating the faithfulness of generated summaries and generating more faithful texts @xcite . for example @xcite proposed a qa-based metric. they produced questions from the generated summary and check that the answer in the document matches the one in the summary. natural language inference (nli) have also been proposed to evaluate the factuality of a generated summary by checking if it entailed the source document @xcite @xcite . most recent studies @xcite used chatgpt foot_0 as nli system to evaluate the factual inconsistency of generated summary. @xcite suggested using contrast candidate generation and selection as a post-processing method to avoid hallucination. they create candidate summaries and replace named entities with ones found in the document to summarize. @xcite introduced an entitybased metric to compare the entities in the gold summary to the generated summary. they also proposed a method based on training data filtering and multi-task learning to avoid hallucinations. this paper studies hallucinations at the entity level, which is a crucial level @xcite in news-related corpora such as cnn/dm @xcite or xsum @xcite . we aim to reduce the "risk" of generating nonfactual summaries by reducing the number of possibly hallucinated entities using a new criterion called named entity hallucination risk (nehr). this criterion stands on the precision-source of nan et al., 2021 for summary selection. based on the assumption that an out-of-document (ood) entities have more chance to be hallucinations, we empirically verify that the entities in the summary are occurring in the source document. our contributions are as follows : ## named entity hallucination risk we propose to assess the risk of using entities and quantities incorrectly, leading to hallucinations in generated summaries. in the following, we call an entity a word string belonging to a named entity category such as person, location, organisation and quantities. an entity can be considered as an indocument entity if its corresponding word string can be found in the source document (exact-match in-document entities), or if it relates to an alternative form (e.g. new york/big apple). if an entity is not mentioned in the source document, it is considered as an out-of-document (ood) entity. ood entites can be either acceptable if they can be inferred from the document (e.g. new @xmath0. we call hallucinations the occurrence in generated summaries of entities that cannot be inferred from the source document. hallucinations can be obvious errors, entities that have nothing to do with the source document context, or entities that could be acceptable by looking at external sources but which were considered as dubious by human annotators: accepting them would require some form of world knowledge beyond general understanding, not directly available from the document. let's point out that an entity can be an in-document entity or an acceptable ood entity but still be incorrect in its context of occurrence in a generated summary. this study aims to reduce the risk of hallucinating entities thanks to a selection criterion called nehr for named entity hallucination risk inspired by the precision-source metrics proposed by @xcite . a simple selection system can be built from our nehr criterion, which selects less risky hypotheses from a large sample of summaries. ## empirical study on the nehr criterion dataset and model we used the test set of the cnn/dm @xcite , a popular abstractive dataset for the automatic text summary task that comes from news articles from the cnn and the daily mail websites; and bart @xcite , a transformer encoder-encoder based model that can be fine-tuned to perform automatic text summarization. bart's pre-training consists in applying an arbitrary noising function to corrupt the text deliberately, and training a model to reconstruct the original text from this corrupted version. we initialised the model with pretrained weights from the hugging face library @xcite resulting from the fine-tuning of bart-large on cnn/dm foot_1 . ## experiments on summary selection we evaluate the impact of using our nehr criterion for selecting a summary at inference time among a possible set of summaries and compare the results obtained in terms of rouge and human evaluation with three baseline selection methods: in section 3, we saw that 37.2% of the entities in the document were variations of other entities belonging to the source document. thus, using heuristics described in appendix c, we identified entities that were possible variations of in-document entities to ensure that the ood entities were not mistakenly considered to be alternative forms of in-document entities by our automatic system .