Evaluating the Factual Consistency of Large Language Models Through News Summarization (2023)

Abstract: While large language models (LLMs) have proven to be effective on a large variety of tasks, they are also known to hallucinate information. To measure whether an LLM prefers factually consistent continuations of its input, we propose a new benchmark called FIB (Factual Inconsistency Benchmark) that focuses on the task of summarization. Specifically, our benchmark involves comparing the scores an LLM assigns to a factually consistent versus a factually inconsistent summary for an input news article. For factually consistent summaries, we use human-written reference summaries that we manually verify as factually consistent. To generate summaries that are factually inconsistent, we generate summaries from a suite of summarization models that we have manually annotated as factually inconsistent. A model’s factual consistency is then measured according to its accuracy, i.e. the proportion of documents where it assigns a higher score to the factually consistent summary. To validate the usefulness of {pasted macro ‘BENCHMARK’}, we evaluate 23 large language models ranging from 1B to 176B parameters from six different model families including BLOOM and OPT. We find that existing LLMs generally assign a higher score to factually consistent summaries than to factually inconsistent summaries. However, if the factually inconsistent summaries occur verbatim in the document, then LLMs assign a higher score to these factually inconsistent summaries than factually consistent summaries. We validate design choices in our benchmark including the scoring method and source of distractor summaries.

## introduction factual inconsistency is a widespread problem in natural language generation tasks (maynez et al., 2020; weng et al., 2020; @xcite . for text summarization in particular, it has been shown that models often hallucinate new information or 1 we include our code in the supplementary ## fib: factual inconsistency benchmark each example in fib consists of a document and two summaries: a factually consistent summary and a factually inconsistent summary. models are evaluated based on the proportion of times they assign a higher score to a factually consistent summary than to a factually inconsistent summary. we define a factually consistent summary as a summary whose contents can be inferred solely from the document. this means that even if a summary contains true information, if the information is not found in the document, then the summary is factually inconsistent. for example, the gold summary in fig. 1 is factually consistent as it is written, but if we swapped peveril point with a cliff, then it would no longer be factually consistent, even if peveril point is technically a cliff, since this fact cannot be inferred from the document. we compare the factual consistency of models on both extractive and abstractive summaries. extractive summaries occur verbatim in the document while abstractive summaries do not. we use two summarization datasets as our testbed: cnn/dm (see et al., 2017; hermann et al., 2015) for extractive summaries and xsum (narayan et al., 2018a) for abstractive summaries. cnn/dm consists of english documents about the news from cnn/daily mail and summaries that are several sentences long with 287k/13k/11k examples for train/val/test. foot_0 xsum consists of english documents about the news from bbc and short summaries with 204k/11k/11k examples for train/val/test. foot_1 the cnn/dm dataset is distributed under an apache 2.0 license and xsum is under a creative commons attribution 4.0 international license. our use is consistent with the intended use and we release our code under an apache 2.0 license and the data for fib under a creative commons attribution 4.0 international license. ## experiments having defined our benchmark, we now evaluate the factual consistency of various llms and compare with several other methods for generating alternative summaries and assigning scores to lm generations. ## analysis to get a better sense of what kind of factually inconsistent model-generated summaries tend to fool models into assigning a higher score than the gold summary, we show some examples for bloom in ## conclusion and takeaways we present fib, a new benchmark for evaluating the factual consistency of language models, and evaluate 23 large language models on fib. our takeaways are: (1) llms tend to assign higher scores to factually consistent summaries than to factually inconsistent summaries, except that llms almost always assign higher scores to extracted summaries even if they are factually inconsistent and (2) length-normalized pmi enables models to most effectively detect factually inconsistent summaries. our results open new avenues for future work, including a more fine-grained study on the type of factually inconsistent errors different llms make and investigating the effect training on summarization has on the factual consistency of llms.