S umm E dits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization (2023)

Abstract: With the recent appearance of LLMs in practical settings, having methods that can effectively detect factual inconsistencies is crucial to reduce the propagation of misinformation and improve trust in model outputs. When testing on existing factual consistency benchmarks, we find that a few large language models (LLMs) perform competitively on classification benchmarks for factual inconsistency detection compared to traditional non-LLM methods. However, a closer analysis reveals issues with existing evaluation benchmarks, affecting evaluation precision. To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. This new benchmark is 20 times more cost-effective per sample than previous benchmarks and highly reproducible, as we estimate inter-annotator agreement at about 0.9. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance, highlighting the gaps in LLMs’ ability to reason about facts and detect inconsistencies when they occur.

## introduction with recent progress in generation capabilities of llms, automatic summarization is making its appearance in practical information consumption situations such as summarizing work meetings @xcite , health records @xcite , or scientific documents @xcite . to ensure the safe and effective implementation of these applications, it is essential to limit the reach of factually inconsistent summaries, a known issue with generated summaries @xcite . prior work @xcite @xcite has annotated corpora of model summaries with labels of factual con- figure 1 : summedits is a benchmark to evaluate the factual reasoning abilities of llms, measuring if models detect factual inconsistencies when they occur in summaries. capable detection models can help build more reliable nlg systems. recent investigations of using llms for evaluation have shown promising results across different nlp tasks @xcite , including factual consistency @xcite . in this work, we continue this line of research and explore applying llms as factuality evaluators in the context of text summarization. we first establish baseline performance for a suite of llms on two existing benchmarks -aggrefact @xcite and dialsummeval @xcite -with results confirming that some llms perform competitively with state-of-the-art specialized methods such as qafacteval @xcite . however, a manual analysis of conflict cases in aggrefact reveals a significant number of mislabeled samples (7+%) of factual inconsistencies undetected by annotators during dataset creation that the gpt4 explanations reveal. this lack of quality of benchmarks lim-its the precise evaluation of model performance at factual inconsistency detection. to address this issue, we introduce a protocol designed to create challenging benchmarks while ensuring the reproducibility of the labels. the protocol involves manually verifying the consistency of a small set of seed summaries and subsequently generating numerous edited versions of these summaries. we discover that assessing the consistency of edited summaries is relatively straightforward and easy to scale for human annotators, thus guaranteeing low cost and high agreement among annotators, yet keeping the task challenging for models. we create the summedits benchmark by implementing the protocol in ten diverse textual domains, including the legal, dialogue, academic, financial, and sales domains. figure 1 summarizes experimental results on the benchmark, which indicate that summedits presents a challenge for all models, with only four llms outperforming the specialized model qafacteval. our estimate of human performance of 90%+ largely outperforms models, suggesting current llms are not yet proficient at complex factual reasoning, and cannot assess the factual validity of summaries with precision. we believe summedits can serve as a tool to evaluate llms' abilities to detect factual inconsistencies when they (inevitably) occur and encourage llm developers to report their performance on the benchmark. for practitioners requiring specific domain expertise, the protocol can be adapted to generate low-cost, in-domain benchmarks that can check model capabilities prior to production use. we release the code and benchmark publicly foot_0 . ## related work annotating factuality of summaries. with advances in language models and the increase in fluency and abstractiveness of summarizers, prior work showed that one of the key challenges in summarization was enforcing factual consistency @xcite , particularly with models trained on datasets with unfaithful references @xcite . several efforts -such as factcc @xcite , summeval @xcite , polytope @xcite , frank @xcite , and cliff @xcite -annotated the generated summaries of tens of model, finding that most models produce a non-negligible portion of inconsistent summaries. although most annotation effort has focused on the summarization of news, some prior work also looked at dialogue summarization @xcite , or the medical domain @xcite . in most work, scalable high-quality annotation is challenging, due to low inter-annotator agreement when relying on crowd-workers, with some work showing that 10+ annotators are required to achieve some level of consensus @xcite , and some work recommending solely relying on experts @xcite . at the heart of the issue, annotating the factual consistency of a summary is challenging: it requires careful reading of long documents and the detection and interpretation of nuanced facts. in this work, we propose a new protocol to annotate factual consistency resources and show that it lowers the cost and increases reproducibility by minimizing the amount of reasoning required for each annotation. some work has also annotated inconsistency of pairs on text on non-summarization tasks, such as paraphrasing @xcite , documentgrounded dialogue @xcite , and wikipedia-editing @xcite . follow-up work has then aggregated and standardized annotations into benchmarks such as summac @xcite , aggrefact @xcite and true @xcite . ## limits of crowd-based benchmarks we first analyze model performance on two popular benchmarks for factual consistency detection in summarization: aggrefact @xcite and dialsummeval @xcite and uncover limitations that guide the design principles of the summedits benchmark. ## conclusion in this work, we explore the capabilities of llms to act as factual reasoners through the lens of factual evaluation in text summarization. as part of this analysis, we uncover and discuss shortcomings of existing benchmarks. using those insights we develop a new protocol for creating inconsistency detection benchmarks, which we implement in a 10-domain benchmark called summedits. the summedits benchmark is highly reproducible and more cost-effective per sample than previous benchmarks. our experiments show that the benchmark is challenging for most current llms, with the best-performing model, gpt-4, still 8% below estimated human performance. we believe that summedits can serve as a valuable tool for evaluating llms' abilities to reason about facts, detect factual errors and promote more reliable nlg systems. we encourage llm developers to report their performance on the benchmark.