Automatic Evaluation of Attribution by Large Language Models (2023)

Abstract: A recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external references to generate and support its claims. However, evaluating the attribution, i.e., verifying whether the generated statement is fully supported by the cited reference, remains an open problem. Although human evaluation is common practice, it is costly and time-consuming. In this paper, we investigate automatic evaluation of attribution given by LLMs. We begin by defining different types of attribution errors, and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. The fine-tuning data is repurposed from related tasks such as question answering, fact-checking, natural language inference, and summarization. We manually curate a set of test examples covering 12 domains from a generative search engine, New Bing. Our results on this curated test set and simulated examples from existing benchmarks highlight both promising signals and challenges. We hope our problem formulation, testbeds, and findings will help lay the foundation for future studies on this important problem.

## introduction generative large language models (llms) @xcite @xcite , inter alia) often struggle with producing factually accurate statements, resulting in hallucinations @xcite . recent efforts aim to alleviate this issue by augmenting llms with external tools @xcite such as retrievers @xcite and search engines @xcite @xcite . incorporating external references for generation inherently implies that the generated statement is backed by these references. however, the validity of such attribution, i.e., whether the generated statement is fully supported by the cited reference, remains questionable. 2 according to @xcite , only 52% of the statements generated by state-of-the-art generative search engines such as new bing and perplexityai are fully supported by their respective cited references. 3 inaccurate attribution compromises the trustworthiness of llms, introducing significant safety risks and potential harm. for instance, in healthcare, an llm might attribute incorrect medical advice to a credible source, potentially leading users to make harmful health decisions. similarly, in finance, faulty investment advice attributed to a reliable source may cause substantial financial losses. to identify attribution errors, existing attributed llms @xcite rely heavily on human evaluation, which is both expensive and time-consuming. for instance, the average cost of annotating a single (query, answer, reference) example is about $1 in @xcite . in the actual use of attributed llms, it is the user who needs to be wary of the attribution and manually verify it, which puts a tremendous burden on their side. therefore, effective and reliable methods to automatically evaluate attribution and identify potential attribution errors are highly desired. towards this goal, we take the first step by introducing attrscore (figure 1 ), a framework designed for automatic evaluation of attribution and identification of specific types of attribution errors. we propose a new problem formulation that categorizes attribution into three types: 1) attributable: the reference fully supports the generated statement; 2) extrapolatory: the reference lacks sufficient information to support the generated state-as an attribution validator, your task is to verify whether a given a reference can support the answer to a provided question. a contradictory error means the answer contradicts the fact in the attribution, while an extrapolatory error means that there is not enough information in the attribution.. ## problem formulation the primary task in this paper is to evaluate attribution, which involves verifying whether a reference provides sufficient support for a generated answer to a user's query. our task setting prioritizes one reference per statement, a unit task that more complex scenarios can be decomposed to. we study such a setting as it forms the basis for dealing with multiple references or distinct segments @xcite . prior work, such as rashkin et al. (2021); @xcite ; @xcite , mainly focuses on binary verification, i.e., determining if a reference supports the generated answer or not. we propose advancing this task by introducing a more fine-grained categorization. specifically, we classify attributions into three distinct categories: foot_3 to illustrate, consider a contradictory example (figure 1 ). the query is "what was the unemployment rate in germany in 2020?", and the generated answer is "4.31%". however, the reference states that the rate was "3.81%", contradicting the generated answer. an extrapolatory instance, on the other hand, would be a query about the "gas price in california". while the reference is relevant, it does not contain specific information to verify the correctness of the generated answer. following these examples, we see the importance of granularity in error classification. a finegrained classification allows us to pinpoint the nature of the errors, be it contradiction or extrapolation. users can better understand the type of errors an llm might make, enabling them to use the model more safely. additionally, such an error identification system can guide future training processes of attributed llms, leading to specific mechanisms' development to correct such errors. our categorization also offers a departure from the existing approach @xcite , which emphasizes on degree of support ("full", "partial", or "none") rather than attribution error types. our approach highlights specific issues in attribution evaluation for more effective error management and system improvement. formally, the task of attribution evaluation involves a natural language query q, a generated answer a, and a reference x from an attributed llm. the goal is to develop a function, denoted as f , that inputs (q, a, x) and outputs a class label indicating whether "according to x, the answer a to the query q is attributable, extrapolatory or contradictory." 5 ## automatic evaluation of attribution following our problem definition, we introduce two approaches for automatic evaluation of attribution: prompting llms and fine-tuning lms on simulated and repurposed data from related tasks. ## conclusion in this paper, we investigate the important problem of automatically evaluating attribution given by llms. we begin by defining different types of attribution errors and then explore two approaches for automatic evaluation: prompting llms and fine-tuning smaller lms. we experiment with both simulated test examples and manually curated test examples from a real-life generative search engine. the results highlight both promising signals and remaining challenges for the automatic evaluation of attribution. we hope our work could lay the foundation for future studies on this important problem.