A ttribution B ench: How Hard is Automatic Attribution Evaluation? (2024)

Abstract: Modern generative search engines enhance the reliability of large language model (LLM) responses by providing cited evidence. However, evaluating the answer’s attribution, i.e., whether every claim within the generated responses is fully supported by its cited evidence, remains an open problem. This verification, traditionally dependent on costly human evaluation, underscores the urgent need for automatic attribution evaluation methods. To bridge the gap in the absence of standardized benchmarks for these methods, we present AttributionBench, a comprehensive benchmark compiled from various existing attribution datasets. Our extensive experiments on AttributionBench reveal the challenges of automatic attribution evaluation, even for state-of-the-art LLMs. Specifically, our findings show that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. A detailed analysis of more than 300 error cases indicates that a majority of failures stem from the model’s inability to process nuanced information, and the discrepancy between the information the model has access to and that human annotators do.

## introduction the advent of large language models (llms) has revolutionized the field of information retrieval and grounded text generation @xcite @xcite @xcite , leading to the development of advanced generative search engines like bing chat, google bard, and perplexity.ai. these platforms excel in generating search results in a natural language format, along with references to the source web pages as evidence to support the truthfulness of the generated response @xcite . however, whether the evi-gpt-3.5 (w/ cot): … the reference discusses the population of thailand and its connection to theravada buddhism, the war... however, it does not provide specific information or data to support the claim that the population of thailand is about 63 million people. dence supports the generated responses or not, also known as the attribution of the response to the evidence, remains an open problem. towards this end, there have been efforts made to build automatic attribution evaluation models. @xcite proposed the framework of attributable to identified sources (ais), i.e., evaluating whether model-generated responses can be verified against given references. under this framework, several approaches were proposed to build the automatic evaluator, including directly utilizing natural language inference (nli) models @xcite , directly prompting llms like gpt-3.5 and gpt-4 @xcite , and fine-tuning smaller lms like flan-t5 @xcite with repurposed data from related tasks like nli, fact-checking, and summarization @xcite . however, these studies define the attribution evaluation task differently (i.e., using different numbers and types of classification labels) and choose different datasets to train and evaluate models, making it impossible to directly compare their performance and derive significant insights into the challenges of attribution evaluation and how to address them in future work. to this end, we take the first step to present a systematic benchmark, attributionbench, for training and evaluating cutting-edge automatic attribution evaluators. specifically, we meticulously sample data from 7 different datasets @xcite @xcite @xcite @xcite that cover different domains of questions and diverse responses and evidence. we unify them into a binary classification format with a label-balanced setting for fair comparison foot_1 . we compile them into a training set and two test sets for in-distribution (id) and out-of-distribution (ood) evaluation, respectively. we conduct extensive experiments and analysis on our proposed benchmark. surprisingly, we find that even fine-tuned gpt-3.5 can only get around 80% macro-f1 score under both id and ood settings, which is far away from practical use. to better understand the challenges of this task, we carefully labeled over 300 error cases from gpt-3.5 under chain-of-thought (cot) prompting @xcite , which generate rationales for the model's prediction that can reveal reasons for an error. we find that 1) over 66% errors are caused by the model's insensitivity to fine-grained information (i.e., ignoring or overlooking details like facts, events, numbers, or lack of necessary summarization and inference from the evidence, tasks that humans naturally perform), and 2) about 26.8% of the errors are caused by the mismatch between information accessible to the model and that accessible to human annotators (see examples in figure 1 ). this is because the references given to the models are manually extracted from web pages by human annotators based on what they think is useful. however, when making the judgment, human annotators may be inherently affected by additional content since they've seen the whole webpage. on the other hand, sometimes the supported evidence spreads across the entire webpage, making it impossible to locate a short snippet to serve as the evidence. these findings provide valuable insights into 1) the challenges for automatic attribution evaluation, 2) the challenges for humans to evaluate attribution, and 3) how to develop stronger automatic attribution evaluation models. we summarize our contributions as follows: benchmark. we propose attributionbench, a comprehensive benchmark with a unified formulation for attribution evaluation, which will enable the community to compare different methods fairly and track the progress on this important task. ## attributionbench in this section, we introduce our task formulation and data collection pipeline for constructing attributionbench. they contain a wide range of diverse questions. "id" and "ood" denote "in-distribution" and "out-ofdistribution", respectively. we make all subsets labelbalanced so that the evaluation can fairly show the performance of each class. more details are listed in appendix ## error analysis after extensive baseline assessments and ablation studies, we sought to pinpoint the major challenges for the task of attribution evaluation. we conduct a detailed qualitative error analysis using gpt-3.5 (w/ cot) by randomly sampling 50 error cases (or all of them if fewer than 50) for each dataset, which results in 321 instances in total. for expertqa, stanford-gensearch, begin, and hagrid, we pick 25 cases for both false positive and false negative examples and label all the error cases for at-tributedqa, lfqa, and attreval-gensearch. we show representative examples in our first observation is that over 66% error cases are caused by fine-grained information insensitivity. this means when making a wrong judgment, the model is most likely to fail in comparing fine-grained information within the claim and the evidence. here, "fine-grained information" encompasses a wide array of specifics, from discrete data points such as numerical figures, dates, names, and locations, to more complex elements like particular events or logical connections. moreover, within this category of error, the model also makes different kinds of mistakes regarding its label error class, i.e., false positive or false negative. in the case of false positives, the model misses certain details within the claim 43% of the time , and in 41% of the instances, it made incorrect connections and inferences to indicate the claim being supported by the evidence, which is actually wrong. conversely, within the false negative category, the model predominantly misinterprets or neglects details in the evidence around 46% of the time, and 39% of them are because of lacking necessary inference and summarization to reach the attribution judgment, which could be very easy for humans. about 26.8% errors are caused by the mismatch between information accessible to the model and that accessible to human annotators. this disparity brings into question the reliability of labels within current attribution evaluation datasets. our analysis suggests that, in certain cases, determining the label based solely on the provided claim and evidence can be challenging due to insufficient information or ambiguous references (error type: need additional information/reference ambiguity). at times, this can even lead to the annotated label being wrong (error type: label mismatch). however, this doesn't mean that nearly a quarter of the labels are wrong. according to the current annotation process @xcite , human annotators usually assess claims sequentially, which allows for additional context to inform their judgment on subsequent claims after reviewing the preceding ones. furthermore, while the models receive manually extracted snippets of evidence from web pages, human annotators have the advantage of implicitly considering the entire web page when making their judgments. this discrepancy between the annotator's access and the model's to information can lead to inconsistencies between the annotation and the prediction process. consequently, even though many studies boast high concordance rates for their labels @xcite , the true reliability of these metrics and the correctness of the labels warrant further investigation. other error types. the model also struggles with dealing with numerical/symbolic operators, misunderstanding the task definition and logical relation implied by labels, and sometimes fails due to the lack of implicit commonsense knowledge compared to the previous two error types, these errors can be more easily fixed by augmenting lms with external tools, injecting necessary knowledge when prompting/inference, and making clearer instructions and in-context examples as guidance. our observations reveal the difficulty of the attribution evaluation task, leading to two potential directions for improvement: (1) for the model side, we should let the models not only be faithful, strongly conditioned on the given evidence, and pay attention to fine-grained information, but also possess the necessary ability of summarizing and reasoning over the evidence. claim: he was once an important member of the party but became a traitor . evidence: he seems to have been a party leader who fell out of favor with the regime . human-annotated ground truth: not attributable gpt-3.5 (w/ cot): the reference supports the claim as it indicates that the person was once an important member of the party (party leader) but then fell out of favor with the regime , which aligns with the claim of becoming a traitor . final judgment: attributable . need additional information/reference ambiguity (15.6%) claim: kellie pickler won season 16 of dancing with the stars. evidence: title: dancing with the stars (american season 16)\n\n pickler and her partner derek hough won the competition, marking his fourth of six wins and increasing his lead on the show as the professional dancer who has won the most mirror ball trophies. human-annotated ground truth: not attributable gpt-3.5 (w/ cot): the reference supports the claim. kellie pickler did indeed win season 16 of dancing with the stars, as mentioned in the reference. final judgment: attributable. label issue (11.2%) claim: ireland last qualified for the world cup in 2002, as highlighted in and . evidence: by the time of their next potential appearance, in 2026, ireland will not have been involved in the finals since they took la roja to penalties in a campaign lit up by the goals of the talismanic robbie keane 24 years earlier. human-annotated ground truth: attributable gpt-3.5 (w/ cot): the reference does not support the claim. the reference only mentions ireland's potential appearance in the 2026 world cup and their absence from the finals since 1994, but it does not specifically mention their last qualification in 2002. therefore, the reference does not support the claim. final judgment: not attributable . green background indicates the correct spans and labels, and red background indicates the wrong spans or labels. case 1: the model over inferred "fell out of favor with the regime" into "became a traitor". case 2: the "pickler" in the evidence is ambiguous, leading to the "not attributable" label by the human annotator while the model failed to capture it. case 3: it's impossible to get an "attributable" conclusion according to the extracted evidence, whereas the original web page may support it. and search engines @xcite . notably, some work explicitly equips generation with retrieved supporting citations or evidence as the attribution for the generated statement. for instance, gophercite @xcite puts the citation after the generation as inline evidence. webgpt @xcite finetunes @xcite to simulate a person's behavior of searching online and finding relevant information. dialogue systems like lamda @xcite and sparrow @xcite improve their safety and factual grounding by finetuning with human-annotated data and enabling the model to seek help from external resources. attribution evaluation. liu et al. (2023) and kamalloo et al. (2023) employ human evaluation methods in which annotators are tasked with assessing the variability of responses generated by either search engines or advanced language models equipped with additional retriever modules, adhering to the criteria outlined in (rashkin et al., 2021). both studies corroborate that these systems often fall short in delivering statements backed with credible support and accurate citations. to avoid the time-consuming human evaluation, gao et al. (2023b); bohnet et al. (2022) employ nli models to provide automated ais (autoais). yue et al. (2023) propose to repurpose related tasks to train a novel automatic evaluation model as well as prompting lms as attribution evaluators. our proposed benchmark systematically evaluates the performance of these approaches under various settings and reveals the challenge of automatic attribution evaluation via in-depth error analysis. ## conclusion in this paper, we propose attributionbench, a comprehensive benchmark for assessing automatic attribution evaluators. we conduct extensive experiments and analysis with cutting-edge llms like fine-tuned gpt-3.5, gpt-4, and a variety of open-source llms. results systematically show that automatically evaluating attribution is still very challenging. through meticulous error analysis, we identify the principal obstacles in this endeavor, notably (1) the models' lack of sensitivity to detailed, fine-grained information and (2) the discrepancy between the information accessible by the model and that by human annotators. we hope our research will shed light on these issues and guide future efforts toward the development of more accurate and reliable automatic attribution evaluators.