Automatic Evaluation of Attribution by Large Language Models
Authors: Xiang Yue
Boshi Wang
Ziru Chen
Kai Zhang
Yu Su
Huan Sun
Venue: EMNLP 2023 Findings (2023)
Keywords: Large Language Models; Attribution Evaluation; Attribution of LLMs; Evaluation of LLMs;
TL;DR: We investigate the automatic evaluation of attribution by LLMs

Abstract: A recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external references to generate and support its claims. However, evaluating the attribution, i.e., verifying whether the generated statement is fully supported by the cited reference, remains an open problem. Although human evaluation is common practice, it is costly and time-consuming.  In this paper, we investigate automatic evaluation of attribution given by LLMs. We begin by defining different types of attribution errors, and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. The fine-tuning data is repurposed from related tasks such as question answering, fact-checking, natural language inference, and summarization. We manually curate a set of test examples covering 12 domains from a generative search engine, New Bing. Our results on this curated test set and simulated examples from existing benchmarks highlight both promising signals and challenges. We hope our problem formulation, testbeds, and findings will help lay the foundation for future studies on this important problem.

Automatic Evaluation of Attribution by Large Language Models
Xiang Yue Boshi Wang
Ziru Chen
Kai Zhang
Yu Su
Huan Sun
The Ohio State University
Cyue. 149, wang, chen, 8336, zhang. 13253, se9,su........... sun. 397 387 edu
Abstract
backed by these references. However, the valid- ity of such attribution, i.e., whether the generated statement is fully supported by the cited reference. remains questionable." According to Liu et al. (2023), only 52% of the statements generated by state-of-the-art generative search engines such New Bing and PerplexityAl Al are fully supported by their respective cited references."
recent focus of large language model (LLM) development, as exemplified by generative search engines, is to incorporate external refer- ences to generate and support its claims. How- ever, evaluating the attribution, i.e., verifying whether the generated statement fully sup- ported by the cited reference, remains open problem. Although buman evaluation is com- mon practice, is costly and time-consuming In this paper, investigate automatic evalu- ation of attribution given by LLMs. We be- gin by defining different types of attribution errors, and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. The fine-tuning data is repurposed from related tasks such as ques- tion answering, fact-checking, natural language inference, and summarization. We manually cu- rate a set of test examples covering 12 domains from a generative search engine, New Bing. Our results on this curated test set and simu- lated examples from existing benchmarks high- light both promising signals and challenges. We hope our problem formulation, testbeds, and findings will help lay the foundation for future studies this important problem.!
Inaccurate attribution compromises the trustwor- thiness of LLMs, introducing significant safety risks and potential harm. For instance, in health- care, an LLM might attribute incorrect medical ad- vice to a credible source, potentially leading users to make harmful health decisions. Similarly, in finance, faulty investment advice attributed re- liable source may cause substantial financial losses.
To identify attribution errors, existing attributed LLMs (Nakano et al., 2021; Thoppilan et al., 2022) rely heavily on human evaluation, which is both expensive and consumin.. For instance, the average cost of annotating a single (query, answer, reference) example is about $1 in Liu et al. (2023). In the actual use of attributed LLMs, it is the user who needs to be wary of the attribution and manu- ally verify it, which puts a tremendous burden on their side. Therefore, effective and reliable meth- ods automatically evaluate attribution and iden- tiyy potential attribution errors are highly desired.
1 Introduction
Generative large language models (LLMs) (Brown et al., 2020; Ouyang et al., 2022; Chowdhery et al., 2022: OpenAL, 2023a.b, inter alia) often struggle with producing factually accurate statements, re- sulting in hallucinations (Ji al., 2023). Recent efforts aim alleviate this issue by augmenting LLMs with external tools (Schick al., 2023) such as retrievers (Shuster et al., 2021; Borgeaud et al., 2022) and search engines (Nakano et al., 2021; Thoppilan et al., 2022; Shuster et al., 2022). Incorporating external references for generation inherently implies that the generated statement is
Towards this goal, we take the first step by intro- ducing AurScore (Figure 1), framework designed for automatic evaluation of attribution and identi- fication of specific types of attribution errors. We propose new problem formulation that catego- rizes attribution into three types: 1) attributable: the reference fully supports the generated state- ment; 2) extrapolatory: the reference lacks suffi- cient information to support the generated state-
Attribution primarily refers to "the act attributing some- thing" this paper, which is similar to "verifiability" as fined in Liu et al. (2023). 3 www.bing. com/new, bing.com//ew.
'Our code and dataset are available at: https://gites com/OSU-NLL htw/OOSUNNLP-Gronne WLLP-Grop/AAtrSSce
www. www.perplexity.att

---

Q: What is the temperature range on the moon?
Q: What is the highest price of a gallon regular gas in the US in 2022?
Q: What was the unemployment
rate in Germany in 2020?
A: The average temperature on moon can range from -298 degrees F (-183 degrees C) at night to 224 degrees (106 degrees C) during the day [1].
article by NPR, of June 2022, the highest average price of a gallon of regular gas in the US was $6.34 in Callfornia [1].
According to
A: According to [1], the unemployment
rate Germany for 2020 4.3 4.31%, which was a 1.17% increase from 2019. It's important to note that the COVID...
References
References
References
cpolcosmos wpw. edu
[1] mpc.arvsanss-hhhhsstsee----
macrotrends netuneemleeeeeettrttee
The temperature the Moon
In 10 states, a gallon of gas now costs more than $5
Germany Unemployment Rate
The average temperature on the Moon (at the equator and mid lattudes) varies from -298 degrees Fahrenheit (-183 degrees Celsius), al night, to 224 degrees Fahrenheil (108 degrees Celsius) during the day. Because the Moon has
1991 12023
.....S. gas prices hit a new record on gallon regular gas costing an average of $4.865 (not edjusting for infiation), according to the most recent data from American Automobile Association...
Unemployment refers to the share of the labor force that is without work but available for and seeking. Germany unemployment rate for 2020 was 3.81%) 0.67% increase 2019.
Monday,
(1) Prompt LLMs with clear evaluation instruction
(2) Fine-tune LMs on a set diverse repurposed datasets
As an Attribution Validator, your laek is to verify whether given a reference can support the snswer provided question. contradictory error meens the enewer contredicts the fact in the attribution, while an extrapolatory error means that there not enough information attribution.. +f
QA
Fact-Chacking
AttrScore
NLI
Summarizztion
Attributable
Extrapolatory
Contradictory
We make the first step towards automatically evaluating attribution and identifying specific types of errors with AttrScore. We explore two approaches in AttrScore: (1) prompting LLMs, and (2) fine-tuning LMs on simulated and repurposed datasets from related tasks.
Figure 1;
ment, and 3) contradictory: the generated state- ment directly contradicts the cited reference. Un- like existing work (Bohnet et al., 2022) that uses binary categorization (i.e., attributable or not) and Liu et al. (2023) that defines the degree of refer- ence support for the generated statement "full", "partial", or "no support". our fine-gratd fine-grained error categorization aids humans in better understanding the type of an attribution error made by an LLM. This not only enhances safe system usage but also provides valuable insights for future development of mechanisms tailored to correct specific errors.
hence avoiding the data contamination issue. Our results indicate that both approaches show reasonable performance our curated and simu- lated test sets; yet there is still substantial room for further improvement. Major sources of evaluation failures include insensitivity to fine- -raiirai infor- mation comparisons, such as overlooking contex- tual cues in the reference, disregard for numerical values, and failure in performing symbolic opera- tions. In light of these findings, we discuss poten- tial directions for improving AttrScore, including training models to be more strongly conditioned on the reference, and augmenting them with external tools for numerical and logical operations.
We explore two approaches in AttrScore: 1) prompting LLMs and 2) fine-tuning LMs on simu- lated and repurposed data from related tasks such as question answering (QA), fact-checking, natural language inference (NLI), and summarization. For evaluation, unlike existing work (Liu et al., 2023; Gao et al., 2023) that only uses queries from exist- ing benchmarks, covering 12 different domains from generative search engine, New Bing. This the first eval- uation set for measuring the attribution of LLMs with queries created based on real-life interactions,
With the new formulation of attribution errors, the development of AttrScore, the introduction of new test sets, and the insights into challenges and potential directions for future work, we hope work can help lay the foundation for the important task of automatically evaluating LLM attributions.
curate a set of test examples
Problem Formulation
The primary task in this paper is to evaluate attribu- tion, which involves verifying whether a reference

---

provides sufficient support for generated answer user's query. Our task setting prioritizes one reference per statement, a unit task that more com- plex scenarios can be decomposed to. We study such a setting as it forms the basis for dealing with multiple references or distinct segments (Liu et al., 2023; Gao et al., 2023).
evaluation for more effective error management and system improvement.
Formally, the task of attribution evaluation in- volves a natural language query q. a generated an- swer a, and a reference x from an attributed L.M. The goal is to develop function, denoted as f. that inputs (q, a, 2) and outputs a class label indicating whether "according to æ, the answer to the query q is attributable, extrapolatory contradictory.
Prior work, such as Rashkin et al. (2021); Gao al. (2022); Bohnet et al. (2022), mainly focuses on binary verification, i.e., determining if a refer- ence supports the generated answer or not. We propose advancing this task by introducing a more fine pneegrained categorization. Specifically, we clas- sify attributions into three distinct categories:*
Automatic Evaluation of Attribution
Following our problem definition, we introduce two approaches for automatic evaluation of attri- bution: prompting LLMs and fine-tuning LMs on simulated and repurposed data from related tasks.
Attributable: The reference fully supports the generated answer.
3.1 Prompting LLMs
The reference lacks sufficient information validate the generated answer.
Extrapolatory:
Recent research (Fu et al., 2023) has demonstrated the possibility of prompting LLMs to evaluate the quality of generated text using their emergent ca- pabilities (Wei et al.. 2022b), such as zero-shot in- struction (Wei et al., 2022a) and in-context learning (Brown et al., 2020). Following this approach, we prompt LLMs, such as ChatGPT (OpenAI, 2023a), using clear instruction that includes definitions of the two types of errors (as shown in Figure 1) and an input triple of the query, answer, and reference for evaluation. The complete prompt used in our study can be found in Appendix Table 6.
Contradictory: The generated answer contra- diets the information presented in the reference.
To illustrate, consider contradictory example (Figure 1). The query is "What was the unemploy- ment rate in Germany in 2020?", and the generated answer "4.31%" However, the reference states that the rate was "3.81% % contradicting the gen- crated answer. An extrapolatory instance, on the other hand, would be query about the "gas price in California". While the reference is relevant, it does not contain specific information to verify the correctness of the generated answer.
3.2 Fine-tuning LMS on Repurposed Data
The primary challenge in fine-tuning LMs for auto- matic attribution evaluation is the lack of training data. One potential approach is to hire annotators to collect real samples, but the cost can be prohibitive. Here, we first repurpose datasets from three re- lated tasks (fact-checking, NLI, and summariza- tion). We then propose to further simulate more realistic samples from existing QA benchmarks. Repurpose data from fact-checking, NLI, and summarization tasks. Given the connections be- tween our attribution evaluation task and the tasks of fact-checking, NLI, and summarization, pose to utilize datasets from these fields enrich our training examples. Fact-checking data and NLT data, with their emphasis on assessing the consis- tency and logical relationship between claims (hy- pothesis) and evidence (premise), mirrors our task's 's
Following these examples, we see the impor- tance of granularity error classification. A fine- grained classification allows us to pinpoint the na- ture of the errors, be it contradiction or extrapo- lation. Users can better understand the type of errors an LLM might make, enabling them to use the model more safely. Additionally, such an er- ror identification system can guide future training processes of attributed LLMs, leading to specific mechanisms' development correct such errors. Our categorization also offers a departure from the existing approach (Liu et al., 2023), which em- phasizes on degree of support ("full", "partial", or "none") rather than attribution error types. Our approach highlights specific issues in attribution
pro-
"We ucknowledge that while these categories are generally mutually exclusive, complex scenarios might blur the bound- aries between them. However, such cases are very rare. For the purpose of this study, we maintain their exclusivity to enable clear and focused error analysis.
'It is important to note that this evaluation focuses on the "verifiability" of the answer based on the reference. It does not measure the "relevance". i.c., whether the answer correctly responds to the query (Liu et al., 2023).

---

Query: Which apostic had thom side?
Query: Which aposile had a thom in his side?
Query: Which apostle had a thom side?
Query: Which aposile had thorn in his side?
Short Ans: Paul
Short Ans: Philip [1]
Short Ans: Paul (1)
Short Ans: Paul [1]
Long Ans: Paul was an aposile who had thom in his side [1].
Long Ans: Phillip thom in his side
Long Ans: Paul was an aposilc who had a thom in his side (1].
Long Ans: aposstic had thorn his side is Paul
References
References
References
References
[1] en.wikiordi org/wili! Thom in the fles
en.wikipetin.no orrwwiki Thom in the flesh
[1] www.wikiiddiiccomm angiviki Thern in the fleeh
wiki/Thorn fletter)
Thom in flesh
Thom the flesh
Thorn in the flesh
Thom (letter)
Thom in the flesh le phrase of New Testament origin used to describe a chronic infirily. annoyance, or trouble one's drawn from Paul Apoetie's use of the phrase in his Second Epistle Corinthanss 12: -9
Thom in the flesh is phrase of New Tastament origin describe a chronic intirmity annoyance, trouble in one's from Paul the Aposile's use of the phrase in his Second Epette the Corinthlans 12
Thom in the flesh a phrese of New Testament origin used to describe a chronic infirmity annoyance. trouble one's life.
Thom or born (P, b) letter the Old Englis, Old Norse, Old Swedish and modern Icelandic alphabets. as well madern transiierations the Gothic alphabet Middle Scote, and dislects Middle English. wes also used
John Apostla's use of the phrase in his Second Epistle to the Cornnhiens 12: 7 9
(A): Attributable
(B): Contradictory
(C): Contradictory
(D): Extrapolatory
Figure 2: Examples simulated from open-domain QA. We I) use the original (question, answer, context) pair as attributable instance (A), 2) substitute the answer or the answer span in the context to simulate a contradictory error example (B, C), and 3) replace the context with alternatives simulate extrapolatory emror example (D). In order for models trained the simulated data generalize well to the long answer setting in real-life search engines like New Bing, we convert the short answer to a long one (using ChaGGPT).
objective of checking the supporting relationship between reference documents and generated state- ments. Summarization datasets, especially those involving the detection of hallucinations (including both intrinsic and extrinsic (Maynez et al., 2020), could provide a useful starting point for identify- ing attribution inconsistencies. Nevertheless, these datasets would require suitable adaptation. We keep their original data sequences and modify their data label space to suit the specific needs of the attribution evaluation definition. Additional infor- mation on this can be found in Appendix A.
tion, retrieving relevant external documents from resources such as Wikipedia, which do not contain the ground truth answers (Figure 2, D). More de- tails regarding the simulation of these errors from QA datasets can found in Appendix A.
Experimental Setup
4.1 Datasets
This section presents the datasets utilized for train- ing and testing methods for automatic attribu- tion evaluation. In particular, develop two evaluation sets, AtrrEval-Simulation and AttrEval- GenSearch, derived from existing QA datasets and a generative search engine, respectively. The dataset statistics are presented in Table 1.
Simulate data from open-domain QA. QA bench- marks provide an ideal platform for data simula- tion, as they comprise questions, their correspond- ing ground truth answers, and reference contexts. These elements can be directly employed at- tributable examples (Figure 2, A). In open-domain QA datasets, answers are typically brief text spans. To cater to the long answer setting in most at- tributed LLMs, we convert these short answers into longer sentences using ChatGPT. For simulat- ing contradictory errors, we propose two methods: (1) The first involves modifying the correct answer with an alternative candidate from an off-the-shed QA model, an answer substitution model, a ran- dom span generator (Figure 2, B). (2) The second retains the original answer but replaces the answer span in the reference context with a comparable candidate (Figure 2, C). To emulate extrapolatory errors, we employ a BM25 retriever on the ques-
Training data. To repurpose and simulate train- ing examples, we follow the method in Section 3.2 based on four similar tasks' datasets. For QA, we consider NaturalQuestions (Kwiatkowski et al., 2019). For fact-checking, we include FEVER (Thorn et al., 2018), Adversarial FEVER (Thome et al., 2019), FEVEROUS (Aly et al., 2021), VI- TAMINC (Schuster et al., 2021), MultiFC (Augen- stein et al., 2019), PubHealth (Kotonya and Toni, 2020), and SciFact (Wadden et al., 2020). For NLI, include SNLI (Bowman al., 2015), MultiNLI (Williams et al., 2018), ANLI (Nie al., 2020) and SeiTail (Khot et al., 2018). For summarization, we include -Suu-Halluu. (Maynez et al., 2020). XENT (Cao et al., 2022), and FactCC (Kryscinski

---

Split
Related Tasks
Data Sources
#Smmples
Daily Life
QA
NaturaQuestioon
20K
NINLP Research 24.3%
FEVER, VITAMINC,
5.0%
Med & Health
Fact-checking
Adversarial FEVER,
20K
6.7%
FEVEROUS, SciFact
History
Business
Train
PubHealth, MultiFC
B 8.2%
NLJ
SNLI, MuliNLI
20K
Law
Gaagraphy
2.7%
ANLI SeiTail
Spiecce
Math
1.2%
Summarization
XSum- Numm-Hlllucinations.
3.8K
Game&Anime
XENT, FacCCC
18.4%
PopQA, Eatity Questions,,
QA
HotporQA, TrriviQQ,, WebQuestions, TREC
Figure 3: Domain distribution of our annotated AttrEval-
GenSearch test set (covering 12 domains in total).
Test
Annotated samples from generative search engine
242
keywords from specific domain randomly generated using ChatGPT, and relevant facts within that domain are compiled from the Web.'
Table I: Statistics of the training and test datasets for attribution evaluation. We include the distributions of the labels and data sources in Appendix B.
In the verification process, queries are sent to the New Bing search engine under a balanced mode fol- lowing Liu et al. (2023), which balances accuracy and creativity. The validity of the output generated by New Bing is evaluated, where consider only the first sentence that answers the question along with its reference. As we state in Section 2, our evaluation emphasizes the error type a single reference per statement. In the case of a sentence having multiple references or distinct segments (for example, "XXX [1][2]" or "XXX [1] and YYY [2]"), each reference or segment is treated as verified
et al., 2020). We use all examples in the summariza- tion task datasets, and sample 20K examples from QA, fact-checking, and NLI task datasets. We com- bine all the simulated datasets to create the training set for our main experiment.
Eiimllaaiaaaoo For testing, we first simu- late examples from six out-of-domain QA datasets: HotpotQA (Yang et al., 2018), EntityQuestions (Sciavolino et al., 2021), PopQA (Mallen et al.. 2022), TREC (Baudis and Sedivý, 2015), Trivi- aQA (Joshi al., 2017), and WebQuestions (Be- rant al., 2013). Note that we intend use dif- ferent QA datasets for training and testing, as to test the model's generalization ability, and evalu- ate its performance across a diverse set of domains and question formats. Our manual examination indicates that 84% of 50 randomly sampled exam- ples accurately align with their category, and the labeling errors are primarily due to incorrect notations in the original QA datasets or heuristics used formulate comparable answer candidates for contradictory errors and to retrieve negative passages for extrapolatory errors.
separate sample, and the attributions individually. Finally, the samples categorized by the annotators as attributable, contradictory, extrapolatory. Detailed annotation guidelines can be found in Appendix D.
4.2 Implementation Details
In the configuration of "prompting LLMs", we test Alpaca (Taori et al., 2023), Vicuna (Chiang et al., 2023), ChatGPT (OpenAI, 2023a) and GPT- 4 (OpenAl, 2023b), where we use OpenAl's offi- cial APIs 3.0t-3.5-turbae weights from Alpaca and Vicuna from the official repository?. For Alpaca and Vieuna inference, doc- uments are tokenized and truncated a maximum of 2048 tokens. We generate text with a temper- ature of O. The prompts for the task of evaluat- ing attribution are provided in Appendix Table 6,
gpt-4-0314) and
Atnreaal-Geneaarch... To examine the real-life ap- plication of automatic attribution evaluation, ap- proximately 250 examples from the New Bing search engine are annotated carefully by the au- thors. This process comprises two subtasks: creat- ing queries and verifying attributions. To avoid the issue of training data contamination, new queries are manually created across 12 domains (Figure 3).6 To facilitate and motivate query annotation,
"We make effort collect facts post-2021 1 to test ahout "knowledge confliction" (Zhou et al., 2023; Xie et al., 2023) between parametric and external knowledge.
* "platform. www.foomoo........ www.cooo//ooooo www.///wo/cceeccccccc docs/appspppp http///caaaa-eeeeeeeeeeeeeeeeeee rww.eeece/cceeaeeeeeeeeeeeeeee Given GPT-4's high cost slow inference speed, we evaluate it random samples from Aturval-Simmlltionn
"The "AINNPP Research" domain is inspired by recent discussions on social media about testing L.L knowledge on rescarchers, egg., "Is XX a co-author of the paper XX2"
"ht //gi thub. https:// /tt/t/gittttt com/lm systtast

---

AttrEval-Simulation
AttrEval-
Setting
Model (Size)
Attri.
Contra.
Extra.
Overall
Attr.
Contra.
Extra.
Overall
Alpaca (7B)
50.0
4.0
1.4
33.6
50.7
8.6
3.6
34.3
Alpaca (13B)
48.3
5.6
2.2
33.5
50.6
6.1
19.3
34.7
Zero-shot
Vicuna (13B)
46.3
8.3
21.6
34.6
54.4
13.3
26.1
41.4
ChatGPT
45.7
17.9
52.7
43.2
61.2
20.6
53.3
$5.0
GPT-
58.7
23.2
61.5
55.6
87.3
45.0
89.6
85.1
Alpaca (7B)
45.4
8.2
9.6
31.9
49.6
5.2
13.5
37.2
Alpaca (13B)
38.9
20.1
2.2
33.1
50.5
10.3
5,6
34.8
Few-shot
Vicuna (13B)
35.4
37.2
0.3
32.6
50.6
9.1
8.4
34.1
ChatGPT
46.6
27.6
35.8
39,2
62.6
26.8
49.5
53.3
GPT-4
61.1
31,3
68.8
60.0
85.2
53.3
88.9
84.3
Roberta (330M)
62.5
54.6
74.7
65.0
47.2
25.2
62.3
49.8
GPT2 (1.58)
63.6
54.6
71.9
63.5
51.1
18.6
60.7
47.4
T5 (770M)
45.9
57.1
71.6
59.1
58.5
24.3
72.5
61.6
Flan-T5 (770M)
57.3
50.1
70.5
59.3
64.3
27.6
72.9
64.5
Flan-T5 (3B)
48.1
48.7
67.1
55.7
77.7
44,4
80.0
75.2
Flan-T5 (1IB)
48.4
49.9
66.5
55.4
81.6
38.9
76.9
72.7
Fine-tuned
I. A (7B)
62.2
50.7
74.6
62.8
77.9
41.1
78.3
72.5
Alpaca (7B)
66.8
41.1
76.8
64.5
73.0
30.2
80.0
72.5
Alpaca (1 (13B)
63.6
48.9
75.8
63.6
77.5
34.5
79.4
73.3
Vicuna (13B)
66.2
49.1
78.6
66.0
69.4
37.7
79.9
72.1
Table 2: The performance (F1 score) AttrScore with different models on AttrEval- Siiuulitiiiinn and AttrEval- GenSearch sets. The best-performing result in each setting is in bold. The results show both promising signals and challenges (e.g., all models struggle with contradictory errors) in automatic evaluation of attribution.
and main results averaged over 4 different prompts. For the few-shot prompting setting, manually write examples as demonstrations for both test sets as shown in Table 7. LLMs yield an attribution label with an explanation, we extract the predicted label with regular expression.
5
Results
5.1 Overall Performance
Table presents evaluation of different models on both the simulated dataset Mitrrvval-Simllaton and the annotated dataset New Bing (AttrEval- GenSearch). Our primary findings are as follows: GPT-4 achieves promising results, reaching an overall accuracy of 81 11-83% on AttrEval-GenSearch and significantly outperforming other models. This suggests promising potential for employing GPT- 4 for automatic attribution evaluation alleviate human annotation workloads, aligning with the emerging trend that uses GPT-4 for different evalu- ation tasks (Chiang et al., 2023; Zheng et al., 2023). However, it may still not be sufficiently accurate for practical use. We also note some potential concerns of bias (see Limitations Section 8).
In the "fine-tuning LMs" setting, fine-tune four types of LMs of various scales: Roberta (340M) (Liu al., 2019), (FLAN-JTS (770M, 3B, 11B) (Raffel et al., 2020; Chung et al., 2022), GPT2
(1,5B) (Radford et al., 2019), LLaMA (7B), Alpaca (7B, 11B) (Taori et al., 2023), and Vicuna (7B, 1 IIB) (Chiang et al., 2023). Our implementation utilizes the Huggingface library (Wolf et al., 2019) and Alpaca examples. The training is performed on A 100 80GB GPUs with maximum of 512 tokens. For the LLaMA family of models, we use batch size of 32 and train for epoch. For the other mod- els, we use a batch size
Automatic attribution evaluation presents sub- stantial challenges. This complex task requires not only understanding the reference information but also comparing it with the information in the statement, all of which can significantly vary across different datasets and test conditions. Against these challenges, models other than GPT-4 exhibit sub- optimal performance in zero-shot and few-shot set- tings. Fine-tuning LMs on the simulated datasets from related tasks significantly improves the per-
and 3 epochs. We set the learning rate as 2e-5 and use cosine learning rate decay with 0.03 warm-up steps.
Metrics. For evaluation, we present the score for each individual class as well as the micro-FI score, which equivalent to the overall accuracy.

---

Error
Type
Example
(percentage)
Fine-grained in-
Query: ketchup medicine before?
lommation imsen-
Auswers: In the eatly 1800s. il was believed that ketchup could be used treat indigestion, jaundice, and diarrhea
sitivity
Reference: According to John Cock Beunet, tomaloes could be used to treal diarchea, ... Soon, Bennetl was publishing tomato ketchup rocipes in pill form ... Accurding to Dr. Beanett, this tomato blend can help people with diarthea, indigestion, theumatism, and jaundice. Ground Truth: Extrapolatory
(30.6%)
o GPT44; Atributable. The reference supports the claim that in the early 1800s. ketchup was helieved to he used as a medicine to treat indigestion, joundice, and diarcea.. li mentions John Cook Bennett's 's beliel the medicinal properties of tomatocs and his publication of tomato ketchup recipes in pill form, which were sold patent medicines in supermarkets and pharmacies.
Misunddrslanding
Query: When GTA6 will be publishued?
the task delini-
Auswer: Rockstur Ganes has not officially unmounced the release date for GTA 6 yet.
tion logical
Reference: most likely GTA release date is holiday 2024, and although is no official confimaation, this estimate confirmed Micmsoft's stating that "is expected" 2024 leegal filing and assorted insider loaks.
relation implied by labels (22.2%)
o Ground Truth: Atrriutablee
b GPT-4: Extrapolstory. The reference provides an estimated release date for GTA based en a legal Giling from Microsoft and insidkr leaks, does confirm official annonncement from Rockstar Games regarding release date.
Failing sym- bolic operatioss (13.9%)
Query: What's 's the average salary for software engineer working Amazon 2022?
Answer: According to Indeed.com, the average salary for soflware engineer working at Amazon in the United States is $131,9300 per year,
Reference: Average salary 1477,147 Salary estimated from 3,61 employees, users, and past and present joh advertisements Indeed in the past 12 mooths. Last updated: April 18, 2023. b Ground Truth: Contradictury
GPTT4: Altributable. The reference does provide average salary of $132,147 a sollware engineer working Amazon, which is very close to the claim's is figure of $131,930.
Table 3: Qualitative analysis on misclassified examples from prompting GPT-4 in AttrScore.
formance. For instance, the Vicuna (13B) model sees the overall accuracy on the two test sets rise from 34.6% and 41.4% in the zero-shot setting to 66.0% and 71.3%, respectively. And the fine-tuned FLAN-T5 (770M) model can even surpass Chat- GPT on both test sets. Despite this, there is still large room for further improvement. Some models that yielded better results on the simulated test set may be less effective on the annotated test set, indi- cating a lack of consistency across diverse testing settings, signaling generalizability challenges. Models struggle most notably with contradic- tory errors. Detecting contradictions is partic- ularly complex because it requires the model to weigh one piece of information in the statement against another in the reference, a process that ne- cessitates advanced tne-ggained information com- parison and reasoning capabilities. Consequently, even the best-performing model GPT-4 and the fine- tuned models often fail when faced with contradic- tory inputs, most often treating them attributable (sce qualitative analysis in Section 5.2).
numbers, dates, and time. Besides, the model mis- understands task definition and misinterprets logi- cal relations implied by labels (22.2%)). The model also struggles with symbolic operators (13.9%). For example, it fails to distinguish "cqual to' (=) and 'approximately equal to' (<)) in numeric com- parisons. In the left cases, the model tends to over- look the context clues and does not make judgments by conditioning on the reference (e.g.. potentially relying on its own parametric knowledge).
Our observations point to two potential direc- tions for improvement: 1) training prompting models to be more faithful and strongly conditioned on the reference (Zhou et al., 2023), especially pay- ing attention to fine ane-grained information; and 2) augmenting an LM-based evaluation method with external tools for different types of numerical and logical operations that are hard to be accurately performed only by the LM itself (Chen et al., 2020; Mialon et al., 2023). Similarly, we do qualitative analysis for ChatGPT in Appendix Section E.
5.3 Ablation Study
5.2 Qualitative Analysis
In this section, we perform an ablation study to test how each task influences the fine-tuned LMs' results and analyze the prompt sensitivity in zero- shot and few-shot settings for prompting LLMs. Contribution of individual task. We show the per- formance of models fine-tuned on individual task datasets and their combinations in Figure 4. We se- lect a representative from each group of the models under the fine-tume fine-tuned setting in Table 2. Our findings
To shed light on the space for future improvements in attribution evaluation, we qualitatively examine all the error examples of GPT-4 in the zero-shot setting, Representative examples are in Table 3. Our first observation that a significant portion (30.6%) of errors happen due to fine-grained in- formation insensitivity: failure in comparing very fine- mneggrinned information such as numerical values,

---

AttrEval- Simulation
AttiEval GenSearch
Task Prompts
Zero-shot
Few-shot
Models
Overall Micro Ft
70-
70
Sim.
Gen.
Sim.
Gen.
80-
Attr.
34.8
34.4
31.3
33.5
50
NLI
32.1
35.5
32.1
33.6
16
Alpaca
Fact.
34,0
33,9
32.7
46.7
30
30
Sum.
33.6
33,5
31.6
34.8
Roborta
FlanT5 GPT2
Alpaca (7B)
Roberta (330M) (770M)
FlanTs
GFTZ (1.58)
Apaca (7B)
Average
33.6
34.3
31,9
37.2
(330M)
(770M) (1.58)
QA
NLI
Fact.
Sum
Combined
Attr.
37.2
45.1
37.6
51.4
NLI
45.0
61.7
35,8
56.1
Figure 4: The influence of individual task data. Com- bining datasets generally improves model performance.
ChaiGPT
Fact.
44.8
54,9
43.2
54.9
Sum.
45.6
58.1
40.2
50.6
Average
43.2
55.0
39.2
53.3
suggest that examples from our simulated QA and fact-checking task most significantly improve per- formance for the attribution evaluation task, hinting at strong link between these tasks. Furthermore, integrating various related task datasets generally leads to better performance, particularly on out-of- domain test instances AtrrEval-GenSearee Sensitivity of prompts. The choice of prompts used to evaluate language models can have an im- pact on their performance. We evaluate the sensitiv- ity of prompts for AttrScore under both zero-shot and few-shot settings of Alpaca (7B) and ChatGPT, We show four types of prompts as mentioned ear- lier: a prompt designed specifically for our evalua- tion setting (Attri.), an NLI prompt, a fact-checking prompt (Fact.), and a summarization hallucination detection prompt (Sum.). These prompts are pre- sented in Appendix Table 6. As shown in Table 4, fact-checking and NLI prompts generally perform better, similar tasks may have been seen during their instruction tuning phase.
Table 4: Sensitivity of prompts for prompting LLMs on AttrEva-SSimllaton (Sim.) and -GenSearch (Gen.). The prompts include a prompt for attribution (Attri.), NLI prompt, fact-checking prompt (Fact.), and a summarization hallucination detection prompt (Sum.).
validity of such attribution remains questionable.
Evaluation of attribution. To evaluate attribu- tion, Liu et al. (2023) conduct a human evaluation to audit the verifiability of responses from genera- tive search engines. They find that these engines frequently contain unsupported statements and in- accurate citations, which strengthen the need to carefully examine the attribution of generations (Rashkin et al., 2021). However, human evalua- tions are very expensive and time-consuming. Gao et al. (2022); Bohnet et al. (2022); Gao et al. (2023) propose to automatically evaluate attribution by lev- ering II.I models (Honovich et al., 2022; Kamoi et al., 2023; Gekhman et al.. 2023). We study this problem in more comprehensive and realistic manner: 1) we explore how helpful other relevant tasks besides NLI
6 Related Work
attribution evaluation; 2) our evaluation setting is based on both benchmark examples and real examples.
Attributed LMs. Generative LMs often produce
hallucinations (Maynez et al., 2020; Dziri et al., 2021; Lee et al., 2018; Shuster et al., 2021; Wang and Sennrich, 2020; Xiao and Wang, 2021; Ji et al., 2023). To alleviate the issue, recent work proposes augment LLMs (Mialon al., 2023) with exter- nal tools (Schick et al., 2023; Li et al., 2023; Qin et al., 2023) such as retrievers (Guu et al., 2020; Lewis et al., 2020; Shuster et al., 2021; Izacard and Grave, 2021; Izacard et al., 2022; Borgeaud et al., 2022; Trivedi et al., 2022; Qian et al., 2023) and scarch engines (Nakano et al., 2021; Komeili al., 2022; Thoppilan et al., 2022; Yao et al., 2022; Glaese et al., 2022; Shuster al., 2022; Peng et al., 2023). Incorporating external references for gen- cration inherently implics that the generated state- ment is backed by these references. However, the
7 Conclusion
In this paper, we investigate the important prob- lem of automatically evaluating attribution given by LLMs. We begin by defining different types of attribution errors and then explore two approaches for automatic evaluation: prompting LLMs and fine-tuning smaller LMs. We experiment with both simulated test examples and manually curated test examples from real-life generative search engine. The results highlight both promising signals and re- maining challenges for the automatic evaluation of attribution. We hope our work could lay the foun- dation for future studies on this important problem.

---

8 Limitations
Acknowledgements
Currently, smaller models in AttrScore are fine- tuned on the combination of simulated or repur- posed datasets from related tasks. However, this dataset still has gaps from the real scenario. More- over, the error patterns in these simulated datasets might he overly simplistic and lack diversity, which can limit the models' ability to effectively handle more complex and varied real-world errors. It is also worth noting that these simulated datasets may contain noise and erroneous labels, which could fur- ther impede the models' learning and subsequent performance. How to obtain higher-quality train- ing data for attribution evaluation at scale can be major focus for future development.
The authors would like to thank the anonymous re- viewers and colleagues from the OSU NLP group for their thoughtful comments. This research was supported in part by NSF ISS 1815674, NSF CA- REER 1942980, NSF BAC-21126066 and Ohio Supercomputer Center (OSC, 1987). The views and conclusions contained herein are those of the authors and should not be interpreted as represent- ing the official policies, either expressed or implied, of the U.S. government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copy- right notice herein.
Our annotated evaluation set, AttrEval- GenSearch, is derived from New Bing, which uses GPT-4 as its backbone. It is crucial to note that we also use GPT-4 for evaluating attribution on which achieves the best per- formance with around 85% overall accuracy. Some bias might come from GPT-4 both generating the test examples and evaluating the attribution, which could potentially skew our understanding of the model's 's true performance. We therefore caution against over-optimism. We also acknowledge that is moderate, which
References
Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thome, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocaraseu, and Arpit Mittal. 2021. FEVEROUS: fact extraction and verifi- cation over unstructured and structured information. In Proceedings of the Neural Information Processing Systems Truck on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual.
Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Chris- tian Hansen, and Jakob Grue Simonsen. 2019. Mul- tiFC: A real-world multi- dataset for evidence- based fact checking of claims. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNILP), pages 4685-4697, Hong Kong, China.
the size of may not fully represent the real use setting of attributed LLMs.
While acknowledging current limitations, sev- eral promising directions emerge for future re- search and enhancement, For example, one can diversify data sources to include examples from variety of generative search engines, not just New Bing. In addition, it may be beneficial to annotate larger-scale queries that cover a broad spectrum of topics, styles, and perspectives.
Petr Baudis and Jan Sedivy. 2015. Modeling of the question answering task the yodaqa system. In Ex- perimental IR Meets Multilinguality, Multimodaliry, and Interaction 6th International Conference the CLEF Association, CLEF 2015, Toulouse, France, September 8-11, 2015, Proceedings, volume 9283 of Lecture Notes in Computer Science, pages 222-228. Springer.
9 Ethics Statement
This research project involves evaluating attribu- tion given by attributed LLMs. We collect and annotate data for evaluation using publicly avail- able information on the web, with the assistance of generative search engine, New Bing. We ac- knowledge that LLMs have the potential to repro- duce and amplify harmful information present in the data. We made an effort mitigate this risk by carefully selecting our evaluation data and by con- ducting analyses to identify and mitigate potential risks in the process.
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-amn answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1533-1544, Seattle, Wash- ington, USA.
Bernd Bohnet, Vinh Q Tran, Pat Verga, Roce Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein. Kuzman Ganchev, Jonathan Herzig, Kai Hui, et al. 2022. Attributed question answering: Evaluation and modeling for attributed large language models. ArXiv preprini, abs/221 1.08037.

---

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiss Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy. Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore. Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving. Oriol Vinyals, Simon Osindero, Karen Si- monyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning, ICMI. 2022, 17-23 July 2022, Bal- timore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 2206-2240.
language modeling with pathways. ArXiv preprint. abs/2204.02311.
Hyung Won Chung, Le Hou, Shayne Longpre. Bar- ret Zoph, Yi Tay. William Fedus, Eric Li, Xuezhi Wang. Mostafa Dehghani. Siddhartha Brahma, al. 2022. Scaling instruction finetuned language models. ArXiv preprint, abs/2210.11411
instruction-finetueed
Nouha Dziri, Andrea Madotto, Osmar Zaïane, and Avishek Joey Bose. 2021. Neural path hunter: Re- ducing hallucination in dialogue systems via path grounding. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Process- ing, pages 21 2197-2214, Online and Punta Cana, Do- minican Republic.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empiri- cal Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal.
Jinlan Fu, See-Kiong Ng, Zhengbat Jiang, and Pengfei Liu. 2023. Gptscore: Evaluate as you desire. ArXiv preprint, abs/2302.04166.
Luyu Gao, Zhuyun Dai. Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vin- cent Y Zhao, Ni Lao, Hongrac Lce, Da-Cheng Juan, et al. 2022. Rarr: Researching and revising what language models say, using language models. ArXiv preprint, abs/2210.08726.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Hemighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling large language models to generate text with citations. ArXiv preprint, abs/2305. 14622705.14667.
Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Sypektor. 2023. Trueteacher: Learn- ing factual consistency evaluation with large lan- guage models. ArXiv preprint, abs/2305.11171
Amelia Glaese Nat McAleese, Maja Trębacz. John Aslanides, Vlad Firoiu, Timo Fwalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoche Thacker, et al. 2022. Improving alignment of dialogue agents via targeted human judgements. ArXiv preprint, abs/2209. 14375.
Meng Cao, Yue Dong, and Jackie Cheung. 2022. Hal-
lucinated but factual! inspecting the factuality of hallucinations in abstractive summarization. In Pro- ceedings of the 60th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers), pages 3340-3354, Dublin, Ireland.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming- Wei Chang. 2020. Retrieval augmented language model pre-training, In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research. pages 3929-3933
Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, and Quae V. Le. 2020. Neural symbolic reader: Scalable integrationn of distributed and sym- bolic representations for reading comprehension. In 8th International Conference on Learning Represen- tations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.
Or Honovich, Roce Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE: re- re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 3905-3920. Association for Computational Linguisties.
Wei-Lin Chiang. Zhuohan Li, Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph E. Gonzalez, Ion Stoica, and Eric Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gauray Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling
Gautier Izacard and Edouard Grave 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th

---

Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages 874-880, Online.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Was Wei Chang. Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452 466.
Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-YY Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with re- trieval augmented language models. ArXiv preprint. abs/2208.03299.
Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fan- njiang, and David Sussillo. 2018. Hallucinations in neural machine translation. In Interpretability and Robustness in Audio, Speech, and Language Work- shop, Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, Canada.
Ziwei Ji, Nayeon Lee. Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveyx, 55(12):1
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zeulemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume Long Papers), pages 1601-1611, Vancouver. Canada.
Patrick S. H. Lewis. Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. eetrieval-augmentt knowledge-intents NI.P tasks. In Advances in Neu- ral Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurlPS 2020, December 12, 2020, virtual.
generation for
Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. 2023. Wice: Real-world entailment for claims in wikipedia. ArXiv preprint, abs/2303.0143.
Minghao Li, Feifan Song. Bowen Yu, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. Api- bank: A benchmark for tool- augmented lims. ArXiv preprint, abs/2304.08244.
Vladimir Karpukhin, Barlas Oguz. Sewon Min, Patrick Lewis. Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLPP), pages 6769-6781, Online.
Nelson F. Liu, Tianyi Zhang, and Perey Liang. 2023. Evaluating verifiability in generative search engines. ArXiv preprint, abs/2304.09848..
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy. Mike Lewis, Luke Zettlemoyer. and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. ArXiv preprint, abs/1907.1 1692..
Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. Scitail: A textual entailment dataset from science question answering. In Proceedings of the Thirty- Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Arti- ficial Intelligence (TAAI-18), and the 8th AAAI Sym- posium on Educational Advances in Artificial Intel- ligence (EAAI-18) New Orleans, Louisiana, USA, February 2-7, 2018, pages 5189-5197.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not trust language models: Inves- tigating effectiveness and limitations of paramet- ric and non non-parametric memories. ArXiv preprint, abs/2212.10511
Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2022. Internet-augmented dialogue generation. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8460-8478, Dublin, Ireland.
Joshua Maynez. Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factual- ity in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, pages 1906-1919, Online.
Neema Kotonya and Francesca Toni. 2020. Explainable automated fact- checking for public health claims. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 7740-7754, Online.
Grégoire Mialon, Roberto Dessi, Maria Lomeli, Christo- foros Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi- Yu, Asli Celikyilmaz, Edouard Grave. Yann LeCun, and Thomas Scialom. 2023. Aug- mented language models: a survey, ArXiv preprint, abs/2302.07842..
Wojciech Kryscinski, Bryan McCann, Caiming Xiong. and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarivation. In Proceedings of the 2020 Conference Empirical Methods Natural Language Processing (EMNLP), pages 9332-9346, Online.
Rejichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse,

---

Shantanu Jain, Vineet Kosaraju, William Saunders. et al. 2021. Webgpt: Browser-assisted question- answering with human feedback. ArXiv preprint,
Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Ture, and David Reitter. 2021. Mcasuring attribution in natu- ral language generation models. ArXiv preprint, abs/21 1 12870..
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language under- standing. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 4885 4901,, Online.
Timo Schick, Jane Dwivedi-Yuu Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. ArXiv preprint, abs/2302.04461..
OpenAL. 2023a. Chatgpt (mar 14 version) [large lan- guage model]. https//chaa.opemaicccm//hhhh com//hat.
Tal Schuster, Adam Fisch, and Regina Barzilay. 2021. Get your vitamin C! robust fact verification with contrastive evidence.
OpenAL GPTT- technical report. ArXiv preprint, abs/2303.08774.
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 624-644 Online.
OSC. 1987. Ohio supercomputer center.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll I.. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models follow instruc- tions with human feedback. In NeurIPS.
Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee. and Danqi Chen. 2021. Simple entity-centric ques- tions challenge dense retrievers. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 6138-6148, Online and Punta Cana, Dominican Republic.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021. pages 3784-3803, Punta Cana, Do- minican Republic.
Baolin Peng. Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu. Weizhu Chen, et al. 2023. Check your facts and uy again: Improving large language models with external knowledge and automated feedback. ArXiv preprint, abs/2302. 12813.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Fric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. 2022. Blenderbot deployed conversational agent that continually learns to responsibly engage. ArXiv preprint, abs/2208.033188
Hongjing Qian, Yutao Zhu, Zhicheng Dou, Haoqi Gu, Xinyu Zhang. Zheng Liu, Ruofei Lai, Zhao Cao. Jian-Yun Yun Nie, and Ji-Rong Wen. 2023. Webbrain: Learning to generate factually correct articles for queries by grounding on large web corpus. ArXiv preprint, abs/2304.04455.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction followwin llama model. https:// gi thub. cow woom/tttsu-llb http:/stdfooddapa....o
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Ren Fung, Yusheng Su, Huadong Wang. Cheng Qian. Runchu Tian, Kunlun Zhu, Shihao Liang. Xingyu Shen, Bokai Xu, Zhen Zhang. Yining Ye. Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu. Zhemning Dai, Lan Yan. Xin Cong, Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, and Maosong Sun. 2023. Tool learning with foundation models. ArXiv preprint, ahs/2304.08335
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha. Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du et al. 2022. Lamda: Language models for dialog applica- tions. ArXiv preprint, abs/2201.088 108239..
James Thorne, Andreas Vlachos, Christos Christodoolopoulls, and Arpit Mittal. 2018. FEVER: large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pages 809-819, New Orleans, Louisiana.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188):
Colin Raffel, Noam Shazeer. Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach, Learn Res,, 21:140: 1-14677
James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2019. The FEVER2.0 shared task. In Proceedings of the

---

Second Workshop on Fact Extraction and VERifica- tion (FEVER), pages 1 1-6, Hong Kong, China.
Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio. William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpoiQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions. ArXiv preprint. abs/2212.10509
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. ArXiv preprint, abs/2210.03629.
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Con- ference Empirical Methods Natural Language Processing (EMNLP), pages 7534-7550, Online.
Liammin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, 7hanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judg- ing llm-as-a-judge with mt-bench and chatbot arena. ArXiv preprins, abs/2300.05685.
Chaojun Wang and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural ma- chine translation. In Proceedings the 58th Annual Meeting of the Association for Computational Lin- Online.
Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. 2023. Context-faithful prompt- ing for large language models. ArXiv preprint, abs/2303. 13315.
guistics, pages
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M. Dai, and Quoc V. Le. 2022a. Finetuned language models are zero-shot learners. In The Tenth International Conference Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022.
Jason Wei. Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022b. Emergent abilities of large language models. ArXiv preprint, abs/2206.07682.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122, New Orleans, Louisiana.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, al. 2019. Huggingface's transformers: State-of- the-art natural language processing. ArXiv preprint,
Yijun Xiao and William Yang Wang. 2021. On hal- lucination and predictive uncertainty in conditional language generation. Proceedings of the 16th Con- ference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2734-2744, Online.
Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2023. Adaptive chameleon or stubborn sloth: Unraveling the behavior of large language models in knowledge conflicts. ArXiv preprint, abs/2305. 13300.

---

A Data Simulation
through the provided link and retrieve negative evi- dence from Wikipedia dumps using BM25, In En- tity Quyestitoonns (Sciavolino et al., 2021), we match positive evidence in Wikipedia texts searched by the question entity and retrieve negative evidence via BM25.
A.1 Simulation QA
Since have questions, and their ground truth answers and reference contexts, we can directly treat them as "Attributable" examples.
Attributable.
Converting short answers to long sentences. Since many of the attributed LLMs generate long sentences to the query, to make it our simulated data more realistic, we convert short answers to long answers using ChatGPT. Specifically, we prompt ChatGPT with the instruction "Convert given question and answer pair into plain sen- tences. [Question| [Answer]".
Contradictory. To simulate contradictory errors, we consider two methods. The first method in- volves modifying the correct answer by replacing it with different candidate generated from off- the-shelf QA model, an answer substitution model, random span generator. The second method involves keeping the original answer and replacing the answer span in the reference context with a sim- ilar candidate. The QA model, the answer substitu- tion model, and the random span generator are all implemented by prompting FLAN-T5-XL. (3B) (Chung et al., 2022) with different task prompts in Appendix Table 5.
A.2 Simulation Fact Checking
With provided Wiki content as evidence in FEVER (Thorne al., 2018) and Adversarial FEVER datasets (Thorne et al., 2019), we repur- pose 'SUPPORTS' examples as attributable, 'RE- FUTES' as contradictory, and 'NOT ENOUGH INFO' as extrapolatory. Using the same label mapping, we apply this approach to the claim and evidence provided in VITAMINC (Schuster et al., 2021), after removing duplicated examples as shown in FEVER. For FEVEROUS (Aly et al., 2021), we concatenate all pieces of evidence, in- cluding tables and texts, and prepend an increasing index as the final evidence. We then ground the label into our three categories using the same label mapping. Regarding natural claim datasets with various label spaces, we keep the top 6 classes out of 117 in MultiFC (Augenstein et al., 2019) and map them to defined three categories. In PUBHEALTH (Kotonya and Toni, 2020), we con- sider both 'unproven' and 'mixture' classes as ex- traplanetary. We also regard the abstract of the article as evidence. For SciFact (Wadden et al., 2020), we repurpose 'SUPPORT" as attributable and "CONTRADICT" contradictory. Addition- ally, randomly sclect sentence from the abstract of other articles evidence for the 'Not enough information' class construct extrapola- tory examples.
Extrapolatory. To simulate extrapolatory errors, we employ a BM25 retriever to retrieve external documents that do not contain ground truth answers from knowledge sources like Wikipedia or the Web. And then we replace the original paragraph with of the retrieved documents. For the answer, we either keep the original ground truth answer or leverage a QA model generate an answer. Here are more details for constructing negative retrieved documents each dataset.
Following previous work (Karpukhin et al., 2020), utilize the passages from Wikipedia dumps for constructing evidence for NaturalQues- tions (Kwiatkowski tions (Berant et al., 2013), and TREC (Baudis and Sedivy, 2015) datasets. In particular, we regard the highest-ranked passage including answers from BM25 as positive evidence and the top passage without answers as negative evidence.
al., 2019), WehQues-
For TriviaQA (Joshi al., 2017), we select the passage with the highest overlap with answers from web texts as positive evidence and the top-ranked wiki passage without answers from BM25 as neg- ative evidence. We exclude examples where the positive evidence has an overlap ratio of less than 0.5 with answers. For HotpotQA (Yang et al., 2018), combine the ground truth passages pro- vided as positive evidence and randomly select two out of eight passages provided as negative evi-
A.3 Simulation NLI
Natural language inference (NLI) aims to de- termine whether hypothesis is true given a premise. In NLI datasets such as SNLI (Bow- man et al., 2015), MultiNLI (Williams et al., 2018), ANLI (Nie et al., 2020), and SciTail (Khot et al.,
dence. Similarly, in PopQA (Mallen et al., 2022),
we find positive evidence from Wikipedia content

---

2018), the hypothesis is considered the claim and the premise regarded the evidence. The orig- inal labels in NLI datasets, namely 'Entailment', "Contradictory". and 'Neutral', are mapped to "At- tributable', "Contradictory", and "xtrapolatorr'.
D Generative Search Engine Examples Annotation Protocol
We show the detailed annotation guidelines in the following.
A.4 Simulation Summarization
Summarization involves condensing a given pas- sage or article into brief sentences while preserving its original meaning. To simulate contradictory ex- amples,
datasets with annotations of halluci- nations. In terms "SuumHHalucinalle al., 2020), we merge examples with the same ID and consider those with the most intrinsic hallu- cination as contradictory and those with the most extrinsic hallucination as extrapolatory. Paired full articles and ground truth summaries are treated as attributable examples. For XENT (Cao et al., 2022), Non-factual Hallucination' and 'Intrinsic Halluci- nation' seen as contradictory, 'Factual Halluci- nation' extrapolatory, and 'Non-ballucinated" as attributable. Each article and reference are paired as attributable examples. Finally, we resplit the manually annotated dev and test sets for training and evaluation in FactCC (Kryscinski et al., 2020), with 'NCORRECT"' labeled as extrapolatory and "CRRRCTT' as attributable.
(Maynez
B
Label and Subset Distributions of Training and Test Sets
We show the label and data sources' distributions of training and -immllatnn sets in Figure 5 and Figure 6.
C Prompts for LLMs as AttrScore
We show different kinds of prompts for using LLMs as AttrScore in Table 6. And we show the few-shot demonstrations in Table 7.

---

Annotation Guidelines
Overview
Thank for participating in this annotation task. The goal of this task is to create query and verify whether given reference document fully supports generation the query.
There are two sub-annotation tasks:
Create query based few given keywords under topic.
Verify whether given answer to query fully supported by references.
Task 1: Create a query for a specific domain.
You will be shown list of keywords (e.g.. inflation rate, CPI, GDP, unemployment rate, etc.) from specific domain tapic (e.g., economics)
demo question (e.g., What the unemployment in Germany 2020?) as an inspiration. Then you will asked to create a new query based an these keywords.
Task 1: Query Creation
Submit
Topic: Economics
Inflation rate
CPI
GDP
Oil price
Unemployment rate
Semple Unemployment nelers the share the lebor force without but available for and soeking employment. Germany unomplayment for 2021 3.54%, 0.28% dealine from 2020. Germany unemployment rate for 2020 3.61%, a 0.57% increase from 2019. Germany unemployment rate for 2019 was 3.14%, 0.24% decline from 2016. Germany unemployment rate for 2018 was 3.38%, 0.57% decline from 2017.
Enter your query
Task Verify whether the generated statement is supported by its reference.
You will be shown a user query, a generative search engine's 's response, and associated references. You will need to read the query. response, and reference carefully and verify whether the cited evidence fully supports the generation of the query.
You will be given three options:
"Attributable": The reference fully supports the generation of the query.
"Extrapolatory": generation contradicts the reference.
c. "Contradictory": The reference does not contain sufficient information to infer the correctness of the generation.

---

Consider the following examples:
(1) "Attributable"
Query: What temperature range on the moon?
Response: The average temperature on the moon can range from -298 degrees F (-183 degrees C) at night to 224 degrees (106 degrees during the day
Reference: The average temperature Fahrenheit (-183 degrees Celsius), at night, to 224 degrees Fahrenheit (106 degrees Celsius) during the day. Because the Moon has no...
Moon (at the equator latitudes) varies -298 degrees
Annotation: Attributable
(2) "Extrapolatory"
Query: What is the highest price of gallon of regular gas in the US in 2022?
Response: According
news article by NPR, of June 2022, the highest average price of gallon
regular gas in US was $6.34 in Califomia [1].
Reference: U.S. gas prices hit a new record on Monday, with gallon of regular gas costing an average of
$4 865 (not adjusting for inflation), according to the most recent data from the American Automobile Association...
Annotation: Extrapolatory
(3) "Contradictory"
Query: What was the unemplayment rate Germany in 2020?
Response: According to [1], the unemployment rate in Germany for 2020 was 4.31% which was 1.17% increase from 2019. However, important to nate that the COVID-19 pandemic had significant impact unemployment rates Germany around world.
Reference: Germany unemployment rate for 2020 was 3.81%.
Annotation: Contradictory
Task 2: Verilication
Submit
Query: What the unemployment rate Germany 2020?
Response: According to [1), the unemployment rate Germany for 2020 4.31%, which increase from 2019. However, is important significant impact on unemployment rates in Germany and around the world.
that the COVID-19 pandemic had
Reference:
[1]
Germany unemployment rate for 2020 was 3.81%.
Does the evidence fully supports the response?
Attributable
Extrapolatory
Contradictory

---

Tasks
Prompts
Context: [Contexi]\n
QA
Based on Context, [Question]
Please provide related term or substitution for the given input, which should be different from the input.
"Input: Biden; Output: Obamaln"
"Input: 1949; Output: 1358'n"
Answer Substitution
"Input: University of Maryland: Output: University of Cambridgeln"
"Input: 09/12/2014 4; Output: 03/30/2008m"
"Input: $431; Output: $769An"
"Iaput: [Ground Truth Answer]: Output:
",
Random Span
Extract phrase from the given passage. in Pussage: [Context]
Generation
Table Prompts for QA, answer substitution, random span generation when simulating contradictory emrors
Label distribution combined
distribution simulation
genssearch
50
40
60
42.48%
33.33%
33.33%
33.33%
52.89%
Perceniage (%)
30
28.51%
29.01%
Percentage
Percentage
40
33.47%
20
8
13.64%
10
10
Albrixutable Contradictory Extrapolatory
Attiiuuablee
Contradictory Extrapolatory
Contredictory Extrspoletory
Figure 5: Label distribution of training and test sets.
Distribution data source the combined training set
Distribution data source AttrEval-Simulation
XSUNHA
VERQUESTIO
5.0%
MITANIN
ENTITYQUEST 0%
13.0%
NO S1 50%
SCRFACT INTTA
TREC 1.3%
90 TAIL
RUSHEALTH
EPC
POPDA
HOTROTOA
FEVER
5%
21.0%
15.15
Figure 6: Data source distribution of combined training and NonEEval-iimulatio sets.
E Additional Qualitative Analysis
time; 2) failure performing symbolic operations (6.8%): the model fails to verify the claim which requires performing symbolic operations over the reference, such as verifying set relationships.
The qualitative results of ChatGPT are shown in Table Our first observation is that significant portion (79.4%) of errors happen due to ChatGPT overlooking the context clues and does not make judgments by conditioning on the reference (c.g., potentially relying on its own parametric knowl- edge). For the remaining error cases, they are: fine-grained information insensitivity (13.8%): Failure comparing very fine-grained informa- tion such as numerical values, numbers, dates, and
F Author Contribution Statement
Xiang Yue conceived the project, conceptualized and designed the study, conducted experiments, wrote the manuscript, and annotated New Bing test examples. Boshi Wang provided critical feedback and edits, revised the manuscript, contributed to the

---

Prompt Types
Prompts
#R8 Instruction:
As Attribution Validator, your task verify whether given context support the claim. A claim can be either plain sentence question followed by its answer. Specifically, your response should clearly indicate the relationship: Attributable, Contradictory or Extrapolatory. contradictory error occurs when you can infer the answer contradicts the fact presented context, while an extrapolatory etror means that you cannot infer conrectness bused the information provided the context.
Auribution
### Input:
Claim: [Question Answer] or [Plain Sentence) Inin
Context: [Context]
#*# Response:
### Instruction:
Fact-check claim based the given evidence.
Options: Supporied, Refuted or Not Enough Information
Input:
Fact-Checking
Claim: <Claim>An'n
Evidence: <Rvidence>>
### Response:
NWW Instruction:
Read the following and determine if the hypothesis can be inferred from the premise. Options: Entailment, Contradiction, or Neutral
### Input:
NLT
Hypothesis: <HHyoothesszAnii
Premise: <Premise>
#NN Response:
#** Instruction:
Read the following and determine whether souree can support the summary. Options: Support, Contradicts, Not Enough Information
Summarization
### Input:
Hallucination
Summary: <Summarysinia
Delection
Source: <Souree>
### Response:
Table 6: Prompt variations for test the sensitivity of different prompts on the results. We the "Attribution" prompt for our main experiments as default as achieves the best performance overall.
conceptualization of the study, conducted experi- ments for ChatGPT, and annotated New Bing test examples. Ziru Chen set up the training code base, annotated New Bing test examples, and conducted experiments for fine-tuning Roberta, Flan-T5,, and GPT-2. Kai Zhang contributed to all the simulation data preprocessing, revised the manuscript, and an- notated New Bing test examples. Yu Su and Huan Sun secured funding for the project, provided su- pervision and guidance throughout the study, con- tributed the conceptualization and design of the study, and edited the whole manuscript. All authors approved the final version of the manuscript.

---

Few-shot demonstrations
Here are some demonstration examples for you.
### Input:
Claim: In what year was the writer of the opera "Mazeppa" born? The writer of the opera "Mazeppa" was born in 1840.
Reference: Mazeppa, properly Mazepa, is an opera in three acts (six scenes) by Pyotr Ilyich Tchaikovsky. The libretto was written by Victor Burenin and is based on Pushkinl's poem "Poltava". Pyotr llyich Tchaikovsky (25 April/7 May 1840 - 25 October66 November 1893), often anglicized as Peter llich Tchaikovsky, was Russian composer of the romantic period, some of whose works are among the most popular music in the classical repertoire. He was the first Russian composer whose music made a lasting impression internationally. holstered by his appearances as guest conductor in Europe and the United States. Tchaikovsky was honored in 1884, by Emperor Alexander III, and awarded a lifetime pension.
#!H Response:
Attributable. From the given reference, Pyotr Ilyich Tchaikovsky was the writer of "Mazeppa". who was born in 1840 stated in the given claim.
### Input:
Claim: In what part of Buenos Aires Province city that has The Juan Carlos Castagnino Municipal Museum of Art located? is located in the southeast part.
The Juan Carlos Castagnino Municipal Museum of is a museum of fine arts Mar del Plata, Argentina. Mar del Plata is an Argentine city in the cast part of Buenos Aires Province located on the coast of the Atlantic Ocean. is the head General Pueyrredón Partido. Mar del Plata is second largest city in Buenos Aires Province. The name "Mar del Plata" has the meaning of "sea of the Silver region" or "adjoining sea to the (River) Silver region". Mar del Plata is one of the major fishing ports and the biggest scaside beach resort Argentina.
Reference:
### Response:
Contradictory. The reference says that Mar del Plata is in the east part of Buenos Aires, so The Juan Carlos Castagnino Municipal Museum of Art is also in the east part of Buenos Aires, not southeast.
### Input:
Claim: In what year was Louis Joseph de Francesl's mother born? Louis Joseph de Frances's mother was born in 1755.
Reference: Lakshman Joseph de Saram is film composer and classical musician. Bom in Colombo. Sri Lanka and educated at the Royal College, Colombo, the High School of Performing Arts. Manhattan School of Music and Juilliard Pre- Pre-College in New York City, Joseph de Saram influential in the music of South Asian art cinema, having scored many international award-winning films like \'Between Two Worlds'' and V Akusa Kusum.V X' His best-known score is to the 2012 film "Bel Ami". Joseph de Saram is also artistic director of The Chamber Music Society of Colombo. Louis Joseph de Lorraine "Duke of Guise" and Duke of Angoulême, (7 August 1650 - 30 July 167 1) was the only son of Louis, Duke of Joyeuse and Marie Françoise de Valois, the only daughter of Louis Fiimanell dV Angoulême, Count of Alès, Governor of Provence and son of Charles de Valois Duke of Angoulême, bastard of Charles IX of France.
##H Response:
Extrapolatory. The answer the question cannot be inferred from the given reference since the person "Louis Joseph de Frances" is not mentioned. We also cannot find any evidence in the Reference which contradicts with the provided answer.
Now here the example that needs your response:
Table 7: Few-shot demonstrations for prompting LLMs AttrScore.

---

Error Type Exumple
(percentage)
Overlooking
Query: ketchup
before?
contextual
Amwerr: Yos, that's irue! Ketchup suld remedy inligestion, diarthen, juundice und ricumatiim in the 1830s
anes
Reference: Decades later, in 1870s, American entreprancur Henry Heinz created modern version ketchup with ripe tomatres, distilled vinegor, hrown sugar, and variety spices. 1905, the company sold five million bottles of keschhup, cementing place as the condiment we know and love today.
(79.4%)
Extapolatory
ChatGPT: Atributable. The reference supports the claim that ketchup sold
remedy for various ailments in the 1830s.
Fine-grained
Query: How many total miles of crastline does Norway have?
information
Amser: The oflicial length was revalculated in 2011 as 100,915 km including Gords and islands.
incensirivity
straight along Norway's
coastal perimeter) 2,6%0
(1,600 long. Along 25,000 kilometers (16,000 mi).
(13.8%)
fjonds, islands, and buys, resulting low-reselation cossiline
Ground Truth: Contradictory
Norway's constline, including and islands, 100,915 km. Therefore, the claim that Norway has total of 100,915 km of cousiline is suppocted by the Reference.
CharGPT: Arrihutable. The reference provides information the official
Failing symbolic operators
on
Query: ls Andrew Ng am muthor the recent paper "generativg agents: inteructive simulacra human behavior")
Answer: Yes, Andrew
of authors paper "Generative agents: Interactive simulacra of human behavior".
Reference: Computer Science > Human-Crome Comuutee Intemetion (Sahmitted on Apr 2023) Generrive Agents: Interaetive Simulacra Meredith Ringel
(6.8%)
Sung Park. Juseph B'Brien. Gronad Troth: Contradictory
Percy Liang
Bermstein.
ChulGPT: Auributable. The reference clearly states that Andrew Ng is one the authors of the puper "Generative Agents: Interactive
Simmlacra of Haman Rehavior", which supports the claim
author the paper.
Table 8: Qualitative analysis on misclassified examples from prompting ChatGPT in AttrScore.


Reviews:
[{"review_id": "jg2d97xhUc", "reviewer": "Reviewer_BA69", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our gratitude for your feedback to our paper. We are pleased that you recognized the contributions of our paper and would like to address the confusion over the aim of our work here: \n\n**Clarification of the Paper's Aim**: Our paper presents the first comprehensive framework for evaluating attribution in LLMs, which includes **three interconnected components:**\n\n- **A new definition of attribution types** (Lines 073-088): We propose a more fine-grained categorization for attribution, identified as attributable, extrapolatory, and contradictory. This nuanced classification not only enhances the safety of LLM usage but also provides critical insights for future mechanisms tailored to correct specific errors.\n\n- **Two approaches for automatic evaluation of attribution** (Lines 089-093): Based on our proposed definition, we explore two methods for identifying and categorizing the attribution errors: prompting LLMs and fine-tuning language models on simulated and repurposed data from related tasks.\n\n- **The introduction of two new datasets for evaluating attribution** (Lines 96-105): To facilitate the evaluation of attribution, we propose two test sets: AttrEval-Simulation and AttrEval-GenSearch. The former is a synthetic dataset repurposed from related tasks, and the latter is an annotated dataset curated from queries in a generative search engine, New Bing. These datasets not only provide a comprehensive testing ground for our evaluation approaches but also contribute to the broader research community by filling the gap in attribution evaluation datasets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. New classification of types of attributions.\n2. Two new methods for automatic evaluation of the attribution ability of LMs.", "reasons_to_reject": "The aim of the article is unclear - is it presenting methods to collect a new dataset, or the actual evaluation approaches?", "questions_to_authors": "", "paper_topic_and_main_contributions": "This article presents two new approaches for the automatic evaluation of LM's ability to provide answers, which are supported by external references. The article presents a new classification of LM attribution, composed of 3 classes, and explains how data is collected.\n\nContributions:\n\n1. New classification of types of attributions.\n2. Two new methods for automatic evaluation of the attribution ability of LMs.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sHJEGU3jdF", "reviewer": "Reviewer_XrPK", "summary": "", "questions": "A) How does the data format/structure in QA, fact-checking, NLI relate to the results shown in Fig. 4? What are the similarities to the references in New Bing dataset? \n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback on our work. We appreciate your recognition of the significance of the attribution problem we study, the different evaluation methods we explore, and the connection of attribution evaluation to tasks like NLI, fact-checking, and QA.\n\n---\n\n**Novelty of Problem Formulation:**\nYes, our attribution error categories do share similarities with NLI categories, but it's important to distinguish between attribution evaluation and NLI tasks. NLI primarily concerns the “logical'' relationship between the premise and hypothesis, with the premise typically being very short. Conversely, attribution evaluation concentrates on verifying whether fine-grained information in the reference supports the generation, with the reference typically being a lengthy document.\n\nWhile Gao et al. (2022) and Bohnet et al. (2022) do frame attribution as a *binary* NLI classification task, we argue that this oversimplification misses the nuance of attribution. Our work proposes a more **fine-grained categorization**, enhancing safe system usage and providing insight for the development of error-specific correction mechanisms. Our experiments show that **NLI models do not perform as well on our two test sets, suggesting that NLI and attribution may require different handling.**\n\n---\n\n**Domain Gap between SNLI and Attribution:**\nYes, there is indeed a gap! We're really glad you brought this up, as it's a crucial aspect of our work. We recognized this gap early on, and **it's precisely why we didn't limit ourselves to just NLI datasets or off-the-shelf NLI models** which existing work mostly focused on. We took a more exploratory route, delving into other related tasks like fact-checking and summarization, and even going a step further to propose methods to simulate data from QA. Our study can provide more insights into how these relevant tasks can be transferred to attribution evaluation (thanks for noting that in the Reasons To Accept). For example, according to the results in Figure 4, open-domain QA and fact-checking share more similarities to the attribution task, and combining the four datasets leads to the best generalizability. \n\n---\n\n**Evaluation on Simulated Data:**\nWhile we acknowledged in the Limitation section (Line 525-530) that the simulated data has gaps from the real scenario, it's worth noting that our **simulated data is designed to mimic real use cases** in generative search engines. Our **simulation methods are inspired by real examples**, thus providing a valuable proxy for testing our framework. For example, to simulate extrapolatory errors where generated statements contain additional information that cannot be verified in the reference, we employ a BM25 retriever on the question, retrieving relevant external documents, which do not contain the ground truth answers. \n\n---\n\n**AttrEval-GenSearch Dataset Size:**\nWe acknowledge the relatively small size of our AttrEval-GenSearch dataset in the Limitation section (Line 547-550). We would like to point out there are similarly small eval datasets that are widely used in evaluating LLMs: HumanEval (164 programming problems) [1], MBPP (500 Python programming problems) [2]\n\n- [1] Chen, Mark, et al. \"Evaluating large language models trained on code.\" arXiv preprint arXiv:2107.03374 (2021).\n- [2] Austin, Jacob, et al. \"Program synthesis with large language models.\" arXiv preprint arXiv:2108.07732 (2021).\n\nAdditionally, it's important to highlight several strengths of our dataset:\n- **Uniqueness:** Our dataset fills a gap in our community, as no similar attribution evaluation datasets exist.\n- **Diversity:** Despite its small size, our dataset spans 12 diverse domains.\n- **Timeliness:** It's based on the latest LLMs and generative search engine, New Bing, ensuring timeliness.\n- **Data Contamination:** We created queries from scratch, avoiding the data contamination issue commonly seen with existing LLMs..\n- **Reproducibility:** We released our annotation protocol and interface in the Appendix, enabling future researchers to replicate our process and annotate more datasets.\n\nWith that said, we would take expanding the dataset size as one of our future steps, as noted in the paper (Line 551-558). \n\n---\n\n**Presentation Improvements**: We appreciate your suggestions for improving the presentation of our paper. We will correct the typo you've pointed out and move more data repurposing details from Appendix A to the main content.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses attribution in LLMs which is important and especially relevant to extend LLMs to further applications.\n\nIt is helpful for the community to understand the effectiveness of various strategies for tasks like attribution; this paper’s exploration of both prompting and finetuning language models, especially recent ones, is appreciated.\n\nThe paper notes parallels in attribution between various subtasks of NLP (NLI, fact-checking, QA); the premise of transferring data/knowledge from those tasks is creative. The experiment to gauge impact by dataset type on performance is interesting. \n", "reasons_to_reject": "Novelty of problem formulation is slightly limited.\n- Formulation of attribute errors is highly reminiscent of NLI categories (i.e. entailment, neutral, contradiction). \n- NLI for attribution has been studied as noted in (Gao et al. 2022) and (Bohnet et al. 2022).\n\nEvaluating on simulated data does not necessarily represent the type of attribution seen in practice with the paper’s use case (e.g. search engines). \n\nThe size of AttrEval-GenSearch is relatively small (~250 images), bringing into question the significance of the results extending past simulated data. Additionally, the dataset may have limited use for future researchers due to this size. \n\nWhen repurposing data from SNLI, the premises are single sentences. However, in the proposed attribution application, the “premises” are likely to be full references. There seems to be a significant domain gap, which could explain the ineffectiveness of NLI data in Figure 4. \n", "questions_to_authors": "A) How does the data format/structure in QA, fact-checking, NLI relate to the results shown in Fig. 4? What are the similarities to the references in New Bing dataset? \n", "paper_topic_and_main_contributions": "The paper explores the problem of attribution with LLMs by defining the task as having the classes attributable, contradictory, and extrapolatory, and testing out whether finetuned and prompted language models can succeed at classification.\n\nThe paper proposes to repurpose data from related tasks (NLI, fact-checking, QA) through mapping samples to the proposed problem formulation.\n\nPerformance is benchmarked and analyzed on two datasets they contribute: a 4K sample simulated dataset using QA data and a 242 sample test set generated from New Bing. Contradictions are found to be a common error pattern. \n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "More data repurposing details would be helpful in the Approach section rather than Appendix A.\n\nLine 361 type - equivelant -> equivalent \n"}, "extra_text": {}}, {"review_id": "yewZCkPV3q", "reviewer": "Reviewer_82hu", "summary": "", "questions": "Question A: While I appreciate the inclusion of the ‘contradictory’ label, I am not sure why a label for partially supported claims was not considered. I am thinking, in particular, about sentences with multiple claims and the scenario where the same sentence contains multiple references.  How often does this occur throughout the dataset?\n\nQuestion B: Why do you think the few-shot performance is so underwhelming for the Alpaca/Vicuna/ChatGPT models (which actually regress on average, compared to the zero-shot setting)?\n\nQuestion C: Does the size of the inputs have an impact on the results?  We are told specifically that inputs for Alpaca and Vicuna are truncated to 2048 tokens but we know that ChatGPT and GPT-4 can accept longer inputs. Are there inputs that do not fit in the 2048 limit (or the 512 size used during fine-tuning)? If yes, this information should be presented (along with the sizes accepted by the versions of ChatGPT/GPT-4 used in the experiments).\n\nQuestion D: I believe at least a few samples might be incorrectly parsed (check lines 22-30 and 38-47 in AttrEval-GenSearch.csv and the parsing code in prompt_chatgpt_gpt4). Also, is the sample starting on line 38 labelled correctly (two games are missing from the reference enumeration and the answer itself contains only 8 examples)?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and constructive feedback on our work! We appreciate your recognition of the significance of the attribution problem we study and the comprehensive evaluation methods we explore.\n\n---\n\n**Potential Bias:**\nWe acknowledge the potential bias of using GPT-4 for both generation and evaluation, as stated in the Limitation section (Lines 537-556). We agree that emphasizing the results of smaller fine-tuned models in the follow-up discussion would be beneficial. To address this, we will solve your comments in Question B and C (see more below). Moreover, we have now **tested the performance of GPT-4 on the full AttrEval-Simulation dataset**, achieving comparable results to the subset evaluation (54.5 vs. 55.6 overall accuracy).\n\n---\n\n**Question A:** \nWe apologize if our original categorization was unclear. We do consider partially supported claims, and their labels depend on whether the unsupported part contradicts the reference (“contradictory”) or is not mentioned in the reference (“extrapolatory”).  For example, in the last example of Table 3, the answer can be mostly supported by the reference except the key information “$131,930” is wrong. We treat such examples as \"contradictory.\" \n\nWe agree that sentences with multiple claims and references do appear in real practice. However, our task setting prioritizes one reference per statement, a unit task that more complex scenarios can be decomposed to (Line 125-130). We study such a setting as it forms the basis for dealing with multiple references or distinct segments. \n\n---\n\n**Question B:** \nYour question regarding the underwhelming few-shot performance of Alpaca, Vicuna, and ChatGPT models is insightful. We hypothesize this is because few-shot methods are often very sensitive to the demonstrations provided, which is a known problem. We will try different demonstration selection strategies and provide more discussion about this in the revision.\n\n---\n\n**Question C:** \nWe appreciate the reviewer's query about the input sizes. While some examples in our datasets do exceed 512 or 2048 tokens, these instances form a relatively small percentage (e.g., 4.1% and 0.2% respectively for the training set). We will add such information in the revision. But we do agree that dealing with very long sequences remains a challenge for future work. \n\n---\n\n**Question D:** \nWe appreciate the reviewer's diligence in identifying these instances! They are correctly parsed in the csv file but wrongly in the browser (due to new line symbol “\\n”). And the label in line 38 should be “extrapolatory” as two games are missing in the reference. We will thoroughly review all the examples and their labels again. \n\n---\n\n**Typos, Grammar, Style, and Presentation:**\nWe are grateful for the reviewer's keen eye in identifying typos and presentation-related issues. We will correct them in our revised manuscript to enhance clarity and readability. Additionally, we will report the performance of GPT-4 on the full set instead of subset as we stated in the previous response under **Potential bias**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper explores an important present-day problem. A large selection of models are investigated on this task. The authors explore the limitations of the strongest available model (GPT-4). Additionally, the value of the provided training dataset is reflected by the competitive performance of relatively small fine-tuned models (Roberta/GPT2), which out-perform significantly larger LLMs (Alpaca/Vicuna) in a zero/few-shot setting.", "reasons_to_reject": "Given the (acknowledged) potential bias of using GPT-4 (which was used for both generation and evaluation) and the fact that it was only evaluated on a subset of the test data, I think it might have been interesting for the follow-up discussion to highlight more the results of the smaller models.\n", "questions_to_authors": "Question A: While I appreciate the inclusion of the ‘contradictory’ label, I am not sure why a label for partially supported claims was not considered. I am thinking, in particular, about sentences with multiple claims and the scenario where the same sentence contains multiple references.  How often does this occur throughout the dataset?\n\nQuestion B: Why do you think the few-shot performance is so underwhelming for the Alpaca/Vicuna/ChatGPT models (which actually regress on average, compared to the zero-shot setting)?\n\nQuestion C: Does the size of the inputs have an impact on the results?  We are told specifically that inputs for Alpaca and Vicuna are truncated to 2048 tokens but we know that ChatGPT and GPT-4 can accept longer inputs. Are there inputs that do not fit in the 2048 limit (or the 512 size used during fine-tuning)? If yes, this information should be presented (along with the sizes accepted by the versions of ChatGPT/GPT-4 used in the experiments).\n\nQuestion D: I believe at least a few samples might be incorrectly parsed (check lines 22-30 and 38-47 in AttrEval-GenSearch.csv and the parsing code in prompt_chatgpt_gpt4). Also, is the sample starting on line 38 labelled correctly (two games are missing from the reference enumeration and the answer itself contains only 8 examples)?\n", "paper_topic_and_main_contributions": "The paper presents an approach to automatically verify if a statement generated by a LLM is supported by the cited reference. The authors formulate the problem as a classification task, wherein a <question, statement, reference> pair is labeled as “attributable”,”extrapolatory” or “contradictory”.\n\nThey study two main directions: fine-tuning open-source LMs, and prompting SOTA LLMs in a zero/few-shot setting. The data used for fine-tuning is repurposed from several representative corpora from the fields of QA, fact-checking, NLI and summarization. For evaluation two types of data are considered, which encompass 12 domains: samples derived from QA datasets and author annotated samples from the New Bing search engine.\n\nThe authors experiment using a comprehensive set of models and provide an interesting discussion of the results.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "L361: equivelant\nL1105: extraplanetary -> extrapolatory\nL1154: “. And” -> and\nTable 5 Description missing “.”\nTable 6 description: “prompt variations for test the sensitivity”\n\nI believe the information that GPT-4 was only tested on 500/4000 samples from the test set should be more prominent, perhaps in the description of Table 2 (rather than in a footnote).\n\nAlso, in Table 2 it may be informative to display the maximum context size (if the size of the inputs makes it relevant).\n"}}, {"review_id": "4hqI1MnM7U", "reviewer": "Reviewer_DLKU", "summary": "", "questions": "A. Regarding evaluations of fine-tuned models on the AttrEval-Simulation dataset, is it technically similar to standard supervised training evaluations? While training and testing differ in that they use distinct datasets (as per line 286), their methods for generating extrapolatory and contradictory examples seem to be similar. This similarity could limit the implications of such evaluations.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful review and constructive feedback on our work. We appreciate your acknowledgment of our effective evaluation approaches and our exploration of repurposed and simulated datasets for fine-tuning smaller models.\n\n---\n\n**Bias in AttrEval-Simulation:**\nWe understand your concern about potential bias in the 'AttrEval-Simulation' dataset. While we acknowledge its limitations in the paper (Line 537-556), we believe it provides a valuable starting point for understanding the strengths and weaknesses of our framework. Our **simulation methods are inspired by real use cases in generative search engines**, providing a valuable proxy for testing our framework. Please also see our response to Reviewer XrPK under **Evaluation on Simulated Data**. \n\n---\n\n**Limitation of the evaluating model:**\nThis is a good point! However, practical considerations like efficiency and budget often restrict the use of large language models like GPT-4 for the evaluation. In such scenarios, our results show that **fine-tuned smaller models can still achieve reasonably good performance** (>70% accuracy) and even outperform ChatGPT in many cases, despite being trained on simulated datasets. We note that in the future if large-scale realistic datasets for training are available, the performance of these smaller models could be higher. \n\n---\n\n**Extrapolatory class is an inappropriate category:**\nThe decision to use a three-way classification reflects our aim to provide a nuanced understanding of attribution errors. **If a reference is “relevant” but does not directly contradict or support the generated statement, we treat this example as an “extrapolatory” error**. This is particularly important in high-stakes domains like healthcare and finance, where seemingly correct but unsupported statements can lead to significant risks. \n\nRegarding “the LLMs might be providing the correct or an appropriate answer due to their inherent generalization abilities,” that’s true, but still, attributable LLMs should always avoid generating statements without supporting references, which is what our extrapolatory class captures.\n\n---\n\n**Questions A. Fine-tuning models on the AttrEval-Simulation dataset:**\nYou made a very good point! Our simulation is designed with model generalizability in mind from the outset. We ensure this by using different QA datasets for training and testing simulations (Table 1; Line 279-289). Although the methods for generating 'extrapolatory' and 'contradictory' examples are similar in both datasets, the **actual data instances used for the simulation come from different sources**. In fact, for the test set simulation, we use **six different QA datasets**. This diversity in sources ensures that our model faces a wide range of scenarios and discourages memorization of specific examples, thus promoting generalizability tests.\n\nNotably, even when trained on the simulated training set, the performance of the model on the simulated test set (AttrEval-Simulation) is not perfect, with the best-performing model achieving just 66% accuracy. This result further emphasizes the generalizability challenge of our AttrEval-Simulation dataset.\n\n---\n\n**Typos, Grammar, Style, and Presentation Improvements**\n\nFootnote 5: We apologize for any confusion. By \"verifiability\", we mean whether the generated statement is backed by the cited reference, which is a similar concept to attribution and is used in prior work (Liu et al., 2023). By \"relevance\" we mean whether the answer (which may or may not be supported by the reference) correctly responds to the user's query. We will make them more clear in the revised version.\n\nLine 226: The simulation method described here was used for generating part of the fine-tuning data and evaluation data. However, as we stated in the response to Question A, the actual data instances used to simulate fine-tuning and evaluation sets are sampled from different QA datasets to ensure the generalizability test. We will revise the text for clarity.\n\nLine 309: ‘Verification process’ does refer to the process mentioned in Line 301. We will work to improve the clarity in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThis paper demonstrates how closely the automatic attribution evaluation aligns with human evaluation.\n2.\tThe research shows that models trained on repurposed (simulated) datasets result in improvements in the AttrEval-GenSearch dataset, which comes from actual generative models.", "reasons_to_reject": "1.\tThe evaluation on 'AttrEval-Simulation' might be biased or may exhibit limited diversity due to the constrained ways of generating each label. This might not truly represent the types or the degree of challenges in evaluating attributions for LLM-generated answers.\n2.\tEvaluation by a model can be limiting when the evaluated model (LLMs which generate answers and attributions) outperforms the evaluating model. The evaluating model might penalize the evaluated model even if the latter's attribution is correct.", "questions_to_authors": "A. Regarding evaluations of fine-tuned models on the AttrEval-Simulation dataset, is it technically similar to standard supervised training evaluations? While training and testing differ in that they use distinct datasets (as per line 286), their methods for generating extrapolatory and contradictory examples seem to be similar. This similarity could limit the implications of such evaluations.", "paper_topic_and_main_contributions": "This paper presents an automatic method to evaluate LLM's attributions, which link the generated text to the evidence texts. Automatic attribution evaluation is treated as a three-way classification task, with labels being: attributable, extrapolatory, and contradictory. The authors evaluated several different models for automatic attribution evaluation. These models include fine-tuning the LLMs on QA, Fact-checking, NLI, and Summarization datasets, as well as zero-shot or few-shot LLMs.\n\nThe models were evaluated on two test sets: one comprised of simulated data built from QA datasets, and the other collected from answers and documents generated by New Bing. The experiments demonstrated varying performances across different models.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Footnote 5: It's unclear how the authors differentiate between “verifiability” and “ ’relevance’, i.e., whether to answer correctly responds to the query”. \nLine226 : This paragraph describes how the ‘contradictory’ and ‘extrapolatory’ examples are generated. I am confused if this data was only used for the fine-tuning of the LLM or if it was also employed in the evaluation for ‘AttrEval – Simulation’. \nLine 309: Does the term ‘verification process’ refer back to what's mentioned in Line 301? It was once lost in connecting it."}}]

Author rebuttal:

Meta-review: The paper focuses on using LLMs for automatic evaluation of attribution. Specifically, it makes three contributions: (a) a new NLI-inspired categorization of attribution types, (b) new datasets (simulated and real) for evaluating attribution, and (c) models for automatic evaluation of attribution.

This paper addresses an important problem and provides reasonable yet clearly imperfect solutions. Specifically, reviewers have concerns that their simulated data are biased and that their "real" test dataset is too small. NLP has had a fairly established precedent of using simulated data (especially for training) in settings where human data collection is costly. Typically, such data are indeed biased in certain ways, but are quite often still useful at least until a large costly human-curated dataset becomes available. The concern that their real/unsimulated evaluation set is small (~250 examples) is a more serious weakness.  Nonetheless, I think a fair bit of work has gone into the paper, and it will likely generate more followup work. Overall, I'd be inclined to see it accepted. I also hope that the authors will consider expanding the size of the evaluation set to make this paper stronger.
Decision: Accept-Findings