Enabling Large Language Models to Generate Text with Citations
Authors: Tianyu Gao
Howard Yen
Jiatong Yu
Danqi Chen
Venue: EMNLP 2023 Main (2023)
Keywords: large language model
llm
citation
attribution
qa
evaluation
benchmark
TL;DR: We propose a benchmark to automatically evaluate using LLMs to generate text with citations.

Abstract: Large language models (LLMs) have emerged as a widely-used tool for information seeking, but their generated outputs are prone to hallucination. In this work, our aim is to allow LLMs to generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search engines and human evaluation, making it challenging to reproduce and compare different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs’ Citation Evaluation. ALCE collects a diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve supporting evidence and generate answers with citations. We develop automatic metrics along three dimensions—fluency, correctness, and citation quality—and demonstrate their strong correlation with human judgements. Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement—For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time. Our analyses further highlight promising future directions, including developing better retrievers, advancing long-context LLMs, and improving the ability to synthesize information from multiple sources.

Enabling Large Language Models to Generate Text with Citations
Tianyu Gao Howard Yen Jiatong Yu Danqi Chen
Department of Computer Science & Princeton Language and Intelligence
Princeton University
hyenn [rlayveseeeen.jlaatoooyoooeoeeep bangicc)es..rrinceto.cco princeton.dd.
Abstract
Question When did the US break away from England?
Large language models (LLMs) have emerged widely-used tool for information seeking, but their generated outputs are prone to halluci- nation. In this work, aim allow LLMs generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search gines and human evaluation, making it chal- lenging to reproduce and compare different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs' Citation Evaluation. ALCE collects diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve sup- porting evidence and generate answers with citations. We develop automatic metrics along three dimensions- flunncy,, correctness, and ci- tation quality- and demonstrate their strong correlation with human judgements. Our exper- iments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement- For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time. Our analyses further highlight promis- ing future directions, including developing bet- ter retrievers, advancing long-context LLMs, and improving the ability to synthesize infor- mation from multiple sources."
step towards gaining independence GB when declared independence on July 2, 1776 (although the event is now commmemtorated on July 4, 1776, date when the officially adopted Congress) (1l12). The Treaty of París was later signed on September 3, 1783, formally separating
Retrieve
Declaration of independence
States from the British Empire (3).
[s] Decoionization the Americas In
Desoloniazioon..
[]] American Bavolu- Hlazs The Treaty
It was off dialy adoo- sed by Co Congress on July 1178.
Corpus
depandiica from
Paris was signed
July 1776
Soptember 3, 3,1353...
The task setup of ALCE. Given a question. the system generates text while providing citing pas- sages from a large retrieval corpus. Each statement may contain multiple citations (c.g., [1][2]).
Figure 1:
to provide citations to one or few text passages for any statement they generate (Figure 1). Incor- porating citations brings several benefits: (1) users can easily verify LLMs' claims with the provided citations; (2) LLMs can generate text that faithfully follows cited passages, which has the promise to improve correctness and alleviate hallucination.
Multiple commercial systems have adopted this paradigm: Bing Chat² and perplexity.ai respond to user questions in natural language with references to Web pages. Nakano et al. (2021); Menick et al. (2022) share similar motivation, but they mainly experiment with commercial search engines and closed-soure source models, making their results difficult to evaluate. aarrivall-augeenee LMs (Borgeaud et al., 2022; Izacard et al., 2022) incorporate re- trieved passages during both training and infer- ence, but do not guarantee faithfulness to retrieved passages explicitly provide citations. Addition- ally, previous studies mostly rely on human eval- uation (Nakano et al., 2021: Menick al., 2022: Liu et al., 2023), which expensive and difficult reproduce. We argue that the absence of automated evaluation hinders the advances of such systems.
1 Introduction
Large language models (LLMs; Brown et al., 2020; OpenAI, 2023) have gained increasing popularity tool for information seeking. While they gener- ate engaging and coherent responses, their outputs are prone hallucination and often contain fac- tually incorrect information (Ji ct al., 2023). This makes harder for users to trust and verify LLM- generated outputs without any supporting evidence. In this work, we study new generation paradigm for LLMs, in which we require LLMs
'Our code and data are available at https://github httppprinneoonnn//AL..
hhtpps///ww. com/new hhttps://ww..eerpeextty tt.pleiitty..i.

---

Dataset
Corpus (#passsages)
Question type
Example
ASQA
Wikipedia (21M)
Factoid (ambiguous)
Q: When did the US break away from England?
The US declared independence July 1776 [1][2] ... The
Treaty of Paris was later signed September 3, 1783 [3].
QAMPARI
Wikipedia (2I
Factoid (list)
Which films have Gong
a member of their cast?
A: The Story of Qiu Ju [1]. Farewell My Concubine [2]. The
Monkey King 2 [3]. Mulan [3], Saturday Fiction [3]
ELI5
Sphere (899M)
Why/How/What
How do student loans affect getting a mortgage?
Student loans affect the deht income [1], which is
a key factor determining the amount that ... [2][3]
Table 1: The three datasets used in our ALCE benchmark. These datasets cover a wide range of question types and the corresponding corpora span from Wikipedia to Web-scale document collection.
We present ALCE, the first reproducible bench- mark for automatically evaluating LLMs' gener- ations with citations. ALCE assumes natural- language question and retrieval corpus, and re- quires building end-to-end systems to retrieve rel- evant passages from the corpus, generate a re- sponse to the question, and cite corresponding sup- porting passages. We compile three datasets that cover different types of questions and corpora- ASQA (Stelmakh et al., 2022), QAMPARI (Ru- bin et al., 2022), and ELI5 (Fan et al., 2019) as shown in Table 1. Different from previous bench- marks (Lee al., 2019; Bohnet et al., 2022), ALCE evaluates long-text generation, focusing on auto- matically evaluating citation quality, and allows citing multiple passages for individual statements.
with post-hoc citing achieves good correctness but much worse citation quality; (2) although interac- tive retrieval approaches (Yao et al., 2023; Schick et al., 2023) offer more flexibility in when/what to retrieve, they do not improve the performance on this challenging benchmark; (3) summarizing the retrieved passages in a shorter text improves correctness but not citation quality; (4) reranking multiple generations boosts citation quality mea- sured by human evaluation; (5) incorporating more retrieved passages in context does not help Chat- GPT but improves GPT-4 performance.
Our extensive analyses highlight three major challenges of building LLMs to generate text with citations: (1) the retrieval quality is crucial the final performance and has substantial room for im- provement; (2) LLMs' limited context window re- stricts the number of passages they can incorporate; (3) current L.Mss struggle to synthesize multiple documents in context without being distracted by irrelevant ones, although better instruction tuning brings significant improvement, These challenges pose promising research directions for developing better systems integrating retrieval and LLMs.
We design automatic evaluation methods in three dimensions: fluency, correctness, and citation quality. Specifically, we use MAUVE (Pillutla et al., 2021) to measure fluency, propose tailored correctness metrics for each dataset, and adopt a natural language inference (NLI) model (Honovich et al., 2022) to measure citation quality. We show- case how the three dimensions together contribute to robust evaluation, preventing systems from ploiting shortcuts. Additionally, we conduct human evaluation and demonstrate a strong correlation with our automatic metrics.
2 Task Setup and Datasets
Our task is formalized as follows: Given query and corpus of text passages DD the system is required to return output S, which consists of statements $1, ... Sn, and each statement s; cites a list of passages C, = {c,, (G,,,.G.,..... C. J"", where Cij E D. In this work, we segment LLMs' output into statements by sentence boundaries." While LLMs may include sentences that do not require a citation, such as "I'm happy to help", we observe that almost all sentences that L.LMs output provide
We experiment on multiple systems with state- of-the-art LLMs and retrievers and also propose novel prompting strategies to synthesize retrieved text into text generation. Although all systems are capable of providing fluent and coherent responses,
there remains substantial room for improvement in terms of correctness and citation quality: For exam- ple, on the EL15 dataset, around 50% generations of our ChatGPT and GPT-4 baselines are not fully supported by the cited passages. Additionally, we find that (1) a closed-book. model (generating an- swers without accessing any retrieved documents)
'In practice, we allow at most 3 citations for each state- ment as more citations usually do not help.
'QAMPARI requires list as the answer, and we choose each entity in the generated list as a statement.

---

valuable information and require citations, similar findings in Liu al. (2023). In this work, cita- tions are enclosed by box brackets such as [1][2].
3 Automatic Evaluation
Our benchmark measures the following three di- mensions of system responses:
We divide the corpus D into 100-word passages following previous works on open-domain question answering (Karpukhin et al., 2020; Petroni et al.,
Fluency: whether the model's generated text is fluent and coherent.
Correctness: whether the answer is accurate and covers all aspects of interest.
2021; Piktus et al., 2021), in contrast to commer- cial systems like Bing Chat, which cite entire Web pages. We take 100-word passages because it is easier for humans to verify, and allows for more retrieved passages to fit in LLMs' limited context.
Citation quality: whether the answer is well supported by the cited passages and no irrelevant passages are cited.
In the following, we present automatic metries for each dimension and discuss why the combination of the three metrics provides a robust evaluation.
We choose QA datasets that (1) they contain factual questions. in which references are impor- tant; (2) questions require long-text answers that cover multiple aspects; (3) answering the questions requires synthesizing multiple sources. We select three datasets (Table 1) and introduce them below. See şB for additional statistics.
3.1 Fluency
We use MAUVE (Pillutla et al., 2021) to evaluate the fluency of the output (SC). We deploy MAUVE for ASQA and ELI5 and omit it for QAMPARI, QAMPARI only requires a list of short answers as the response and LLMs consistently adhere to the format in our experiments. As MAUVE is sensitive to output length and text style, and most LLMs are capable of producing fluent text, we mainly employ it as sanity check as long as the MAUVE scores are high enough.
ASQA (Stelmakh et al., 2022) is a long-form fac- toid dataset. As shown in Figure I, each question is an ambiguous question from AmbigQA (Min et al., 2020) that requires multiple short answers to cover different aspects, and the dataset provides long- form answer that covers all short answers. Since most questions can answered by Wikipedia, we use the 2018-12-20 Wikipedia snapshot as D.
QAMPARI (Rubin et al., 2022) factoid QA datasct constructed from Wikipedia, where the an- swer is a list of entities that are drawn from differ- ent passages. Same as ASQA, we use the 2018-12- 20 Wikipedia as the corpus.
3.2 Correctness
Our objective is to measure the informativeness and utility of the generation to the question. Liu et al. (2023) propose to directly evaluate perceived utility by humans, process difficult automate. There- fore, we use correctness -whether the response is accurate compared to a ground truth answer- as a proxy. Evaluating the correctness of long-form gen- cration is challenging task (Krishna et al., 2021), and we describe our strategy for each dataset be- low. Figure 2 illustrates the metrics and we include additional implementation details in SC.
EL15 (Fan et al., 2019) is a long-form QA dataset built on the Reddit forum "Explain Like I'm Five".6 6 Most ELI5 questions are how/why/what questions that require long answers and multiple passages evidence. Due the diverse topics dis- cussed in the questions, we use Sphere (Piktus et al., 2021)-a a filtered version of Common Craw1? as the corpus. The ELI5 dataset is widely used in related work due to its challenging nature (Nakano et ul., 2021; Menick et al., 2022; Liu et al., 2023).
For ASQA, we follow Stelmakh et al. (2022) and calculate the recall of correct short answers by checking whether the short answers (provided by the dataset) are exact substrings of the generation (exact match recall; EM recall).
We randomly select 1,000 examples from the development set of each dataset for ALCE. Our benchmark primarily assesses the citation capabili- ties of existing LLMs and does not provide training data, as there are no available examples that pro- vide supervision for citations in these datasets.
For QAMPARI, we follow Rubin et al. (2022) and calculate the precision and recall of the model prediction, by checking the exact match to the gold answer list. We add one additional adjustment: considering that users often want to know only a few example answers of the question, our evalua- tion considers recall to be 100% if the prediction includes at least correct answers (recall-5).
ttt http://////o/a///////o/ hww.mmmmooaawlo.rrgoogo We also filter out any Weh pages from Reddit.

---

ASD4 Exort Match Recall
Quession
When did the break away from England?
When did the US break away from England?
Modhloutput
[tatement 1) (3][2) (statement 2) [B)/ (2](4)83
Modalousper lorppr
it declared independencee on 3, 1776... The Treaty
Paris signed Soptember 2,1733...
Citation Recall
Short mooen (fom the datoue))
cuarg Recall:33.88
Racall If the concatenation of all cieed possages fully supports the sigment. determine "fully support".
Jully 1778
1,778,
September 3,1895
[statement [1]12] 4
Premise: passage (3I(2)
QAMPARI Precision, Recall
[statment [3]
Hypothesis: (statement 13
model
"ontallment"
Question
films Gong li member their
[2][4][5]
For this question, citation recall -2/3- 68%
Saturday Fiction. The Great Wall Mulann Shadow,
Short unswar Ifrs the clcase))
Soring exact
Ciesbon Presition
Story of Mulan Saturday Fiction
Recall-s6.8% Precision-50h
support the claim, removing does not offect other citations combined to support the claim.
"imreevant"
Recall
|statement [3]]2]
Queetion
do student leans affect getting mortgage?
Priscision 11
Pracision 1
They not appear credit history and do debt to income ratio. Thus they do not affect getting mortgage.
[statement
recall precision
[statement [2](4]]6]
Clime igenenned by coxt-dox -00--000
gold anowers
hu Claim recall-33..6
O 14
[2] "imveleann"" Precision =
Student does not appear credit history.
[2]15]
Student loan can the debt income ratic.
[5] [2]]4]
O Precision
Debc to income affects mortgage applications.
For this question, citation precision -//6- 66%
Figure 2: Evaluation of correctness (details in $3.2).
Figure 3: Evaluation of citation quality (details $3.3). We use NLI model verify whether statement is supported by its citations.
Unlike ASQA and QAMPARI, the ELI5 dataset does not provide short entity answers. Fan al. (2019) use ROUGE for evaluation, which does not reflect the correctness well (Krishna et al.. 2021: SA). Inspired by works in summariza- tion evaluation (Zhang and Bansal, 2021; Kamoi et al., 2023; Wang et al., 2020), we use Instruct- GPT (text-davinci-003) Ouyang et al., 2022) generate three "sub-claims". Then we use TRUE (Honovich et al., 2022), a T5-11B (Raf- fel et al., 2020) model fine-tuned on a collection of natural language inference (NLJ) datasets, to check whether the model output entails the sub-claims (claim recall). TRUE targets factual correctness and has been used by previous works in similar context (Bohnet et al., 2022; Gao et al., 2023). We demonstrate that claim recall provides more ac- curate measure of correctness than existing metrics (more details in SA).
passages. Figure 3 provides an illustrated example. We use the NLI model TRUE (Honovich et al.. 2022) again to automatically examine whether the cited passages entail the model generation. We con- duct human evaluation ($6) to demonstrate strong human correlation of our metric.
Citation recall. We calculate the citation recall of each statement (0 or 1) and average over all state- ments in the model response. For each statement 8; its citation recall is if and only if there is at least one citation (C; + 0) and ((concat(C.). s;) = 1, where o @prrmisee, hypothesis) is the NLI model that outputs 1 if the premise entails the hypoth- esis, and 0 otherwise; concat(C.) concatenates all passages in C₁ together (details in SC). The NLI evaluation is in accordance with the atributable to identified sources (AIS) framework (Rashkin et al.. 2023): (cconcat(C.). s:) = implies that 8, is true based solely on concat(C,).
3.3 Citation Quality
We evaluate citation qualities using two metrics: (I) citation recalf, which determines the output is en- tirely supported by cited passages, and (2) citation precision, which identifies any irrelevant citations. Although we prioritize citation recall as it entails a well-supported and truthful answer, enhancing precision is crucial for better user satisfaction, re- ducing the need for human review of extraneous
Citation precision. Our citation precision evalua- tion detects citations that are irrelevant, but it does not require citing a minimal set. We follow this de- sign because human writing often cites redundant sources to enhance credibility; human readers may also appreciate multiple citations, especially when it pertains to critical claims such as medical advice.
We calculate the citation precision for each ci- tation (0 or 1) and average over all citations in the
Phttps://huget inngggiing ng face. wt. htt///ooll/ttxxlll//e 11155_x1__rrr nii Details in SC.

---

Instruction: Write an accurate, engaging, and concise answer for
et al., 2022) and DPR (Karpukhin al., 2020); we use BM25 for Sphere. For each question, we retrieve the top-100 passages.
<Reriiee for the question>
Document [1](Title: American Decolonization)
4.2 Synthesis
Document [2](Title: Decolonization)
Document [3](Title: American Revolution)
We focus on how to prompt an LLM to interact with the retriever, and synthesize and cite the ev- idence (without fine-tuning internal parameters). One noteworthy challenge is that existing LLMs all have limited context window and thus can only fit a handful of passages.
Question: When did US break away from England? Answer: The United States took the first step towards gaining independence [1][2] The Treaty of Paris was later signed [3].
Table 2: An example of our VANILLA method. Differ- ent colors represent prompt, model generation, and <actions>. We also provide in-context demonstra- tions before the test example.
VANILLA. We simply provide the model with the top-k" passages and instruct the model cite cordingly (Table 2). We also use in-context learn- ing (Brown et al., 2020) and prepend two demon- strations. The complete instruction is in Table 23.
response. We first define citation is "irrele- vant". Intuitively, a citation Ci is "irrelevant" if (a) Cij itself cannot support and (b) removing cij does not affect the rest of the citations to support 8,. Formally, Cij is "irrelevant" if and only if
SUMM/SNIPPET, With a 4K context window, we can at most safely fit k = 5 passages. As shown in Figure 4, top-5 retrieved passages can only cover 56.8% percent of the answers in ASQA.
(a) @(cij, 8:) = 0, AND (b) @cconcat(C, \ (c..))), 8;) 1.
To tackle this limitation, we propose to provide summaries or snippets of passages instead of the full text (summaries are abstractive but snippets are spans from passages). We acquire summaries and snippets by prompting ChatGPT with instructions (prompts in Table 25 and 26). Then replace all passages with summaries/snippets. Summaries or snippets significantly reduce the passage length, allowing for more passages to fit in: for ASQA, they reduce passage length by 6x on average.
Gu has precision of if s; has recall-1 and aj is not irrelevant. For example (Figure 3), when 83 cites three references [2][4][5] and 1ecall=1, [2] is "irrelevant" if [[[2], 83) = and o[[4][5], 83) = 1. For condition (b) to work, we set recall=1 as a prerequisite for precision= 1. Note that this algorithm overlooks the scenario when one citation partially supports the statement, We discuss the details in $E.
Though SUMM/SNIPPET allows for more trieved passages, they are lossy compressions. To alleviate this problem, we propose INTERACT, an interactive prompting scheme to allow the model check the full text of certain passages. At each step. the model can execute one of three actions: (1) "Check: Document [1][2]" to check the full text of the corresponding documents; (2) "Output:" to output statement of the answer, (3) "End. " to end the generation. $C provides more details.
3.4 ALCE is Robust to Shortcut Cases
We showcase how the ALCE evaluation is robust to two possible shorteuts $D: (1) using the top-1 retrieved passage as the response and citing itself, and (2) using the first two sentences of the top-1 passage. Both cases have almost-perien citation scores, but (1) has low fluency due to its unnaturally long length compared to human answers, and (2) has low correctness due to low coverage.
The above methods all dis- play retrieval results the beginning. In INLI- NESEARCH, we allow LLMs to call "search" dur- ing the generation process (Yao et al., 2023; Press et al., 2022; Jiang et al., 2023). At each step, the model can execute one of three actions: "Search:
INLINESEARCH.
4 Modeling
In this section, we discuss three major modeling components for an ALCE thesis, and post-editing.
syn-
4.1 Retrieval
"We can most = 3 for models with 2K window at most for models with 4K context window. "We also query ChaiGPT whether the passage is relevant to the question, and filter out passages that are "irrelevant".
We explore simple, off-the-shuee retrievers. We use dense retrievers for Wikipedia, including GTR (Ni

---

Instruction:
Fluency
Correct.
Citation
(MAUVE)
Rec.)
Prec.
<Retrieee for question "...">
Question: When us break away from England?
ChatGPT
Search: Declaration of Independence
VANILLA (5-psg)
66.6
40.4
73.6
72.5
<Search the query among the top-10e passages>
w/ RERANK
77.0
40.2
84.8
81.6
Document [1](Title: ...) ...
SUMM (10-peg)
70.0
43.3
68.9
61.8
Output: The United States
[1].
INTERACT
69.0
39.1
73.4
66.5
<Remove Document [1] from context>
SNIPPET (10-psg)
69.8
41.4
65.3
57.4
Search: Treaty of Paris
INLINESEARCH
58.7
32.4
58.3
58.2
<Search the query among the top-100 100 passages>
CLOSEDBOOK
52.7
38.3
26.7
26.7
Document [3](Title: ...)
GPT-4 (VANILLA prompting)
Output: The Treaty of Paris [3]. <Remove Document [3] from context>
GPT-4 (5-peg)
67.1
41.3
68.5
75.6
GPT-4 (20-peg)
64.9
44,4
73.0
76.5
End.
LLaMA (VANILLA prompting)
LLaMA-13B (3-psg)
68.4
26.9
10.6
15.4
Table 3: An example of INLINESEARCH.
Vicuna- 13B (3-psg)
82.6
31,9
51.1
50.1
Chat-13B (5-psg)
72.4
35.2
38.4
39.4
{query}" to search among the top-100 100 passages" by using GTR; the "Output" and "End" actions are the same INTERACT. For each "Search" action, we display the best retrieved passage in the context. The passage is removed after one action to save context space. Table 3 shows an example.
Chat-70B (5-psg)
88.3
41.5
62.9
61.3
Table 4: Experiments ASQA. For CLOSEDBOOK. we use POSTCITE to get citations. k-psg: putting top- passages from the retrieval results into the context. Chat-13B and Chat-70B refer to LLaMA-2-Chatt
CLOSEDBOOK. We also add a simple closed- book baseline, where the model is only prompted with the instruction and the question, without any retrieved passages provided. Consequently, this variant does not cite any evidences.
Oasst (Köpf et al., 2023). They all have a 2K con- text window. We use short instructions for LLaMA (Table 24) to save context budget. Additionally, we test LLaMA-2-Chirs which were also trained to follow instructions (Touvron et al., 2023b). These models have a context window of 4K tokens, which allows for 5 passages per question.
4.3 Post-editing
In this section we discuss two strategies for refining the output to further improve its quality.
5.1 Main Results
We present the main results three datasets in Table 4, 5, and respectively (full results in $G.6). We first note that all models achieve good fluency scores (except some models on ELI5 mainly due to their longer generations). We summarize the main takeaways from the experiments below.
RERANK. We randomly sample Ylsample = 4 re- sponses for each question, and select the best re- sponse using the automatic citation recall score. we expect RERANK to improve the citation quality.
POSTCITE. For each statement, we find the best matching passage among the top-100 retrieved pas- sages using GTR and cite it. We combine this with CLOSEDBoOK our experiments.
VANILLA achieves strong performance. Despite its simplicity, VANILLA (putting retrieved passages in context) achieves close-to-the performance among all prompting strategies.
5 Experiments
Using summaries or snippets improves correct- ness, We see a universal trend that SUMM or SNIP- PET improves correctness, though on ASQA and EL15, such an improvement comes at cost of citation quality due to the lossy compression. Com- bining INTERACT with SUMM/SNIPPET does not bring improvement, and we hypothesize that check- ing the full passages offers limited benefit and cur- rent LLMss are not proficient in an interactive usage.
We describe experiment details in SC. We use Chat- GPT 18pt-3.22 1ont-3.5-turban (-tuurbo-0001) with a 4K context window for most main experiments and ablations. We also report results with ChatGPT-16K (gpt- 3. 5-turboo16k-0613) and GPT-4 (gpt-4-0613;
8K context window). For open-source models, we test LLaMA (Touvron et al., 2023a) and its instruction-tuned versions, including Alpaca (Taori et al., 2023), Vicuna (Chiang et al., 2023), and
Retrieving text on the fly does not improve per- formance. All datasets show that VANILLA out- performs INLINESEARCH on citation quality (and
" We do not search over the entire corpus because (query) leave out certain context in the question and searching among the already-retrieved passages gives better results.

---

Correctness
Citation
Fluency
Correct.
Citation
Rec.-5
Prec.
Rec.
Prec.
(MAUVE)
(Claim)
Rec.
Proc.
ChatGPT
ChatGPT
VANILLA (5-psg)
20.8
20.8
20.5
20.9
VANILLA (5-psg)
57.2
12.0
51.1
50.0
w/ RERANK
22.8
21.4
21.2
w/ RERANK
56.1
11.4
69.3
67.8
SUMM (10-psg)
23.6
21.2
23.6
25.7
SUMM (10-psg)
40,3
12.5
51.5
48.2
SNIPPET (10-psg)
24.5
21.5
22.9
24.9
SNIPPET (10-psg)
62.9
14.3
50.4
45.0
w/ INTERACT
21.9
23.0
21.9
23.4
w/ INTERACT
68.0
13.3
47.8
45.0
INLINESEARCH
17.2
20.4
14.9
14.9
INLINESEARCH
49.7
13.4
45,6
43.7
CLOSEDBOOK
32.9
19.8
10.0
10.0
CLOSEDBOOK
32.6
18.6
15.5
15.5
GPT-4 (VANILLA prompting)
GPT-4 (VANILLLA prompting)
GPT-4 (5-psg)
22.2
25.0
25.9
27.0
GPT-4 (5-psg)
38,4
14.2
44.0
50.1
GPT-4 (20-psg)
29.6
26.2
27.4
28.5
GPT-4 (20-psg)
41.5
18.3
48.5
53.4
LLaMA (VANILLA prompting)
LLaMA (VANILL prompting)
LLaMA-13B (3-psg)
9.7
9.1
6.7
7.1
I.aMA-138 (3-pag)
3.1
Vicuna-133 (5-psg)
14.0
15.9
12.5
13.4
Vicuna- 13B (G-pag)
58.2
10.0
15.6
19.6
Chat-138 (5-psg)
21.1
18.2
9.6
9.7
Chat-138 (5-peg)
34.7
13.4
17.3
15.8
Chat-70B (5-psg)
21.8
18.4
15.1
15.6
Chat-70B (5-peg)
38.6
12.8
38.3
37.9
Table 5: Experiments QAMPARI. "Rec.-5": we set the recall to be 100% the prediction includes at least 5 correct answers.
Table Experiments on ELI5. We use claim recall for the correctness evaluation. Chat-13B and Chat-70B refer to I.aMA-2-Chat.
on correctness for ASQA and ELI5). By manually examining the examples, we find that it is challeng- ing to ask detailed questions without sceing any passages. To improve INLINESEARCH, one may need provide more context about the questions in advance or encourage the model to call retrievers with more detailed and diverse queries.
GPT-4 brings limited improvement but is better at using long context. We evaluate GPT-4 with VANILLA and different numbers of passages (more results in $G.6). GPT-4 brings consistent (but lim- ited) improvement on correctness, but often at cost of citation quality. GPT-4 can also incorporate more passages due to longer context window, which boosts both correctness and citation qual- ity. On the contrary, including more passages with ChatGPT-16K does not improve the results (Ta- ble 7), suggesting that processing more passages is non-trivial and GPT-4 is better at synthesizing information from its long context than ChatGPT.
RERANK boosts citation quality, We observe that RERANK leads to consistent improvement in cita- tion quality (on ASQA and EL15). As the automatic scores may be biased in RERANK, also conduct human evaluation ($6) and verify effectiveness.
CLoNEDBOOK+PPSTCTTE delivers strong cor- rectness but poor citation quality. CLOSED- BOOK outperforms VANILLA in correctness EL15 and QAMPARI, and has only a 2% gap on ASQA. However, CL.OSEDBoOK cannot provide any citation; when combined with POSTCITE, the citation quality remains inadequate. For instance, citation recall of CLOSLDBoOK+POSTCIIE lower than VANILLA by 47% on ASQA.
5.2 Comparison of Different LLMS
Table 7 compares different LLMs on ASQA using VANILLA (more results in $G.6). No- tably, instruction-tuned models (Vicuna-133) 13B and LLaMA-2-CHN outperform the original LLaMA models in correctness and considerably enhance the citation quality. We observe that while the orig- inal LLaMA models are able to copy facts from the context, they struggle with accurately citing the sources or simply do not cite. Notably, the best open-source model, ----MA----7B--hii achieves comparable correctness score as the OpenAl mod- els, but still lags behind in citation quality.
is
To understand why CLOKeDBoOK achieves het- ter correctness and why POSTCITE cannot deliver satisfying citation quality, we manually examine model outputs and find that: (1) open-book mod- cls easily distracted by irrelevant passages and generate responses with lower correctness, phe- nomenon also observed by Shi et al. (2023); (2) C1oOsEDBoOK often generates texts that are cor- rect but not similar to any retrieved passages, mak- ing it difficult to match a citation hoc.-hoc.
5.3 Retrieval Analysis
The retrieval results play crucial role to the cor- rectness and the citation quality. Figure presents the retrieval recall @ with different datasets and

---

QAMPARI
80
30
Fluency
Correct.
Citation
(MAUIEE)
(EM)
Rec.
Prec.
Racaligk
60
Oracte
20
ChatGPT(VANTLLA)
-0
Graci
DPR pagl
Vinio pg91
GTR (1-pag)
69.5
38.4
56.0
64.0
1PP
DPM 10
GTR (3ppa)
66.6
90
GTR (3-pag)
66.6
40.4
73,6
72.5
Figure 4: Retrieval @ on ASQA (EM recall), QAMPARI (recall-5), and ELI5 (claim recall). Retrieval recall serves an upper bound for model performance, and we compare them with two models' correctness results in the figure (dashed lines): "Vanilla (5-psg)" is ChatGPT VANILLA with top-5 passages in context; "Oracle" is the same model except that it uses gold passages ($G.I), whose recall matches Recall @ 100 on all three datasets.
Fluency
Correct,
Citation
improvement for ChatGPT. Specifically, correct- ness platcaus at top-1 passage and citation qual- ity plateaus at top-3. GPT-4 (Table 7) exhibits an increasing trend with more passages, but the improvement is not proportional to the retrieval performance. This indicates the limited ability of LLMs in utilizing multiple passages within context.
(MAUVE)
(EM Rec.)
Roc.
Prec.
Open-source (max #tokens2K-41
LLaMA-13B (3-psg)
68.4
26.9
10.6
15.4
Vicuna- 13B (3-psg)
82.6
31.9
51.1
50.1
Chat-13B (5-pag)
72.4
35.2
38.4
39.4
Chat-70B (5-psg)
88.3
41.5
62,9
61.3
ChatGPT (max #tokens=/K))
ChatGPT (3-psg)
66.6
39.6
72.8
73.9
ChaiGPT (5-psg)
66.6
40.4
73.6
72.5
5.4
Other Ablations
ChatGPT-16K (max #tokens=16K)
We provide additional ablations in $G. summary, we find that (1) using comprehensive instructions enhances the citation quality of instruction tuned models ($G.2); (2) including at least one demon- stration improves the performance ($G.3); (3) fine- tuned models (FiD: Izacard and Grave, 2021) with POSTCITE lag behind LLMs in both correctness and citation quality and fail to generalize ($G.4).
ChatGPT (5-psg)
60.3
36.1
76.2
76.5
ChaiGPT (10-pag)
56.3
36.7
75.3
75.0
ChatGPT (20-pag)
56.7
36.1
73.7
73.5
GPT-4 (max #tokens=8K)
GPT-4 (5-pag) GPT-4 (10-psg)
67.1
41.3
68.5
75.6
71.5
43.1
72.0
75.5
GPT-4 (20-psg)
64.9
44.4
73.0
76.5
Table 7: Comparison of different L.MMs on ASQA (GTR+VANILLA). -LaMA-13B and Vicuna- 13B have a context limit of 2,048 tokens, and thus can only use a short version of instructions and at most top-3 passages, Chat-13B and Chat-70B refer to LLaMA-2-Chat,
Human Evaluation
To verify that our automatic evaluation correlates with human judgement, we conduct human eval- uation on selected models and request workers to judge model generations on three dimensions simi- lar to Liu et al. (2023)((1 utility: a 1-to-5 score indicating whether the generation helps answer the question; (2) citation recall: the annotator is given a sentence and all passages that the sentence cited, and asked to judge whether the passages fully support the sentence; (3) citation precision: given a sentence and one of its citations, the annotator is asked to judge whether the citation "fully sup- ports", "partially supports", or "does not support" the sentence. Each citation gets a precision score 1 if the output sentence has a citation recall of I and this citation least "partially supports" it. See Appendix for more details.
retrievers, As the number of passages increases, re- trieval recall steadily improves. Additionally, Fig- ure shows the correctness performance of two models: (1) ChatGPT VANILLA with top-5 pas- sages (our primary baseline); (2) an oracle ver- sion of the same model employing 5 gold passages (SG.1; the 5 gold passages match the retrieval re- call@ 100). Notably, both models' correctness lags behind the corresponding retrieval recall (except for ELI5 top-5). The discrepancy suggests that de- spite the presence accurate answers in context, LLMs struggle to utilize them in their outputs.
We compare the impact of different retrievers and different numbers of passages LLMs. Fig- ure (right) shows that GTR outperforms DPR in both correctness and citation quality, emphasiz- ing the importance of deploying better retrievers. Contrary to the retrieval recall trend in Figure 4, more passages in context do not yield substantial
Model outputs score high utility. The utility scores do not differ significantly between models, ranging 3.7-3.9 for ASQA and 3.5-3.6 for ELIS. Upon inspection, all tested models are mostly able

---

Human scores
ALCE scores
model is provided the papers-to-cite and context and is required to recover the citing text. differ- ent from ALCE as citations are provided and the model only needs to perform the summarization.
Rec.
Prec.
Rec.
Prec.
ChatGPT VANILLA
74.7
76.6
75.3
74.4
w/ RERANK
79.3
81.9
83,9
80.8
Vicuna-13B VANILLA
51.6
51.5
50.3
50.1
Retrieval-augmmnted LMs. Many studies have ex- plored augmenting LMs with externally retrieved information. Guu et al. (2020); Borgeaud et al. (2022); Izacard et al. (2022) pre-train language models with retrieved passages, while Khandel- wal et al. (2020); Zhong et al. (2022) augment LLMs' output by interpolating with kNN mod- ule: though none of them explicitly provide cita- tions to the retrieved sources. Other works prompt or fine-tune LLMs to "retrieve on-the-fly" (Parisi et al., 2022; Schick et al., 2023; Shuster et al., 2022; Jiang et al., 2023; Yao et al., 2023; Press et al., 2022), which offers flexibility of when and what to search. Gao et al. (2023); He et al. (2022) propose to first generate text without accessing external doc- uments and then retrieve relevant documents and revise the generation to be consistent.
Table 8: Human citation quality evaluation vs. ALCE
citation quality evaluation on ASQA.
Human scores
ALCE scores
Rec.
Prec.
Rec.
Prec.
CharGPT VANILLA
50.8
52.4
52.8
50.4
w/ RERANK
59.7
60.6
63.0
60.6
Vicuna-13B VANILLA
13.4
19.2
13.6
18.1
Table 9: Human citation quality evaluation ALCE citation quality evaluation on ELI5.
to output fluent answers that are related to the ques- tion, despite differences in factual correctness.
Our automatic evaluation of citation quality strongly correlates with human judgements. As shown in Table 8 (ASQA) and Table 9 (ELI5), the relative rankings induced by human and our auto- matic metries are consistent. The absolute citation scores from human and ALCE are very close except for RERANK (which uses the automated citation recall for reranking). This suggests that an im- provement on ALCE citation metrics translates to improvement on human preferences. Furthermore, the Cohen's 's kappa coefficient between human and ALCE suggests substantial agreement for citation recall (0.698) and moderate agreement for citation precision (0.525). We also show in $G.5 that our automatic evaluation achieves high accuracy when treating human annotations as gold labels (85.1% for citation recall and 77.6% for citation precision).
Among previous explorations. Nakano et al. (2021); Menick et al. (2022) are the closest to our setting, where LLMs are trained to answer ques- tions while providing citations. However, they do not explore retrieval strategies and simply use com- mercial search engines, which are not reproducible, and their models and training data are closed- source. To the best of our knowledge, we are the first to implement end-to-end systems that retrieve, synthesize, and cite documents with LLMs.
8 Conclusion
We propose ALCE, the first automatic benchmark for evaluating LLM generations with citations. We deploy automatic metrics to measure fluency, cor- rectness, and citation quality, and verify their effi- cacy via human evaluation. We explore a variety of strategies for incorporating citations in LLMs and demonstrate that current systems have considerable room for improvement on ALCE.
7 Related Work
Evaluating citations. Generating text with ci- tations is closely related to attribution. Rashkin al. (2023) define the "attributable identified sources" (AIS) score to measure how faithful a gen- erated text to its sources. Bohnet al. (2022) ap- ply AIS scores on a single-document short-answer QA dataset. Honovich et al. (2022); Yue et al. (2023) study automatic evaluations for the AIS score. A concurrent work (Liu et al., 2023) conduct human evaluation commercial generative search engines examine their citation qualities.
Our experiments highlight a number of promis- ing research directions, including (1) enhancing re- trieval and refining retrieval integrations in LLMs, (2) developing long-context LLMs, and (3) advanc- ing LLMs' ability to synthesize multiple sources. What's even more intriguing that these research proposals extend beyond the ALCE setup (for ex- ample, long-context LLMs have numerous exciting applications), and ALCE can serve as a valuable testbed for their development.
Scientific citation text generation (Funkquist et al., 2022) is a related task to ALCE where the

---

Limitations
Lespiau, Bogdan Damoe, Aidan Clark, Diego De Las Casas, Aurelia Guy. Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Mag- giore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals. Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens. In Inter- national Conference on Machine Learning (ICML). volume 162. pages 2206-2240.
Our evaluation still has room for improvement: (1) MAUVE is found to be sensitive to output length and may provide unstable results; (2) for the ELI5's correctness evaluation, the automatically generated claims may not cover all possible answers due to the open-ended nature of the questions; (3) ci- tation quality evaluation is limited by the accuracy of the NLI model; for citation precision, the NLI model cannot detect the case of "partially support" and thus Icads to a lower citation precision score than the human evaluation.
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher Manning. 2013. A large annotated corpus for learning natural language inference. In Empirical Methods in Natural Language Processing (EMNLP).
Although we believe our curated datasets closely resemble the distribution of real-world user ques- tions, we acknowledge that they do not cover more challenging scenarios, such multi-hop reasoning, math reasoning. and code completion.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah. Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam. Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Process- ing Systems (NeurIPS).
In our experiments, we focus on prompting LLMs without updating their model weights. Train- ing a model directly to incorporate citations re- mains challenging due to the lack of supervised data. However, we observe that certain human- instruction datasets contain examples similar to our task setup. We leave the exploration of training LLMs generate citations for future work.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Angela Fan, Yacine Jernite, Eathn Perez, David Grang- ier, Jason Weston, and Michael Auli. 2019. ELI5; Long form question answering. In Association for Computational Linguistics (ACL), pages 3558-3567.
Acknowledgments
We appreciate the helpful feedback from the mem- bers of the Princeton NLP group. We thank Alexan- der Wettig, Nelson Liu, Tianyi Zhang, Yu Meng, Sadhika Malladi, Yangsibo Huang, Zhiyuan Zeng, and Dan Friedman for the valuable discussion. We thank Surge Al (especially Anna Folinsky and Ed- win Chen) for their support with the human eval- uation. Tianyu Gao supported by an IBM PhD Fellowship. This research is supported by an NSF CAREER award (II--223929),, a Sloan Research Fellowship, and Microsoft Azure credits through the "Accelerate Foundation Models Academic Re- search" Initiative.
Martin Funkquist, Ilia Kuznetsov, Yufang Hou, and Iryna Gurevych, 2022. CiteBench: A benchmark for Scientific Citation Text Generation. arXiv preprint wrXiv:2212.09577.
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vin- cent Y Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2023. RARR: Researching and revising what language models say, using language models. In Association for Computational Linguistics (ACL).
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-We Wei Chang. 2020. REALM: Retrieval- augmented language model pre-training. In Interna- tional Conference on Machine Learning (ICML).
Hangfeng He, Hongming Zhang, and Dan Roth. 2022. Rethinking with retrieval: Faithful large language model inference. arXiv preprint arXiv:2301 00303.
References
Bernd Bohnet, Vinh Q Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, et al. 2022. Attributed question answering: Evaluation and modeling for attributed large language models. arXiv preprint 22120020000
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text de- generation. International Conference on Learning Representations (ICLR).
Or Honovich, Roce Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom. Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE; Re-evaluating factual
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, -aanttap

---

consisteney evaluation. In North American Chapter of the Association for Computational Linguisties: Hu- man Language Technologies (NAACI-HILT), pages 3905-3920.
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte. Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens. Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richard Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Amav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. 2023. Openassistant conversations de- mocratizing large language model alignment. arXiv preprint arXiv:2304.072:
Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages 874-880, Online. Association for Computa- tional Linguistics.
Kenton Lee, Ming-Wee Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering, In Association for Com- putational Linguistics (ACL), pages
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Atlas: Few-shot learning with re- trieval augmented language models, arXiv preprint arXiv:2208.03299.
Nelson F Liu, Tianyi Zhang, and Percy Liang, 2023. Evaluating verifiability in generative search engines. arXiv preprial arXiv:2304.09844
Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese Susannah Young. Lucy Campbell- Gillingham, Geoffrey Irving, et al. 2022. Teaching language models to support answers with verified quotes. arXiv preprint 2rXiv:2203.1114:
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
Sewon Min, Julian Michael. Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: Answering am- biguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Processing (EMNLP), pages 5783- 5797, Online. Association for Computational Lin- guistics.
Zhengbao Jiang. Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-YY Yu, Yiming Yang. Jamie Callan, and Graham Neubig. 2023. Ac- tive retrieval augmented generation. arXiv preprint arXiv:2305.0695.
Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. 2023. WiCE: Real-World En- tailment for Claims in Wikipedia. arXiv preprint arXiv:2303.01432.
Retichrro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu. Long Ouyang. Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. WebGPT: Browser-assisted question- answering with human feedback. arXiv preprint wrXvv:212.009332
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5769-6781. Online. Association for Computational I. Linguistics.
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez. Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wes Wei Chang, and Yinfei Yang. 2022. Large dual encoders are generalizable retrievers. In Empirical Methods in Natural Languuge Processing (EMNLP), pages 9955.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations (ICIR).
OpenAL. 2023. GPT-4 Technical Report.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural In- formation Processing Systems (NeurIPS), 35:27730- 27744.
Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. Scitail: A textual entailment dataset from science question answering. In Conference on Artificial In- telligence (AAAI), volume 32.
Aaron Parisi, Yao Zhao, and Noah Fiedel. 2022. TALM: Tool augmented language models. arXiv preprint vrXiv:2205112255
Kalpesh Krishna, Aurko Roy, and Mohit lyyer. 2021. Hurdles to progress in long-form question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 9940-4957, Online, Association for Computa- tional Linguistics.
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thoree, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian

---

Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2523-2544, Online. Association for Computational Linguistics.
Freda Shi, Xinyun Chen. Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In Interna- tional Conference on Machine Learning (ICML).
Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, Stephen Roller, Arthur Szlam, and Jason Weston. 2022. Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 373-393.
Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izac- ard, Patrick Lewis. Barlas Oguz. Edouard Grave, Wen-tau Yih, al. 2021. The Web Is Your Oyster- Knowledge- Intensvve NLP against a Very Large Web Corpus. arXiv preprint axXiv2212.0990
Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers. John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. MAUVE: Measuring the gap be- tween neural text and human text using divergence frontiers. In Advances in Neural Information Pro- cessing Systems.
Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming- Wei Chang. 2022. ASQA: Factoid questions meet long-form answers. Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 8273-8288, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Mcasuring and narrowing the compositionality gap in language models. arXiv preprint arX0v:2210.03350.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-followinn LLaMA model.
Colin Raffll, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with unified text-to-text Trans- former. The Journal of Machine Learning Research (JMLR), 21(140).
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In North American Chapter of the Association for Computational Linguisties: Human Language Technologies (NAACL-HLT)) pages 809-819.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Empirical Methods in Natural Language Processing (EMNLP), pages 2383-2392
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. ILLMA: Open and Effi- cient Foundation Language Models. arXiv preprint 132.1.2302.1397.
Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Ture, and David Reitter. 2023. Measuring Attribution in Natural Lan- guage Generation Models. Computational Linguis- tics, pages 1-64.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura. Maric-Anne Lachaux, Thibaut Lavril, Jenya Lee, Di- ana Liskovich, Yinghai Lu, Yuning Mao, Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subrama- nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay- lor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan. Melanie Kambadur, Sharan Narang, Aurelien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models.
Samuel Joseph Amouyal Ohad Rubin, Ori Yoran, Tomer Wolfson, Jonathan Herzig. and Jonathan Berant. 2022. QAMPARI: An Open-domain Question An- swering Benchmark for Questions with Many An- swers from Multiple Paragraphs. arXiv preprint arXiv:2205.1 12665.
Timo Schick, Jane Dwivedi-YY Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.0477
Tal Schuster, Adam Fisch. and Regina Barzilay. 2021. Get your vitamin C! robust fact verification with con- trastive evidence. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 624 643.

---

Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the fac- tual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Compu- lational Linguistics, pages 5008-5020, Online. Asso- ciation for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies (NAACI-HHLT).
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting language models. In International Conference on Learning Representations (ICLR).
Xiang Yue, Boshi Wang, Kai Zhang. Ziru Chen, Yu Su, and Huan Sun. 2023. Automatic evaluation of at- tribution by large language models. arXiv preprint arXiv:2305.06311.
Shiyue Zhang and Mohit Bansal. 2021. Finding bal- anced degree of automation summary evaluation. In Empirical Methods in Natural Language Process- ing (EMNLP), pages 6617-6632.
Yuan Zhang, Jason Baldridge, and Luheng He. 2019. PAWS: Paraphrase adversaries from word scrambling. In North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (NAACI-IIL.T), pages 1298-1308.
Zexuan Zhong, Tao Lei, and Danqi Chen. 2022. Train- ing language models with memory augmentation. In Empirical Methods in Natural Language Processing (EMNLP), pages 5657-5673.

---

A Generating Claims for EL15
NLI model accurately predicts the entailment of sub-claims. We further analyze our sub-claim evaluation metries by checking the error rate of the final prediction of the NLI model. To this end, we first manually annotate the entailment scores between 40 outputs and their sub-claims (in total of 120 pairs; these are the same questions from the previous analysis). We then use the NLI model to obtain the entailment scores for the output and sub-claims. Using the human annotations as the ground truth label, we found that the NLI model achieved an accuracy of 80.0%.
We elect not to use ROUGE-L as our main cor- rectness metrics since it does not account for the different ways of expressing the same answer and it can be easily gamed (Krishna et al., 2021). We fur- ther illustrate this issue in Table 10. A system can easily achieve high ROUGE-L score by retrieving and returning the top passage from a BM25 index. However, the claims evaluation metric does not reward this approach since the output often lacks different aspects of the answers.
ROUGE-L
Claim recall
B Dataset Statistics
ChatGPT VANILLA
20.6
12.0
For ASQA, human answers have average length of 65 words. For QAMPARI, each question has on average 13 answers. For EL15, human answers have an average length of 131 words.
ChatGPT ORACLE
21.2
21.3
LLaMa-13B VANILLA
16.2
3.9
Top-1 passage
19.1
3.0
Table 10: Comparison between ROUGE-L and claim recall scores ELI5.
C Implementation Details
Instead, we leverage the original answers to gen- erate sub-claims and use them to serve as an esti- mate of the different aspects of the answers that we expect the model to cover. This approach is inspired by works in summarization evaluation and claim verification (Zhang and Bansal, 2021; Kamoi al., 2023; Wang et al., 2020).
NLI model. We use the version of TRUE model from https: //ugg/nggaggccceeeeee hww.giigggacc.ccoooolleeeee co/goggle/ t5_xxl @eel__rue__ie true_nli_mix_e xture, which is trained on SNLI (Bowman al., 2015), MNLI (Williams et al., 2018), Fever (Thorne al., 2018), Sci- tail (Khot et al., 2018), PAWS (Zhang et al., 2019), and VitaminC (Schuster et al., 2021). This model uses the following prompt: "premise: [PREMISE} hypothesis: ()" and outputs "1" the premise entails the hypothesis. We format each passage (when used as premise) by the format of "Title: (TITLE)M(TEXTT" and concatenate all passages with "\n" separator.
Specifically, we use text-davinci-ee 0003 to gen- erate the sub-claims. We first manually annotate three question and answer pairs from the original EL15 training set with sub-claims each. Then, we prompt text-davinci-000 with these pairs as demonstrations. The full prompt with an example is shown in Table 22.
MAUVE. When rumning MAUVE, we concatenate the question and the model output (or human an- swer) by space. We truncate both the references and the model generations to 100 words, as we found MAUVE results are unstable beyond this length for EL.5 (this is due to that ELI5 has a lot of extremely long human answers).
InstructGPT generates coherent and faithful sub-claims. To ensure that the generated sub- manually inspect
claims are of good quality, a random sample of 40 answers and their generated sub-claims (totaling to 120 sub-claims). For each sub-claim, we assign a score of it is relevant to the question and faithful to the facts presented in the ground truth, and 0 otherwise. We found that 112 out of the 120 (93.33%) sub-claims received a score of 1, meaning that our generated sub-claims are of high quality and faithful to the ground truth. Furthermore, the average number of words in the generated sub-claims 14 words, and they are typically just one sentence long. This is aligned with the intent behind the metric-to capture short factual claims made by the original answer.
Exact match for ASQA and QAMPARI. Both ASQA and QAMPARI provide aliases for their short answers. We normalize the response and the short answers similarly to Rajpurkar et al. (2016) and report the score with the best-matching aliases. For ASQA, Stelmakh et al. (2022) also propose a QA-based evaluation which we found to be not as stable, and thus we do not report in our paper.
Output truncation. Before evaluation, we trun-

---

cate model output by new lines, as non-instruction- tuned models may generate more content after new lines that are irrelevant.
[2] will be counted as "irrelevant" while it should not be penalized. Liu et al. (2023) conduct hu- man evaluation on citation precision in a different way: For each citation, they ask annotators to judge whether the citation (1) fully support, (2) partially support, or (3) does not support s, One citation Cijj is precise if (a) Crjj fully supports s; (b) C, fully supports 8,, Cij partially supports 8;, and no E C alone fully supports 8;- This evaluation solved the comner case we mentioned in the main paper (one citation partially supports the claim but identi- fied as "irrelevant)). However, it is challenging to conduct such evaluation automatically, as there is no existing model that can judge whether cita- tion "partially" supports claim. We also explore prompting ChatGPT to conduct such task, which yields poor results. We defer it to future work collect supervised data train better that can detect "partial support".
INTERACT. Empirically, found that models tend to execute too many consecutive "check" ac- tions, so we force the model to always "output" after each "check". We limit the maximum number of passages to check as 3 to avoid exceeding the length limit. The full passages are removed from the context after one action to save context space. Table 27 provides an example for INTERACT.
Main experiments. For all experiments except each model three times with different seeds and each time sample two demonstrations from pool of four. We report the averaged scores for all experiments in the main paper and we report the standard deviations in Ap- pendix G.6.
ChatGPT RERANK,
Decoding methods. Based on preliminary experi- ments we choose the following decoding methods: For ChatGPT and GPT-4, we use sampling with temperature 0.5; for all open-source models, we use Nucleus sampling (Holtzman et al., 2020) and set top_p 0.95.
F Human Evaluation
We employ Surge Al (https://ww/ surgehq. ai/) for our human evaluation. The average pay to workers is 20 USD per hour. We randomly sample 100 examples from ASQA and ELI5 and annotate outputs of selected models: ChatGPT VANILLA, ChatGPT RERANK, and Vicuna- 13B VANILLA.
D ALCE Catches Shortcut Cases
Fluency (MAUVE)
Correct.
Citation
F1 Utility
(EM Rec.)
Rec.
Prec.
To check if the model output is useful down- stream users, we measure the utility of the response
ChatGPT
66.6
40.4
73.6
63.0
Top-1 passage
20.8
99.4
99.4
S. We first show the query g and model response S to the worker and ask them to rate their agree- ment with the statement "The response a helpful and informative answer to the query" on a Likert scale of 1-5, corresponding to Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree.
First 2 sents
67.2
18.9
98.7
98.7
Table 11: ASQA cheating cases. "ChatGPT": the Chat- GPT VANILLA model with GTR-retrieved top-5 pas- sages. "Top-1 passage": the top-1 retrieved passage as the response. "First 2 sents": use the first sentences the 1 retrieved passage.
F.2 Citation Recall
Table 11 demonstrates the experiments show that ALCE is robust to shorteut cases. Using the top-1 passages or first two sentences of the top-1 passages induces almost perfect citation quality, but fluency and correctness are dramatically lower.
The annotators are shown the question the state- ment 8;, and all of its citations C. and they rate if the joint set of citations fully support the statement (recall=1) or if they do not support all the claims (recall=0). We calculate the overall recall score for the generation by taking an average of all the statements' recall scores.
E Citation Recall Discussion
Our citation precision evaluation cannot detect citation that partially supports the statement and hence will falsely penalize it. Consider a state- ment $3 and its citations [2][4][5]: if [2] entails partial information of sy that [4][5] also entails,
F.3 Citation Precision
We show the question g and a pair of statement Sj and one of its citation C.j E C, to the annotator. We ask the annotator the citation fully supports,

---

R@1
R@3
R@5
R@20
R@
R@1
R@3
R@5
R@20
R@
100
DPR
29.6
44.5
$1.5
64.6
74.1
BM25
3.0
6.6
9.6
19.3
31.8
GTR
35.1
50.7
56.8
70.3
78.4
Oracle
25.3
29.7
31.8
Oracle
63.8
72.8
78.4
Table 14: Retrieval results ELI5 (claim recall).
Table Retrieval results for ASQA (EM recall).
Fluency
Correct.
Citation
R@
R@3
R
R@20
@ 100
(MAUVE)
(EM Rec.)
Rec.
Prec.
DPR
6.7
13.5
17.6
30.4
47.6
ChatGPT (VANILLA, 5-doc)
GTR
14.6
24.7
31.6
49.7
65.6
Oracle
44.3
58.7
63.6
Short instruction
64.1
39.5
69.6
73.2
Full instruction
66.6
40.4
73.6
72.5
Table 13: Retrieval results for QAMPARI (recall-5).
Table 15: Effect of different instructions ASQA.
partially supports, or does not support the factual claims in 8, Citation Ctj has citation precision of if 9t has recall of and Cij fully partially take average of preci- sion scores of all citations in the statement to obtain the citation precision score.
G.3 Effect of Demonstrations
Table 16 shows results on effect of different num- bers of demonstrations. We see that numbers of demonstrations do not affect ChatGPT's correct- ness but using at least one demonstration ensures high citation recall, For the original LLaMA model. Table 16 shows the trend that more demonstrations lead to better performance.
supports 8t. Finally,
G More Experiments
G.1 Retrieval Analysis
G.4 Fine-tuned Models
Oracle. Since the original datasets do not contain gold passages at the same granularity level as our setting (100-word passages), approximate gold passages by rumning the following algorithm on the top-100 retrieved passages. We first calculate the recall score for each passage. Then, we sort the passages using their recall score and take the top passages as our initial oracle set. Finally, we iterate through all passages that were not initially in the or- acle set and try to replace the passages in the oracle set in greedy fashion: we calculate the change in the recall score of the oracle set for every possible replacement and proceed with the replacement that results in the largest recall improvement. The of oracle passages were able to match the recall scores of the t0p-100 retrieved passages.
To better understand the differences between fine- tuned models and prompted large language mod- cls, train statc-of-the-arr question answer- ing model, Fusion-in-Decoder (FiD; Izacard and Grave (2021)), and evaluate in conjunction with POSTCITE. Due the lack of training data with citation annotation, we first train a T5-base FiD model for 5 epochs on the ASQA training set with a batch size of 64 and a learning rate of lc-4, Dur- ing evaluation, we use POSTCITE to add citations to the output. We also use h 5 passages during both training and evaluation of the FiD model.
Then, we evaluate this model on both ASQA (in-domain) and ELI5 (out-of-domain), and the re- sults can be found in Tables 17 and 18. Note that this is not a direct comparison, as ALCE assumes only evaluation data available and uses only few- shot data for prompting. As the results show, the FiD baseline still significantly lags behind prompt- ing ChatGPT in both correctness and citation qual- ity (even though it is trained on 4000+ examples).
Detailed retrieval results. We show detailed re- trieval results in Tables 12, 13, and 14.
G.2 Effect of Instructions
Table 15 shows results of using full instruction (Table 23) and a short version of the instruction (Table 24). We see that the full version induces stronger correctness and citation recall, while the two instructions lead to similar citation precision.
When tested on another dataset (ELIS), FiD per- forms even worse, showing that it is challenging to solve the problem by fine-tuning a small pre- trained model.

---

Fluency
Correct
Citation
(MAUVB)
(EM)
Rec.
Prec.
ChatGPT (VANILLA)
#demo = 0
74.5
41.9
69.3
73.4
#demo 68.9
39.8
73.2
#demmo 2
66.6
40.4
73.6
72.5
Table 16: Different demonstrations on ASQA.
Fluency
Correct.
Citation
(MAUVE)
(EM Rec.)
Rec.
Prec.
CharGPT
66.6
40.4
73.6
72.5
FID + POSTCITE
75.8
28.4
58.1
58.0
Table 17: Comparison of Fusion-in in in-Decoder with Chat- GPT on ASQA. Both models use top-5 GTR passages.
G.5 More Human Evaluation
We evaluate the accuracy of our automatic metrics by treating the human annotations as gold labels. For citation recall, ALCE achieves an accuracy of 85.1%; for citation precision, ALCE has an accu- racy of 77.6%. Regarding detecting insufficient citations, ALCE has recall of 82.3% and a pre- cision of 84.2%; regarding detecting "irrelevant" citations, ALCE has recall of 75.6% and pre- cision of 66.1 6..1% 11-1% is effective in detecting "irrelevant" citations, but due to the limitation of the NLI model (cannot detect "partial support"), has a relatively high false positive rate.
Fluency
Correct.
Citation
(MAUVL)
(Claim)
Rec.
Prec.
CharGPT FID POSTCITE
57.2
12.0
51.1
50.0
25.2
39.3
39.3
Table 18: Comparison of Pusion-in-Decoder with Chat- GPT ELI5. Both models use top-5 GTR passages.
G.6 Main Results
We show full results of our experiments along with the standard deviation in Tables 19, 20, and 21. We repeat all experiments with three different random seeds. However, for ChatGPT RERANK, we use only one seeded run since each run repeats the generation step four times, and more experiments would incur significant costs.
Prompts
We show detailed prompts used in our paper in Tables 23, 24, 25, 26, 27, 28, and 29,
I Examples
Tables 30 and 31 we show some examples of questions and model generated outputs.

---

Fluency
Correct.
Citation
(MAUVE)
(EM Rec.)
Rec.
Prec.
ROUGE-L.
Length
ChatGPT
VANILLA (5-psg) w/ RERANK SUMM (10-psg)
66.8 (2:0)
40.4 (0.6)
73.6 (1.1)
72.5
37.0
40.0 (3.1)
77.0 (0.0)
40.2 (0.0)
84.8 (0.0)
81.6 (0.0)
36.9 (0.0)
40.8 (0.0)
70.0 (1.2)
43.3 (0.8)
68.8 (0.6)
61.8 (1.1)
36.9 (0.2)
49.8
w/ INTERACT SNIPPET (10-psg)
69.0 (2.7)
39.1 (0.6)
73.4 (0.2)
66.5 (4.9)
35.7 (0.2)
34.0 (0.9)
69.8 (2.6)
41.4 (0.6)
65.3 (0.6)
57.4 (0.9)
36.4 (0.4)
43.0 (3.5)
INLINESEARCH CLOSEDBOOK ORACLE(S-PSg)
58.7 (1.3)
32.4 (0.6)
58.3 (1.3)
58.3 (1.3)
58.2 (1.1)
23.7 (1.3)
52.7 (4.9)
38.2 (0.1)
26.7 (1-1)
26.7 (1.1)
37.1 (0.3)
61.1 (4.5)
64.4 (0.6)
48.9 (1:2)
74.5 (0.6)
72.7 (1.0)
38.2 (1.0)
37.4 (3.0)
ChatGPT-16K
VANILLA (5-psg)
60.3 (-)
36.1 (-)
76.2 (-)
76.5 (-)
36.2 (-)
24.7 (-)
VANILLA (10-psg)
56.3 (-)
36.7 (-)
75.3 (-)
75.0 (-)
35.6 (-)
23.5 (-)
VANILLA (20-psg)
56.7 (-)
36.1 (-)
73.7 (-)
73.5 (-)
35.5 (-)
23.1 (-)
GPT-4
VANILLA (5-psg)
67.1 (-)
41.3 (-)
68.5 (-)
75.6 (-)
39.2
31.8
VANILLA (10-psg)
71.5 (
43.1 (-)
72,0
75.5
39.7
33.8
VANILLA (20-psg)
64.9 (-)
44.4 (-)
73.0
76.5
40.1
34.3
Open-source
LLaMA-7B VANILLA (3-psg)
69.8 (2.0)
22.6 (0.9)
6.2 (2.7)
9.2 (2.8)
29.1 (0.2)
61.3 (14.3)
Alpaca-7B VANILLA (3-psg)
84.2 (2.7)
32.1 (1.7)
12.3 (7.2)
14.1 (7.0)
33.1 (0.8)
51.7 (12.8)
Vicuna-7B 7B VANILLA (3-psg)
82,9 (5.0)
34.6 (0.7)
40.3 (0.5)
42.6 (1.0)
35.9 (0.7)
48.9 (6.6)
LLaMA-13B VANILLA (3-psg) w/ RERANK
68.4 (6.4)
26,9 (0.4)
10.6 (4.7)
15.4 (5.2)
29.8 (0.5)
67.1 (19.1)
60.9 (14.5)
(2.6)
28.1 (9.3)
37.0 (7.2)
27.9
50.5 (14.3)
LLaMA-13B SUMM (10-psg)
76.8 (4.7)
33.3 (0.7)
19.6
23.7
32.1 (0.3)
54.4 (1.3)
LLaMA-13B SNIPPET (10-psg)
72.0 (0.8)
31.3 (1.1)
18.2 (3-1)
21.1
30.8 (0:4)
50.5 (4.5)
LLaMA-13B ORACLE (3-psg)
69.5 (13.4)
34.3 (0.9)
10.8 (4.9)
15.8
30.6 (v.1)
67.3 (17.9)
Vicuna-13 VANILLA (3-psg)
82.6 (9.4)
31.9 (3.9)
51.1 (1.4)
50.1 (2.5)
34.9 (1.3)
39.1 (6.6)
w/ RERANK
73.5 (2.1)
32.9 (1:38
71.9 (1.9)
65.4 (1.5)
34.6 (0.3)
35.7 (4:2)
Vicuna-I 13B SUMM (10-psg)
67.7 (0.3)
43.2 (0.1)
52.7 (2.6)
50.0 (2.1)
36.7 (0.2)
66.0
Vicuna-133 13B SNIPPET (10-psg)
81.4 (3.0)
42.1 (1.2)
53.4 (1.9)
48.7
36.9 (0.4)
61.2
Vicuna- 133 ORACLE (3-psg)
72.9 (3.6)
42.5 (1.6)
$2.2 (0.8)
50.7
36.5 (0.9)
38.7
LLaMA-33B VANILLA (3-psg)
83.7 (5.4)
31,0 (0.8)
(5.3)
(5.3)
32.3 (0.6)
44.1 (9.3)
w/ RERANK
(3.0)
31.3 (1.1)
(6.4)
44.7 (5.5)
32.5 (0.9)
39.4 (8.0)
LLaMA-33B SUMM (10-psg)
72.0 (3-0)
33.1 (1.9)
34.7 (5.8)
35.2 (6.0)
31.1 (0.8)
43.7 (5-0)
LLaMA-33B SNIPPET (10-psg)
70.8 (3.1)
30.9 (1.4)
(4.20
31.5 (5.3)
30.1 (0.7)
42.8 (3.6)
LLaMA-33B ORACLE (3-psg)
82.6 (7.1)
39.3 (2.9)
20.2 (6.2)
23.9 (6.3)
33.1 (0.9)
42.0 (9.3)
Oasst-33B VANILLA (3-psg)
82.9 (2.7)
34,8 (1.5)
36,2 (1.7)
38.3 (2.7)
35.5 (0.7)
45.2 (6.3)
w/ RERANK
83.2 (2.4)
35.1 (1.4)
66.7 (0.2)
64.3 (1.9)
35.0 (0.6)
41.8 (c.0)
Oasst-33B SUMM (10-psg)
74.3 (4.6)
40.9 (1.1)
45.5 (1.9)
44.0 (2.9)
35.8 (0.6)
54.3 (4.8)
Oasst-33B SNIPPET (10-psg)
79.3 (1.0)
40.1 (0.9)
45.0 (1.3)
43.3 (2.2)
35.8 (0.2)
50.9 (4.1)
Oasst-33B ORACLE (3-psg)
85.1 (2.8)
44.3 (2.4)
37.0 (1.0)
39.6 (1.5)
36.5 (1.1)
44.2 (5.8)
LLaMA-2-7B-CII VANILLA (5-psg)
80.1 (6.6)
33.9 (2.1)
50.9 (4.6)
47.5 (3.7)
35.1 (0.9)
42.3 (10.1)
LLaMA-2- 13B-Chat VANILLA (5-psg)
72.4 (6.3)
35.2 (1.2)
38.4 (5.0)
39.4 (4.8)
35.8 (0.9)
38.0 (6.4)
LLaMA-2-706 MAMMA-2-7BB-Caa -hB- VANILLA (5-psg)
88.3 (1.1)
41.5 (0.8)
62.9 (1.4)
61.3 (2.1)
37.1 (0.4)
$2.9 (9.3)
Table 19: ASQA full results.

---

Correctness
Citation
Rec.-5
Prec.
Rec.
Prec.
Num Pred.
ChatGPT
VANILLA (5-psg) w/ RERANK
20.8 (2.2)
20.8 (0.2)
(u.7)
(0.7)
5.0 (0.5)
22.8 (0.0)
21.4 (0.0)
21.2 (0.0)
21.4 (0.0)
5.4 (0.0)
SUMM psg)
23.6 (0.9)
21.2 (0.5)
23.6 (0.7)
25.7 (0.8)
6.7 (0.4)
SNIPPET (10-psg)
24.5 (1.4)
21.5 (1.8)
22.9 (1.6)
(0.4)
7.2 (0.9)
w/ INTERACT INAINES CLOSEDBOOK
21.9 (0.9)
23.0 (0.4)
(1.2)
(0:9)
6.7 (0.4)
17.2 (1.1)
20.4 (0.8)
14.9 (0.8)
(0.8)
6.7 (0:2)
32.9 (1.1)
19.8 (1.6)
10.0 (0.4)
(0.4)
17.0 (2.9)
ORACLE
37.0 (3.1)
36.9 (0.6)
24.1 (1.2)
24.6 (1.3)
5.3 (0.6)
ChatGPT-16K
VANILLA (5-psg)
21.1 (-)
22.0 (-)
20.7 (-)
21.2 (-)
4.9 (-)
VANILLA (10-psg)
23.4
21.9 (-)
21.6 (-)
22.0
5.7 (-)
VANILLA (20-psg)
26.4
21.1 (-)
19.4 (-)
19.7
7.6
GPT-
VANILLA (5-psg)
22.2 (-)
25.0 (-)
25.9 (-)
27.0 (-)
4.4 (-)
VANILLA (10-psg)
26.8
25.1 (-)
26.2 (-)
27.2 (-)
5.7
VANILLA (20-psg)
29.6 (-)
26.2 (-)
27.4 (-)
28.5 (-)
6.8
Open-source
LLaMA-7B VANILLA (3-psg)
7.8 (3.4)
7.4 (2:7)
5.1 (0.5)
5.7 (0.8)
5.7 (0.6)
Alpuca-7B VANILLA (3-psg)
9.4 (3.7)
9.5 (a.6)
6.4 (o.0)
6.8 (a.5)
5.1 (o.1)
Vicuna-7B VANILLA (3-psg)
11.3 (1.4)
13.3 (2.3)
10.1 (0.6)
10.9 (0.5)
3.9 (0.3)
LLaMA-13B VANILLA (3-psg)
9.7 (3.6)
9.1 (3.1)
6.7 (0.9)
7.1 (a.9)
5.9 (0.6)
w/ RERANK
10.0 (3.3)
10.7 (3.3)
9.9 (1.2)
10.2 (1.1)
5.4 (0.5)
ILLMA-13B SUMM (10-psg)
14.8 (2.6)
12.6 (1.6)
7.4 (0.6)
8.0 (a.6)
8.1 (0.9)
LLAMA-I3B SNIPPET (10-psg)
17.7 (1.4)
15.7 (0.9)
8.8 (0.7)
9.9 (o.6)
8.2 (0.4)
LLaMA-13B ORACLE (3-psg)
16.8 (8.6)
15.4 (5.6)
7.7 (1.0)
8.3 (1.13
5.7 (0.7)
Vicuna- 13B VANILLA (3-psg)
14.0 (0.6)
15.9 (1-7)
12.5 (0.8)
13.4 (0.7)
4.7 (0.3)
w/ RERANK
13.0 (0.7)
17.2 (2:2)
17.3 (0.8)
17.7 (0.6)
4.4 (0.3)
Vicuna-13B SUMM (10-psg)
21.1 (1.4)
17.1 (0.3)
15.7 (0.2)
17.8 (0.1)
6.9 (0.7)
Vicuna-138 SNIPPET (10-psg)
21.9 (0.8)
18.2 (0.3)
16.8 (0.3)
19.7 (0.6)
7.5 (0.4)
Vicuna-13B ORACLE (3-psg)
25.9 (1.6)
28.4 (2.6)
15.8 (1.4)
16.8 (1.4)
4.9 (0.0)
LLaMA-33B VANILLA (3-psg)
14.7 (3.3)
12.0 (2:2)
7.9 (0.7)
8.3
7.2 (0.7)
w/ RERANK
14.0 (3.4)
13.9 (2.6)
10.7 (0.6)
11.1 (0.3)
6.4 (0.7)
ILLMAA33B SUMM (10-psg)
19.0 (1.9)
14.8 (0.8)
12.5 (0.2)
15.0 (0.3)
7.6 (0.6)
ILLMA-33B SNIPPET (10-psg)
19.6 (1.1)
15.7 (o.1)
12.8 (1.1)
15.2 (1.2)
7.8 (0.5)
LLaMA-33B ORACLE (3-psg)
23.9 (0.9)
20.3 (5.2)
9.8 (1.2)
10.4 (1.2)
6.8 (0.9)
Oasst-33B VANILLA (3-psg)
15.5 (1.5)
14.9 (1-4)
9.0 (1.6)
10.1 (1.8)
5,6 (0.3)
w/ RERANK
14.1 (1.1)
15.8 (1.0)
15.0 15.0 (1.6)
15.9 (1.6)
4.7 (0.3)
Oasst-33B SUMM (10-psg)
21.0 (0.6)
17.5 (1.0)
12.9 (1.2)
16.6
7.1 (0.4)
Oasst-33B SNIPPET (10-psg)
22.0 (a.4)
17.4 (0.3)
13.6 (1.7)
17.7 (1.6)
7.5 (0.1)
Oasst-33B ORACLE (3-psg)
26.9 (3.7)
26.0 (3.3)
11.7 (1.0)
12.9 (1:2)
5.6 (0.4)
LLaMA-2-7B-Cha. VANILLA (5-psg)
16.2 (1.3)
15.3 (1.6)
10.6 (0.9)
10.9 (1.0)
5.5 (0.0)
I.I.M-2-133B VANILLA (5-psg)
21.1 (0.9)
18.2 (0.0)
9.6 (1.5)
9.7 (1.5)
6.5 (0.3)
LLaMA-2-70B- VANILLA (5-psg)
21.8 (0.7)
18.4 (o.2)
15.1 (1.2)
15.6 (1.3)
7.1 (0.2)
Table 20: QAMPARI full results.

---

Fluency
Correct.
Citation
(MAUVE)
(Claim)
Rec.
Prec.
ROUGE-L
Length
ChatGPT
VANILLA (S-p5g) w/ RERANK SUMM (10-psg)
57.2 (1.0)
12.0 (0.6)
51.1 (4.2)
50.0
20.6 (v.2)
91.5 (6.6)
56.1 (0.0)
11.4 (0.0)
69.3 (a.0)
67.8 (0.0)
20.3 (0.0)
103.4 (0.0)
40.2 (1.2)
12.5 (0.2)
51.5 (1.1)
48.2 (2.0)
20.3 (0.1)
90.0 (6.0)
SNIPPET (10-psg)
62.9 (2.2)
14.3 (0.1)
50.4 (1.1)
45.0 (2.w)
21.0 (o.1)
100.0 (6.8)
w/ INTERACT INLINESEARCH CLOSEDBOOK
68.0 (5.8)
13.3 (0.6)
47.8 (3.3)
45.0 (3.1)
20.1 (0:2)
99.8 (6.1)
49.7 (4.6)
13.4 (1.1)
45.6 (2.6)
43.7 (3.9)
20.4 (0.3)
103.0 (18.1)
32.6 (1.1)
18.6 (0.6)
15.4 (0.3)
15,4 (0.3)
22.8 (0.1)
108.3 (8.9)
ORACLE (5-psg)
59.4 (4.1)
21.3 (0:2)
57.8 (3.7)
56.0 (3.8)
21.2 (0.3)
93.0 (7.8)
ChatGPT-16K
VANILLA (5-psg)
31.6
14.4
44.6
44.1
21.4
87.6
VANILLA (10-psg)
26.6
14.4 (-)
45.5
43.3
21.5
87.5
VANILLA (20-psg)
31.6
15.9 (-)
43.4
40.9
21.7
92.6 (-)
GPT-4
VANILLA (5-psg)
38.4 (-)
14.2 (-)
44.0 (-)
50.1 (-)
20.6 (-)
79.6 (-)
VANILLA (10-psg)
39,9 (-)
15.7 (-)
49.5 (-)
54.2 (-)
21.2 (-)
88.2 (-)
VANILLA (20-psg)
41.5 (-)
18.3 (-)
48.5 (-)
53.4
22.2 (-)
97.0 (-)
Open-source
LLaMA-7B VANILLA (3-psg)
28.6 (17.9)
1.6 (0.9)
1.2 (0.0)
2.7 (0.1)
12.2 (1:3)
46,9 (1.2)
Alpaca-7B VANILLA (3-psg)
45.9 (0.3)
9.2 (0.1)
4.5 (1.6)
5.2 (1.9)
18.8 (o.3)
67.1 (1.2)
Vicuna-7B VANILLA (3-psg)
43.2 (3.9)
10.0 (o.5)
12.6 (2.3)
16.3 (2.6)
19.1 (0.4)
68.7 (2.0)
LLaMA-13B VANILLA (3-psg)
50.0 (2.0)
3.9 (o.4)
3.1 (0.9)
5.3 (1.3)
16.1 (0.6)
63.3 (2.0)
w/ RERANK
46.7 (2.9)
4.3 (0.4)
9.7 (2.1)
15.0 (2.2)
16.1 (0.7)
63.0 (2.3)
LLaMA-13B SUMM (10-psg)
28.6 (1.9)
2.9 (o.1)
2.5 (0.8)
3.8 (0.9)
8.5 (0.3)
33.1 (0.6)
LLaMA-13B SNIPPET (10-psg)
48.4 (3.1)
5.7 (0.9)
5.8 (0.6)
7.6 (0.9)
15.1 (1.3)
60.2 (3.2)
LLaMA-13B ORACLE (3-psg)
49.5 (2.4)
6.4 (0.6)
3.7 (0.7)
6.5 (1.0)
16.8 (0.5)
64.5 (1.7)
Vicuna- 13B VANILLA (3-psg)
58.2 (25.1)
10,0 (0.3)
15.6 (2.2)
19.6 (2.0)
19.1 (0.3)
69.6 (0.6)
w/ RERANK
45.9 (4.3)
9.2 (0.0)
31.7 (2.9)
38.2 (1.6)
18.6 (0.5)
69.7 (1.0)
Vicuna- 13B SUMM (10-psg)
22.4 (3.0)
4.9 (0.1)
9.7 (1.3)
12.2 (1-2)
9.3 (0.4)
33,0 (9.7)
Vicuna-133 13B SNIPPET (10-psg)
48.1 (5.3)
11.2 (1.4)
27.2 (3.0)
27.9 (1.9)
18.4 (1.9)
76,8 (8.7)
Vicuna-13B ORACLE (3-psg)
41.6 (3.1)
17.1 (0.4)
20.2 (3.0)
26.5 (3.0)
20.0 (0.3)
72.0 (0.2)
LLaMA-33B VANILLA (3-psg)
58.8 (a.3)
6,2
9,3 (3.0)
12.1
16.9
60.0 (1.3)
w/ RERANK
65.9 (2.5)
6.0 (0.7)
22.5 (5.2)
26.1 (0.9)
17.5 (0.4)
61.0 (1.2)
LLaMA-33B SUMM (10-psg)
23.3 (2.0)
3.0 (0.2)
6.2 (0.6)
8.2 (0.7)
7.5 (0.4)
26.2 (2.3)
LLaMA-33B SNIPPET (10-psg)
53.2 (4.0)
7.4 (1.3)
13.7 (0.n)
15.1 (0.4)
14.4 (1.7)
53.3 (9.6)
LLaMA-33B ORACLE (3-psg)
63.7 (2.8)
11.4 (0.6)
11.9 (2.6)
15.4 (3.6)
17.9 (0.2)
61.7 (2.6)
Oasst-33B VANILLA (3-psg)
46.8 (7.6)
9.5 (0:2)
16.0 (2.5)
21.6 (3.5)
18.6 (0:3)
67.8 (1.5)
w/ RERANK
52.1 (6.1)
8.5
34.4 (2.9)
41.5 (2.5)
18.2 (0.3)
67.0 (1.5)
Oasst-33B SUMM (10-psg)
24.8 (2.8)
3.9
12.3 (0.2)
16.3 (0.3)
9.1 (0.3)
31.6 (1.5)
Oasst-33B SNIPPET (10-psg)
50.7 (4.0)
10.7 (1.2)
25.8 (3.a)
26.7 (2.3)
17.8 (1.8)
69.6 (8.6)
Oasst-33B ORACLE (3-psg)
50.7 (12.1)
15.8
20.8 (2.8)
28.0 (3.2)
19.4 (0.3)
70.3 (1.1)
LLaMA-2-7B-CHE VANILLA (5-psg)
27.8 (3.0)
10.9
19.8 (1.2)
15.0 (1.4)
20.5 (0.2)
87.8 (8.1)
LI 1---A---11---- 13B-Chat VANILLA (5-psg)
34.7 (1.5)
13.4 (0.4)
17.3 (1.3)
15.8 (1.4)
20.9 (0.2)
88.3 (6.3)
LLaMA-2-70B-4 VANILLA (5-psg)
38.6 (4.9)
12.8 (1.0)
38.3 (2.4)
37.9 (1.9)
21.3 (0.3)
110.8 (5.6)
Table 21: ELI5 full results.

---

Read the original question and passage, and generate 3 additional claims that are supported by the passage and answer the question.
Original question: What's 's the difference between Shia Sunni Islam?
Passage: The main difference between Shia and Sunni Muslim is related to ideological heritage and issues of leadership. This difference is first formed after the death of the Prophet Muhammad in 632 A.D. The ideological practice of the Sunni branch strictly follows Prophet Muhammad and his teachings, while the Shia branch follows Prophet Muhammad's son-in-law Ali. Nowadays, Sunni and Shia the major branches Islam.
Claim The major branches Islam Sunni and Shia. Claim Prophet Muhammad died 632 A.D. Sunni branch strictly follows Prophet Muhammad and his
Claim The ideological practice teachings.
Original question: what causes Bi-polar disorder?
Passage: Bipolar disorder is an emotional disorder that causes extreme mood swings between excitement and depression. The spectrum mood swing may span from days to months. We are still not certain of the exact factors that cause such disorder, but genetics is considered a major factor.
Claim 1: One sympton of Bi-polar disorder extreme mood swings between excitement depression.
Claim 2: Genetics could be one the major factors that causes Bi-polar disorder.
Claim 3: The mood swing from Bi Bi-polar disorder can last days to months.
Original question: How do we hear differences in sound besides volume and pitch?
Passage: Pitch refers to the frequency of soundwave, and volumn refers to the amplitude of the soundwave. Besides volumn and pitch, we can also tell the difference between sounds based on the tone of sound. For example, we can differentiate the sound of different instruments based on the tone of the sounds.
Claim 1: Volume of sound is the amplitude of the soundwave.
Claim 2: Pitch is the frequency of soundwave.
Claim 3: We can use the tone of the sounds to differentiate the sound of different instruments.
Original question: How behind us?
able to discern whether sound is coming from in front of
Passage: There multiple explanations for why we can localize sounds. explanation that sounds travelling to the corresponding side of one's ear will slightly louder. Another explanation that there is a slight difference in the hitting time one's left and right ear based the sound's 's direction. However, these explanation means that when a sound exactly front someone or exactly behind someone, he or she can not tell the difference.
Claim 1: We can localize sounds by recognizing that the sound travelling to the corresponding side of one's ear will be slightly louder.
Claim 2: We can also localize sounds by recognizing the difference in hitting time to one's left and right ear based on the sound's direction.
Claim cannot tell the difference between sound that is exactly in front of exactly behind us.
Table 22: Prompt used to generate the sub-claims for ELI5 questions. Blue text is model generation. Brown text is the ELI5 example that want to generate sub-claims for. We construct the prompt by manually writing the sub-claims for three questions from the training set.
Instruction: Write accurate, engaging, and concise answer for the given question using only the provided search results (some of which might be irrelevant) and cite them properly. Use unbiased and journalistic tone. Always cite for any factual claim. when citing several search results, []][2][3]. Cite least document and most three documents in each sentence. If multiple documents support the sentence, only cite minimum sufficient subset of the documents.
Table 23: Instruction for VANILLA.
Instruction: Write a high-quality answer for the given question using only the provided search results and cite them properly using [1][2][3].
Table 24: Short instruction for VANILLA.

---

Sunmarize the following document within 50 words with the question of interest "[QUESTION)"
Return "irrelevant" if the document is irrelevant to the question. Try to keep all the important dates, numbers, and names.
Title: (TITLE)
Text: (TEXT)
Sumnary:
Table 25; Prompts SUMM.
Given the follow passage and the question "[QUESTION)", extract useful span from the passage that
question. Resolve all the coreference issues make the extracted span understandable standalone. If passage is not helpful for answering question, return "irrelevant".
Title: (TITLE)
Text: (TEXT)
Extracted span:
Table 26: Prompts for SNIPPET.
Instruction: Write an accurate, engaging, and concise answer for the given question using only the provided search results and cite them properly. Use an unbiased and journalistic tone.
Always cite for any factual claim.
You are provided summaiiesssnipett of the search results. You can use "Check: Document [1][2]" to check the corresponding full documents (you should only check relevant documents and you can at most check 3 documents at time) and use "Output:" to output a sentence in the answer. In the answer, cite properly by using [1][2][3]. Cite at least one document and at most three documents in each sentence. If multiple documents support the sentence, only cite minimum sufficient subset of the documents. Use "End" to end the generation.
<Retrieve for question "...">
<Get summriies/nnppets for the passages and delete those that are "irrelevant"> Document [I](Title: ...) (SUMMARY OR SNIPPET)
Question: When did break away from England?
Check: Document [1][2]
Document [1] (FULL TEXT)
Document [2] {FULL TEXT)
Output: The United States ... [1] ... [2].
<Remove the full text of [1][2] from context>
Check: Document [3]
Document [3] (FULL TEXT)
Output: The Treaty of Paris ... [3].
<Remove the full text of [3] from context>
End.
Table 27: An example for INTERACT.
Instruction: Write an accurate, engaging, and concise answer for the given question using only the provided search results and cite them properly. Use an unbiased and journalistic tone.
Always cite for any factual claim.
You can use "Search: key words" to check the most relevant document's 's full text and use "Output:" output sentence in the answer. the answer, cite properly by using [1][2][3]. Cite at least one document and at most three documents in each sentence. If multiple documents support the sentence, only cite a minimum sufficient subset of the documents. Use "End" to end the generation.
Table 28: Instruction for INLINESEARCH,
Instruction: Write an accurate, engaging, and concise answer for the given question. Use an unbiased and journalistic tone.
Table 29: Instruction for CLOSEDBOOK.

---

Instruction: Write an accurate, engaging, and concise answer for
Document [I](Title: How Treat and Prevent Food Poisoning MsPrepper) just typical gastro upset. Salmonella most commonly caused by eating undercooked or raw foods like eggs meat. You know how your mom always warned you not to eat raw cookie dough? This is why. Most people do cookie dough and they are fine, but salmonella could start to feel bad within in couple of hours after eating contaminated food, and sometimes it could take day or two. Common symptoms are nausea and vomi ting, loose stools (sometimes bloody), flu like symptoms, and stomach cramps. To treat
a risk. If you do contract salmonella, you
Document (Title::: FDA Issues Warning About Eating Raw Cookie Dough, But For Salmonella Risks) FDA Issues Warning About Eating Raw Cookie Dough, But Not For Salmonella Risks Used to licking the spoon or placating yourself with full-on chunks of raw cookie dough? The Food and Drug Administration issued a warning on Tuesday that strongly advises against continuing the habit. The agency asserted that consuming raw batter of any kind, whether for bread, cookies or pizza, could make a person sick. While you may have been warned in the past against eating raw dough due to the risk of contracting salmonella from raw eggs, the FDA is citing raw flour as the culprit for
Document [3](Title: It's Probably OK to Eat Raw Cookie Dough - As Long As You're Smart About It- The Crux Very Top Secret Information) First, when most people think about health risks and cookie dough, they think about raw egg. Eggs can be contaminated with salmonella bacteria, and food safety recommendations encourage people to cook eggs until the white and yolk are firm in order to kill any bacteria. However, anyone making cookies can do things to reduce this risk by using pasteurized egs products. When my kids and I make cookie dough, we never use regular eggs. Instead, we use shell eggs that have been pasteurized to kill any harmful bacteria without actually cooking the egg itself. (A great public health innovation, if
Document [T]tle:::: How Dangerous Is It to Eat Raw Cookie Dough? | Men's s Health) Can Eating Raw Cookie Dough Really Make You Sick? Scientists reveal the truth about this supposedly dangerous delicacy By Katherine Dempsey There are few things more tempting in life than eyeing a bowl of cookie dough and deciding whether or not to stick your finger in for a scoop. It's bit like playing Russian roulette. You could get lucky and enjoy the delicious dough wi wituuut consegence, but there's s always the risk of getting serously sick with food-borne illness. That's because multiple ingredients within the dough could be contaminated by pathogens such as Salmonella and E. coli, says Soahyoun
Document [5](Title: How Dangerous Is It Eat Raw Cookie Dough? Men's Health) is usually pasteurized, so it's not likely that the would make you sick. However, other ingredients in the dough could potentially harbor pathogens. Experts say that a prime suspect in a 2009 E. coli outbreak linked to prepackaged cookie dough actually the flour. Nuts and chocolate have also been linked to Salmonella outbreaks. Bottom line: You're better off skipping At least you'll 11 have killer cookies consolation. The article How Bad Is It To Eat Raw Cookie Dough? originally ran on Prevention.com con The Foods Most Likely to Make You Sick Mnm, Just Don't
cookie dough.
Question: Why is it bad to eat cookie dough for risk of salmonella but things like Cookie Dough Bites are ok?
Answer: Raw cookie dough is not recommended to be eaten due to the risk of salmonella [1][2]- Eating raw flour is also a risk for food poisaning [2]. The egg in cookie dough is usually pasteurized, so it's not likely that the egg would make you sick, but other ingredients in the dough could potentially harbor pathogens [4][5]. However, prepackaged cookie dough like Cookie Dough Bites is safe to eat because the dough is made with pasteurized egg products and heat-treated flour [2][3]..
Ground truth claims:
1. Cookie Dough Bites are safe to eat since they do not contain eggs. 2. Salmonella is bacteria found in eggs.
3. Eating raw cookie dough carries the risk of salmonella infection.
Table 30: ELI5 example I.

---

Instruction: Write an accurate, engaging, and concise answer for
Document [1](Title: Snapchat really worth $19 billion? CoMoni com) reporting that the Los Angeles-based company is aiming to raise $500 million at valuation of $16 billion to $19 billion, making it the third most highly valued tech start-up backed by venture capitalists. The Chinese handset maker Xiaomi is valued $45 billion, while Uber estimated to be valued about $40 billion, according to data from Insights. Read MoreVC investment hits S86B thanks to Uber, Xiaomi Snapchat was valued at $10 billion in August, according to Dow Jones report. Some of its investors from previous rounds include Benchmark, Lightspeed Venture Partners and Kleiner Perkins Caufield Document [2](Title: What Are Venture Capital Investments? DollarsAndSense my) Ever wondered how highly valued technology giants like Google and Facebook were able to grow so fast and pay their employees so well in such a short amount of time, or how still growing start-ups Like Uber are able to lose 1.2 billion US dollars in just the first half of this year alone and still command
valuation upwards of 50 billion US dollars? The answer lies with a special category of investment activity known as venture capital. Venture capitalists are professional investors who invest in a number of highly scalable high-risk technology ventures hoping to make a multi-fold
Document (Title: Opinion what Dara Khosrowshahi Must Do to Save Uber The New York Times) at a discount. These are troubling signs. Every start-up must one day fulfill the market's 's demand that it turn profit, but Uber has never figured out how to do that. While ride sharing in some form will probably survive, it's more likely that without some drastic changes, Uber won't t be around in three to five years. Mr. Khosrowshahi must avoid the mistakes of his predecessor by accepting that "pivots" (Silicon Valley-speak for the desperate changes troubled companies make to reassure their venture capitalist funders) are not the answer. None of the pivots Mr. Kalanick tried - like on-demand delivery
Document [4](Title: Snapchat raising funding round at $19 billion valuation: Report) Snapchat raising funding round at $19 billion valuation: Report The mobile messaging app Snapchat is said to be seeking funding at a $19 billion valuation, according to report, citing person with knowledge of the matter. Bloomberg is reporting that the Los Angeles-based company is aiming to raise $500 million at valuation of $16 billion to $19 billion, making it the third most highly valued tech start-up backed by venture capitalists. The Chinese handset maker Xiaomi is valued at $45 billion, while Uber is estimated to be valued at about $40 billion, according to data from CB Insights.
Document [5][Title: Unicorns And Wall Street | MoneyTips) not start showing profits soon. Even Uber, with its massive $62-billion-plus valuation, will test investor's patience if it does not start showing signs of profit in the next few years. Cracks are starting to show in terms of pre-IPO devaluations. Dropbox and Snapchat have seen a nearly one-quarter drop in their pre-IPO valuations, while Square's pre-IPO value has dropped by approximately one-third. Larger investors and venture capitalists sending message to the unicorns: Outline a solid path profits and start producing them soon, or expect your valuation to be slashed, Diving ahead with an IPO is not
Question: How are firms like snapchat, uber etc valued so highly while still not making profit? Do venture capitalists not expect some form of repayment within a number of years?
Answer: Venture capitalists invest in a number of highly scalable high-risk technology ventures hoping to make a nulti-fold return on their investment in a short amount of time [2]. Highly valued companies like Snapchat, Uber, and Xiaomi are able to grow so fast and command high valuations because they have the potential to generate significant revenue in the future [2][4]. Venture capitalists expect a return on their investment within a number of years, but they also invest in the terg-tern potential of the company [2]. It is important to note that every start-up must eventually turn profit, but some start-ups like Uber have not yet figured out how to do that [3][5].
Ground truth clains:
1. Firms like Snapchat and Uber need to establish their brand and amass users before introducing ads.
2. Introducing ads too early can deter potential users.
3. Uber is reinvesting a lot of money to make their service better.
Table 31: ELJ5 example 2.


Reviews:
[{"review_id": "5IiP6FXgxd", "reviewer": "Reviewer_CvfW", "summary": "", "questions": "Question A: Why is there no comparison of the presented results with some existing work in the literature?\nQuestion B: What is the extent of hallucinations, bias, fairness etc. in the text generated by LLMs in the task?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback! We are encouraged that you found our benchmark ALCE novel and useful for highlighting the challenges of LLMs. We address your questions below.\n\n**(1) Why is there no comparison of the LLM results with some existing work in the literature?**\n\nWe focus on LLMs because they become popular tools for information-seeking due to their ability to generate engaging output and handle a variety of topics. Without using LLMs, it remains challenging to generate fluent and coherent text and to generalize to a diverse set of questions.\nOur paper tackles a new setting — LLM generations with citations.\n\nThanks to the reviewer’s suggestion, we added experiments of a state-of-the-art question answering model, Fusion-in-Decoder (FiD; Izacard and Grave, 2021), trained on the training set of ASQA. We train a T5-base FiD model for 5 epochs on the ASQA training set with a batch size of 64 and a learning rate of 1e-4.  Since **there is no citation annotation data available**, we train the model to generate the answer and use our PostCite method (Section 4.3) to generate citations. We evaluate this model on both ASQA (in-domain) and ELI5 (out-of-domain).  **Note that this is not a direct comparison, as ALCE assumes only evaluation data available and uses only few-shot data for prompting**. \n\nASQA results:\n\n| Model | Fluency (MAUVE) | Correctness (EM recall) | Citation recall | Citation precision\n| -------- | -------- | -------- | -------- | -------- |\n| ChatGPT Vanilla  | 66.6  | 40.4  |  73.6   | 72.5                |\n| FiD + PostCite| 75.8 |   28.4             |  58.1                 | 58.0 |\n\n\n\nELI5 results:\n\n| Model | Fluency (MAUVE) | Correctness (Claim recall) | Citation recall | Citation precision\n| -------- | -------- | -------- | -------- | -------- |\n| ChatGPT Vanilla  | 57.2  | 12.0  |  51.1  | 50.0                |\n| FiD + PostCite | 25.2 | 4.4          |  39.3                 | 39.3 |\n\nAs the results show, the FiD baseline still significantly lags behind prompting ChatGPT in both correctness and citation quality (even though it is trained on 4000+ examples). When tested on another dataset (ELI5), FiD performs even worse, showing that it is challenging to solve the problem by fine-tuning a small pre-trained model.  We are happy to add these results and more discussion in the revision.\n\nWe are unsure what the reviewer refers to as “existing literature”. As we stated in the paper, there are no available baselines to compare to for LLM generations with citations, and there is no available training data. WebGPT (Nakano et al., 2021) and GopherCite (Menick et al., 2022) take a similar setting, but their training data, code, and models are not released, and thus it is impossible to reproduce them. \n\nThe main contribution of our work is to propose the first automatic benchmark that can evaluate generations with citations, and by evaluating several simple but effective baselines, we demonstrate promising directions to work on. We look forward to new methods emerging to tackle this problem.\n\n**(2) What is the extent of hallucinations, bias, fairness etc. in the text generated by LLMs in the task?** \n\nOur correctness and citation quality evaluations are about hallucination — they examine whether the outputs are factually correct and faithful to the retrieved evidence. We believe that other factors, such as bias and fairness, are beyond the scope of the work.\n\nTo provide more error analyses of LLMs’ outputs on our benchmark, we manually examined 30 outputs generated by ChatGPT Vanilla on ASQA and logged different error types here (one output can have multiple types of errors).\n\n| Error type | Occurrences (out of 30 outputs) |\n| -------- | -------- |\n| Missing a short answer because of the retrieved passages do not contain it  | 11 |\n| Missing a short answer while the retrieved passages contain it  | 6 |\n| Hallucination | 2 |\n| Correct but irrelevant content to the question | 2 |\n\nMost of the errors are caused by the low coverage of the retrieval passages. LLMs also fail to extract the useful information from the passages often. There are only 2 instances among all the 30 questions we examined where the LLM made up some factually incorrect contents.\n\nNakano et al., 2021. WebGPT: Browser-assisted question answering with human feedback. \n\nMenick et al., 2022. Teaching language models to support answers with verified quotes. \n\nIzacard and Grave, 2021. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper presents an interesting problem with the benchmark -- ALCE, the first benchmark for Automatic LLMs' Citation Evaluation, which has some novelty over previous benchmarks.\n\nThe evaluation is done considerating fluency, correctness, and citation quality.\n\nIt highlights the efficacy as well as the challenges of LLMs in this task.\n\nHuman evaluations add more quality to the benchmark.", "reasons_to_reject": "Apart from LLMs, no other baseline models are considered for comparison; this makes it difficult the gauge the relative efficacy of the LLMs in light of the state of the art. It is important to know if the LLMs advance the existing literature in performance.\n\nI would have loved more analysis of factors like hallucinations, bias, fairness etc. of LLMs in this task\n\nThese have been now addressed in the rebuttal.", "questions_to_authors": "Question A: Why is there no comparison of the presented results with some existing work in the literature?\nQuestion B: What is the extent of hallucinations, bias, fairness etc. in the text generated by LLMs in the task?", "paper_topic_and_main_contributions": "The paper \"Enabling Large Language Models to Generate Text with Citations\" presents an interesting problem on a proposed benchmark and solutions that leverage LLMs. After the LLM application, the authors apply re-ranking and post-citing.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CXb0uL2xFi", "reviewer": "Reviewer_D8zW", "summary": "", "questions": "Could you provide the code used in this experiments and share the assessments performed by the annotators for reproducibility purposes?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback! We are encouraged that you found our experiments and analysis thorough and that the human evaluation helped validate our metrics. We answer your questions below.\n\n**(1) Why do you randomly select 1,000 examples from the development set of each dataset for ALCE?**\n\nWe randomly sample examples mainly for efficiency. Running inference of LLMs can be slow and costly; furthermore, in our preliminary experiments, using more examples for evaluation does not change the results or the variance significantly.\n\n**(2) How do you measure citation recall?**\n\nSorry for any confusion in the paper. Citation recall is defined as “the percentage of output statements that are fully supported by their citations”. We use TRUE (Honovich et al., 2022) to determine whether the cited passages support the statement. TRUE is a T5 model fine-tuned on multiple natural language inference (NLI) datasets.\n\nAn NLI model takes a “premise” and a “hypothesis” and outputs whether the “premise” entails the “hypothesis”. We concatenate all cited passages as the “premise”, and take the statement as the “hypothesis”. We say the passages support the statement if the NLI model outputs “entailment”. Our Figure 3 provides an example for the process.\n\n**(3) Could you provide the code used in this experiments and share the assessments performed by the annotators for reproducibility purposes?**\n\nWe provided our code, data, and evaluation instructions in the supplementary materials. We will release them as well as the full annotator assessment logs upon acceptance.\n\nOr Honovich et al. TRUE: Re-evaluating factual consistency evaluation. NAACL 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Besides the core topic addressed by the paper (adding citations to the generated text, which is one of the most important problems of LLMs), the experimental setup and analysis are very well presented.\n\nThe quality of the output which is produced by LLMs is analyzed according to three different dimensions (fluency, correctness, quality of citations).\n\nThe human evaluation is an added value to this paper in order to double check the quality of the results.", "reasons_to_reject": "There are a couple of minor issues that I would like to point out:\n- why do you \"We randomly select 1,000 examples from the development set of each dataset for ALCE.\"? \n- how do you measure citation recall?  I mean, recall in terms of total number of citations that support the claim. It is not clear even from the papers that are cited in Section 3.3.\n\n", "questions_to_authors": "Could you provide the code used in this experiments and share the assessments performed by the annotators for reproducibility purposes?", "paper_topic_and_main_contributions": "In this paper, the authors present a novel approach to LLMs where citations are required for specific text passages that LLM generates.\n\nThe introduction of this feature (citation) offers advantages such as verify the claims made by LLMs (by referring to the provided citations), and reduce hallucination.\n\nThe key results of this study are that: the quality of retrieval plays a pivotal role in determining the final performance, the limited context window of LLMs restricts the number of passages they can integrate, LLMs encounter difficulties in effectively synthesizing multiple documents in context while avoiding distraction from irrelevant information.\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GZey0RvP6O", "reviewer": "Reviewer_jGRf", "summary": "", "questions": "- exact match seems very strict, what if, for example a date was formatted in a different way from the exact answer, or are there multiple answer models per question?\n- recall 100% if at least 5 correct answers is given - but I assume also 100% if there are less than 5 relevant and all are given?\n- Post-editing; why n=4 answers?\n- in terms of retrieval analysis. Is there any effect in the overlap of selected passages on correctness? Aka if the retrieved passages overlap a lot is the correctness then higher?\n- Would it be possible to instruct chatGPT to reuse the text from the passages in order to maybe improve the exact match?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback! We are encouraged that you found ALCE important and relevant, and our work thorough and well written. We answer your questions below.\n\n**(1) Unclear whether the dataset will be released.**\n\nWe attached our data as well as evaluation code in the supplementary material. We will release them upon paper acceptance. \n\n**(2) Unclear how the kappa is calculated between ALCE (binary?) and human judgement (3 classes) for citation precision.**\n\nSorry for the confusion. As stated in Section 6 (line 476-478), in human evaluation, we consider a citation to have a precision score of 1 if the citation “partially supports” or “fully supports” the statement. Then, we take the ALCE citation precision and human citation precision results (both binary) for calculating the Kappa.\n\n**(3) Exact match seems very strict. What if, for example, a date was formatted in a different way from the exact answer, or are there multiple answers per question?**\n\nBoth ASQA and QAMPARI provide aliases for each entity answer. We normalize the model outputs and answers (deleting all punctuations and lower-casing all letters) and report the exact-match with the best matching aliases (line 962). Most ASQA and QAMPARI short answers are entity names or dates so exact match with aliases should suffice.\n\n**(4) Recall is 100% if at least 5 correct answers are given. Is the recall also 100% if there are fewer than 5 correct answers and all are given?**\n\nYour understanding is correct. If all correct answers are given, the recall is also 100%.\n\n**(5) Why do you choose n=4 for Rerank?**\n\nIn our preliminary experiments, we find that n=4 achieves a good balance between citation quality, correctness, and inference speed. Using larger n leads to marginal improvement on citation quality but is significantly slower.\n\n**(6) In terms of the retrieval analysis, is there any effect in the overlap of selected passages on correctness? Aka if the retrieved passages overlap a lot is the correctness then higher?**\n\nThis is a great question. We did not conduct such an analysis but will make sure to add it in the revision. Our intuition is that a more diverse set of retrieved passages will benefit the correctness, as most questions require multiple different sources to answer. This is also corroborated by findings in ASQA (Stelmakh et al., 2022).\n\n**(7) Would it be possible to instruct ChatGPT to reuse the text from the passages in order to maybe improve the exact match?**\n\nThis is a good point. In our preliminary experiments, we tried a similar prompting strategy where we asked ChatGPT to explicitly copy the text from the retrieved passages, but it did not lead to an improvement in correctness.\n\nStelmakh et al. ASQA: Factoid Questions Meet Long-Form Answers. EMNLP 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- important topic, very relevant for current challenges\n- well written, thorough work", "reasons_to_reject": "unclear wether the dataset (at least the randomly selected examples) will be released - without it, it would be hardly a reproducible benchmark. (for the remainder I have assumed that they will release it as they do state \"We present ALCE, the first reproducible benchmark\" and the rest of the article is very nice)\n\nminor point: \na little bit unclear how the kappa is calculated between ALCE (binary?) and human judgement (3 classes) for citation precision", "questions_to_authors": "- exact match seems very strict, what if, for example a date was formatted in a different way from the exact answer, or are there multiple answer models per question?\n- recall 100% if at least 5 correct answers is given - but I assume also 100% if there are less than 5 relevant and all are given?\n- Post-editing; why n=4 answers?\n- in terms of retrieval analysis. Is there any effect in the overlap of selected passages on correctness? Aka if the retrieved passages overlap a lot is the correctness then higher?\n- Would it be possible to instruct chatGPT to reuse the text from the passages in order to maybe improve the exact match?", "paper_topic_and_main_contributions": "\nthe authors present a method for generating citations with LLMs as a means to battle factual correctness and verifiability. They provide a benchmark for citation evaluation and use automatic metrics for fluency, correctness and citation quality. These metrics correlate well with human judgements. Nice contribution to the field.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed generating citations with LLMs as a means to enhance factual correctness and verifiability. A benchmark for citation evaluation is created which uses automatic metrics for fluency, correctness, and citation quality. Besides the core topic addressed by the paper, the experimental setup and analysis are very well presented. In general, it is a nice contribution to the field.
Decision: Accept-Main