Evaluating Verifiability in Generative Search Engines
Authors: Nelson F. Liu
Tianyi Zhang
Percy Liang
Venue: EMNLP 2023 Findings (2023)
Keywords: generative
search
engines
verifiability

Abstract: Generative search engines directly generate responses to user queries, along with in-line citations. A prerequisite trait of a trustworthy generative search engine is verifiability, i.e., systems should cite comprehensively (high citation recall; all statements are fully supported by citations) and accurately (high citation precision; every cite supports its associated statement). We conduct human evaluation to audit four popular generative search engines—Bing Chat, NeevaAI, perplexity.ai, and YouChat—across a diverse set of queries from a variety of sources (e.g., historical Google user queries, dynamically-collected open-ended questions on Reddit, etc.). We find that responses from existing generative search engines are fluent and appear informative, but frequently contain unsupported statements and inaccurate citations: on average, a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence. We believe that these results are concerningly low for systems that may serve as a primary tool for information-seeking users, especially given their facade of trustworthiness. We hope that our results further motivate the development of trustworthy generative search engines and help researchers and users better understand the shortcomings of existing commercial systems.

Evaluating Verifiability in Generative Search Engines
Nelson F. Liu"
Tianyi Zhang
Percy Liang
Computer Science Department
Stanford University
nfliuêcs. stanford edu
Abstract
What are the latest discoveries from the James Webb Space Telescope?
Generative search engines directly generate re- sponses to queries, along with in-line ci- tations. A prerequisite trait of trustworthy generative search engine verifiability, i.e. systems should cite comprehensively (high ci- tation recall; all statements are fully supported by citations) and accurately (high citation pre- cision; every cite supports its associated state- ment). We conduct human evaluation to audit four popular generative search engines-Bing Bing Chat, NeevaAI, perplexity.ai, and YouChat- across a diverse set of queries from variety of sources (e.g., historical Google user queries, synamically-collecte
The Webb Space Telescope designed peer into the dusty clouds of gas where stare and planetary systems born. Wabb has captured the first direct image of an exoplanet, and the Pillars of Crestion the Eagle Nebula[]][]], Additionally, the telescope will be used to study the next interatellar Interloper(3].
(*Some generated statements may not be fall supported atstions whie others fuly supported)
Cited Webpages
[1]
nesa gov (Xcoitation does not support it associated statemett
NASA's Webb Canfirms Its First Exoplanet
Resesrchers confirmed exoplanet, a planet orbits
star, using NASA's Jamas Webh Space Telescope the time.
[3]
cnn. cam ctstion partelly surports associsted stwtement) Pillars Creation: James
Space Telescape ...
The Pillars Creation, in Engle Nebula, is star-forming region captured in new image (rght) by the James Webl Space Teleacope that reveals more detail than 2014 image (lef) by Hubble
open-ended questions on Reddit, etc.). We find that responses from existing generative search engines are fluent and appear informative, but frequently con- tain unsupported statements and inaccurate ci- tations: on average, a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence. We believe that these results are con- cerningly low systems that may serve as a primary tool for information-sedee seeking users, especially given their facade of trustworthiness. We hope that our results further motivate the development of trustworthy generative search engines and help researchers and users better understand the shortcomings of existing com- mercial systems.
nese gov (Sottation full suppors assodiated statement) Studying the Next Interstellar interloper with Webh Sciemists have had only limited ability to study these objects ance discovered, but that about to change with NASAs James Webo capabilties both the near-infrared midiinfrared bands study two different espacts interstellar object
will Webb's spectroscopic
Figure I: Generative search engines answer user queries by generating tailored response, along with in-line citations. However, not all generated statements are fully supported by citations (citation recall), and not every citation supports associated statement (citation precision).
[Bing] Chat daily". and that Bing Chat served 45 million chats in the first month of its public preview (Mchdi, 2023). Generative search engines have the potential to transform how people find informa- tion online, but generated responses from existing large language model-backed generative search en- gines may not always be accurate (Maynez et al., 2020). Given their potential and rapid mainstream adoption, it is critical to evaluate these systems to better understand their potential limitations (akin to prior work in algorithmic auditing: Metaxas and Pruksachatkun, 2017; Buolamwini and Gebru, 2018: 8; Kiritchenko and Mohammad, 2018; Robert- son et al., 2018; Metaxa et al., 2019; Green and Chen, 2019; Birhane et al., 2022, inter alia).
1 Introduction
Generative search engines fulfill user information needs by directly generating responses to input queries, along with in-line citations (Figure 1). Existing generative search engines are rapidly gain- ing users in March 2023, Microsoft reported that "roughly one third of daily preview users are using
'This work would not possible without the annotators who performed human evaluation; we thank them for their contributions.
'In contrast, conventional search engines are limited to retrieving pre-existing webpages.

---

A prerequisite trait of trustworthy generative search engine is verifiabilityy" that is, each gen- erated statement about the external world should be fully supported by a set of in-line citations, and each provided citation should support its associated statement. Verifiability enables readers to easily check that any generated statement is supported by its cited source.
knowledge about the James Webb Space Télescope (motivating a query about its recent discoveries) will likely struggle to identify unsupported state- ments in the generated response. We hypothesize that citation precision is inversely correlated with perceived utility because generative search engines often copy or closely paraphrase from their cited webpages ($4.4). This improves citation preci- sion because copied text is often supported by the cited webpage, but decreases perceived utility when copied statements are irrelevant to the query or the rest of the generated response.
We conduct human evaluation audit four popular commercial generative search engines (Bing Chat, NeevaAI, perplexity.ai, and YouChat) across a diverse set of information-ssckingg queries (e.g., various types of historical Google user queries from NaturalQuestions (Kwiatkowski et al.. 2019), dynamically-colleeted open-ended ques- tions from Reddit; see Appendix A for examples). For each query-response pair, use human evaluation to measure variety of dimensions: 1. fluency (whether the generated text is fluent and cohesive; $2.2);
We make the following contributions: first, we define the citation recall and citation precision eval- uation metrics, which aim encourage the devel- opment of systems that cite comprchensively and correctly. Second, we conduct a human evaluation of four popular generative search engines, finding that responses are broadly fluent and appear useful, but frequently contain unsupported statements and inaccurate citations, increasing their potential to mislead users. Third, we observe that perceived utility is inversely correlated with citation precision in existing generative search engines, and hypothe- size that this inverse correlation occurs when some systems copy or closely paraphrase from cited web- pages. To facilitate further work on developing trustworthy generative search engines, we have re- leased our human evaluation annotations.
2. perceived utility (whether the generated an- swer is helpful and informative; $2.2);
3. citation recall (the proportion of generated statements about the external world that are fully supported by their citations; $2.3); and
4. citation precision (the proportion of generated citations that support their associated state- ments; $2.4).
A trustworthy generative search engine should achieve high citation recall and precision, indicat- ing that its generated citations are comprehensive (every generated statement is fully supported by citation) and correct (every citation supports its associated statement).
2 Human Evaluation of Fluency, Perceived Utility, and Verifiability
In this section, formalize the inputs and outputs of the generative search engines we study, describe the evaluation of fluency and perceived utility, and define and describe the evaluation of citation recall and precision. Citation recall and precision are de- signed to reward systems that cite comprehensively (i.e., high recall; all statements are fully supported by citations) and accurately (i.e., high precision; ev- cry cite supports its associated statement). We also define citation Fi, metric that combines citation precision and citation recall.
We find that existing generative search engine re- sponses often have high fluency and perceived util- ity ($4.1), 1), but frequently contain unsupported state- ments or inaccurate citations (low citation recall and precision; $4.2). On average, merely 51.5% of generated sentences are fully supported with cita- tions (citation recall), and only 74.5% of citations support their associated sentence (citation preci- sion). Furthermore, citation precision inversely correlated with perceived utility (r 0.996); the responses that seem more helpful are often those with inaccurate citations ($4.3). This facade of trustworthiness increases the potential for existing generative search engines to mislead users. For examplc, in Figure I, user with little background
2.1 Task Formulation
Given user query input, a generative search engine produces text response which is string with embedded in-line citations. For the example in Figure 1, the query g "What are the latest discov-
WW adopt the term verifiabiliiy from the Wikipedia com- munity. Verifiability is core content policy of Wikipedia: statements must be fully supported by provided sources.
hwwpa///ww..ooooooooolaaltinnnnn generaaive-sccrcc-onninee

---

eries from the James Webb Space Telescope?" and the response is the string paragraph "The James Webb Space Telescope ... used to study the next in- terstellar interloper [3].", with embedded citations "[1J", "[2]", and "[3]".
First generated statement 23
Second generated statement [1 [
Third generated statement [4ma][5 1
Citation Recall: 3/3 100%
Citation Precision: 3/8 37.5%
First generated statement [1 12 Second generated statement [2x] Third generated statement.
To evaluate citation precision and recall, we first segment the into set of statements S {81,...,8p] 8p]. In this work, the segmenta- tion is set of sentences in the response r. For each statement 8; S, we construct (possibly empty) set C, - (c,,,...,, of & citations as- sociated with the statement 8;, where Cij is the jth citation associated with the ith response state- ment. For each citation Cigi we have a URL Wij and its contents Pij- In this work, Cs is set of cita- tions that occur in 8j (e.g., for 8; "Blueherries(1), cherries/2], and grapes[3] grow on trees.[4]". C,
Citation Recall: 1/3 33%
Citation Precislon:
First generated statement (1 [3x] Second generated statement. Third generated statement.
Citation Recall: 1/3 33% Citation Precision: 2/3 66%
highligtted statement is fully supported by citations
highlighted statemant not fully supported citations.
citation fully supports its sesociated statement
partially supports citation does not support Its associated statement.
{[1], [21, [31, [4]]).
Figure 2: Examples of calculating citation recall and precision. Citation recall measures the proportion of generated statements that are supported by citations. Ci- tation precision measures the proportion of citations that support their associated statements. Partially-supporting citations only improve citation precision when their as- sociated statement is supported by the union of its cita- tions and no other associated citation fully supports the statement by itself (middle example).
In practice, sentence may contain multiple independenty-verrffabble claims (e.g., conjuncts such as "Cups can be made of glass[1] or plas- tie[2]."), and single in-line citation's scope is often ambiguous (e.g., cite marker after two state- ments could be interpreted as either supporting both statements, or merely the final one); we leave finer-grained evaluation to future work.
2.2 Measuring Fluency and Perceived Utility
that are not verification position that every generated statement about the external world is verification-worte those that might seem obvious, trivially true, "common sense". Generated statements may be incorrect, and statements that seem obvious some readers may be less than obvious to others (e.g., "The Pope is Catholic"). We believe that systems should aim to provide a source for all generated statements about the external world, enabling readers to easily verify any statement in generated response.
We take the
To measure response fluency, annotators were shown the user query, the generated response, and the claim "The response is fluent and cohesive". We ask annotators rate their level of agreement with the claim on five-point Likert scale from Strongly Disagree to Strongly Agree. We use similar process to measure perceived utility, asking annotators to rate their level of agreement with the claim "The response is a helpful and informative answer to the query".
even
In practice, almost all system-generated state- ments are verification-wortie include statements about the speaker (the system) itself (e.g., "As a language model, do not have the ability to ban books.") and questions posed to the user "Would you like to learn more?", generated by systems like Bing Chat and YouChat that deployed in conversational settings).
2.3 Measuring Citation Recall
exceptions
Citation recall is the proportion of verification- worthy statements that are fully supported by their associated citations (see Figure 2 for several ex- amples). Thus, computing citation recall requires (i) identifying the verification-worthh statements in a response and (ii) evaluating whether each verification-worthy statement fully supported by its associated citations.
Evaluating whether a verification-workke state- ment is fully supported by its associated cita- tions. Given the verification-worthy statements in a response r, annotators evaluate whether cach statement is fully supported by its associated cita-
Identifying verification-worth statements. Given the statements S in a response r, we first ask annotators to remove statements in the response

---

tions (see the sentences of generated response in Figure I for examples). To collect these binary judgments, we use the attributable to identified sources (ATS) evaluation framework of Rashkin et al. (2022). In particular, a statement 8; is fully supported by its associated citations C, if a generic hearer would affirm the statement "According to cited webpages C,, s,", within the context of the query q and response r, and unsupported otherwise.
in the response. (Tys Tp.)/N.
Then, the citation precision is
2.5 Citation Fi
Citation Fi is metric that combines citation pre- cision and citation recall by taking their harmonic mean:
F = citation precision citation recall citation precision + citation recall
2.4 Measuring Citation Precision
To achieve high citation F1. systems must have high citation precision and high citation recall.
Citation precision is the proportion of generated citations that support their associated statements (Figure 2). In contrast to citation recall, citation precision rewards systems for citing accurately- response that cites every webpage on the Internet for each generated statement would have high cita- tion recall, but low citation precision (since many articles are irrelevant and do not support their as- sociated statement). To measure citation precision for response r, first ask annotators judge whether each citation GiP contributes full, partial, or no support for its associated statement 8; (see cited webpages in Figure 1 for examples):
3 Evaluation Setup
In this section, we deseribe the evaluated generative search engines ($3.1), the diverse query distribu- tions we use for evaluation ($3.2), and the details of our human evaluation protocol ($3.3).
3.1 Evaluated Generative Search Engines
We evaluate four existing commercial generative search engines: Bing Chat, NeevaAI, perplexity.ai ai, and YouChat. These systems pattern after prior work (e.g., Nakano et al., 2021; Menick al., 2022; Glaese et al., 2022; Thoppilan et al., 2022, inter alia) and generate responses by conditioning large language models on the input query and retrieved content (c.g., search results from a conventional search engine). For each input, we save the sys- tem's 's first complete response (i.e., single-tum).. Re- sponses were scraped between late February and late March 2023.
Full support: all of the information in the statement is supported by the citation.
Partial support: some of the information in the statement is supported by the citation, but other parts are not supported (e.g., missing or contradictory).
No support: the citation docs not support any part of the statement (c.g., the cited webpage is completely irrelevant or contradictory). For statements that have multiple associated ci- tations, additionally ask annotators whether the union of its associated cited webpages collectively provides full support for the statement (a binary judgment). Similar to citation recall, we use the AIS evaluation framework of Rashkin et al. (2022) to collect these binary judgments.
Note that evaluated generative search engines have differing abstention rates (Table 1), which can make direct comparison difficult-one might ex- pect that systems with higher abstention rates might also have higher evaluation performance, since they can simply abstain from generating responses difficult queries (we do not find this to be the case in practice). NeevaAl abstains from responding on nearly 23% of evaluated queries, since its response
To calculate citation precision, let Tya be the number of citations that fully support its associ- ated statement, and let Tps be the number of cita- tions that partially supports its associated statement, where the associated statement is fully supported by the union of its associated citations and no as- sociated citation fully supports the statement by itself.4 4 Let N be the total number of citations 'For an intuitive example of when surtiall-suppooringg cites count toward improving precision (greater T Tpx). consider the statement "Health henefits of cycling include improved car- diovascular health[1] and lowered cholesterol levels[2]." with
its associated citations and [2]. Suppose that these citations each contribute partial support for the entire statement-the first citation [1] only states that "Health benefits of cycling include improved cardiovascular health", and second citation [2] only states that "Health benefits of cycling include lowered cholesterol levels". Taken together, the citations offer full sup- port
statement. Although these citations do not fully support statement on their own, they still meaningfully contribute to its verifiability sssssssssessssss should not be penalized for aggregating information from multiple citations.
"We do not evaluate OpenAl's ChatGPT or Google's Bard hecause they do not provide in-line citations for their responses (as of March 2023), and thus trivially have low verifiability.

---

Abstention Rate (L)
(KILT) uses historical queries from the KILT ELI5 dataset (Fan et al., 2019; Petroni et al., 2021), drawn from posts created before July 2018. A retrieval-based system could hypothetically per- form well on ELI5 (KILT) by simply identifying the query's 's source Reddit ELI5 post and copying its content. As a result, we also evaluate generative search engines on the ELI5 (Live) subdistribution, which increases ecological validity by evaluating systems on real user queries at their time of creation and reducing the incidence of search results with the query's 's exact keywords. We continuously listen to the stream of new Reddit ELI5 posts and immediately query generative search engines for responses whenever new post is created. This en- sures that the source ELI5 post will not have been indexed (and thus, cannot be retrieved) by conven- tional search engines. minimizing the possibility that the generative search engine has access to the source ELI5 post.
Bing Chat
0.5%
NeevaAI
22.7%
perplexity.ai
0.5%
YouChat
0.5%
Table 1: Generative search engines may be designed for deployment in different contexts. NeevaAI abstains from responding to 22.7% of our 1450 queries, since its response designed for display within conventional search results page. In contrast, the conversational in- terface of Bing Chat, and YouChat means that systems must generate response for nearly every input user query (excepting, e.g., query character length limits).
is displayed within a conventional search engine results page. In contrast, Bing Chat, perplexity.ai. and YouChat respond to almost every user query.
3.2 Evaluated Query Distributions
To gain a broader understanding of the strengths and weaknesses of existing commercial generative search engines, we evaluate on a diverse set of queries from a variety of sources (e.g., Google user queries, open-ended Reddit questions, how-to queries) requiring knowledge from several different answer types (e.g., short textual spans, long-form paragraph, lists, or tables). See Appendix A for example queries from each distribution. Each sys- tem is evaluated on 1450 queries-11 -150 randomly- sampled queries from each of AllSouls, davinci- debate, EL15 (KILT/ / Live), and WikiHowKey- words, and 100 randomly-sampler queries for each of the seven NaturalQuestions subdistributions.
WikiHowkeywords We evaluate systems queries derived from WikiHow articles. We found that directly querying generative search engines with WikiHow article titles yields responses that largely paraphrase or copy text directly from Wik- iHow. As result, we use text-davinci-O0 to paraphrase article titles (e.g.. "How to Cut An Avo- cado") into keyword queries (e.g.. "cut avocado"). NaturalQuestions. We evaluate generative search engines on NaturalQuestions (Kwiatkowski et al., 2019) queries, stratified by their answer type. NaturalQuestions contains historical queries issued to the Google search engine coupled with long and short answers extracted from Wikipedia. We evaluate on queries from 7 NaturalQuestions subdistributions: queries with paragraph-type long answers (i) with and (ii) without short answers, queries with list-type long answers (iii) with and (iv) without short answer, queries with table-type long answers (v) with and (vi) without short answers, and finally (vii) queries with no long answer (and thus short answer either).
AllSouls. We evaluate systems on open-ended essay questions taken from the entrance exam (gen- eral paper component) for All Souls College, Ox- ford University. These questions cover topics in- cluding the arts, science, politics, literature, current events, and issues in education and sport.
davinci-debate. We evaluate systems on debate topics generated from text-davinci- -003. To gen- erate debate queries, we follow the procedure of Bakker et al. (2022); see Appendix B. for details.
Summary. In total, we evaluate existing gener- ative search engines on 12 total query distribu- tions. Eight query distributions are taken from prior work (ELI5 (KILT) and the seven Natural tions query distributions), while four query distri- butions were constructed for this work: AllSouls,
ELIS. We take queries from the "Explain Like I'm Five" (ELI5) subreddit, where users provide long-form layperson-accessiin answers to submit- ted questions. Submitted questions are required to admit objective explanations, and answering them often requires forg-form textual responses.
'The goal of the ELIS (Live) subdistribution is not to iden- tify queries with no relevant webpages on the Internet. We recognize that many user queries express previously-seen in- formation neods, even if the precise wording differs.
We consider two subdistributions of ELI5 queries: ELI5 (KILT) and ELIS (Live). ELIS

---

davinci-debate, EL15 (Live), and WikiHowKey- words. These diverse settings provide broad cover- age of several potential use cases and information needs, helping us gain a comprehensive understand- ing of systems' strengths and weaknesses.
4 Results and Analysis
This section presents the results of our human eval- uation study and discusses our main observations and analyses. We see that fluency and perceived utility are generally high across different generative search engines ($4.1), while citation recall and pre- cision are quite low ($4.2), though performance cer- tainly varies by system and query distribution-the low citation recall and precision, when combined with the facade of trustworthiness from fluency and high perceived utility, increase the potential for isting generative search engines to mislead users. Our results also show that citation precision is in- versely correlated with perceived utility existing generative search engines ($4.3). We hypothesize that this is a byproduct of systems' propensity to copy or closely paraphrase text from cited web- pages, which may increase citation precision and decrease perceived utility ($4.4).
3.3 Human Evaluation Protocol
Annotation process. Evaluating a single query- response pair requires human annotators to com- plete three-step The first step measures the re- sponse's fluency and perceived utility ($2.2), and the second and third step provide the judgments necessary to measure citation recall ($2.3) and pre- cision ($2.4). See Appendix for screenshots of the annotation interface and Appendix D for the annotation guidelines.
Annotator recruitment and training. Annota- tion was performed on Amazon Mechanical Turk. Annotators were pre-screened with qualification study, which required them to read annota- tion guidelines document and evaluate five repre- sentative query-response pairs. We individually reviewed submitted annotations for qualification study and provided annotators with personalized feedback to help correct any misconceptions or con- fusion about the task. Annotators who performed well on the qualification study and demonstrated thorough understanding of the task and annotation guidelines were permitted to participate in the main round of human evaluation. We remained in con- stant contact with annotators throughout the hu- man evaluation process to answer questions about corner-cases and clarify intended behavior. In total, 34 annotators participated in human evaluation.
4.1 Fluency and Perceived Utility
See Appendix F for full fluency and perceived util- ity results for every generative search engine on each of our query distributions.
Generated responses are fluent and appear help- ful. Averaging across all systems and responses yields average rating of 4.48 for fluency and 4.50 for perceived utility, indicating that annota- tors generally found generated responses fluent and helpful for answering the user's input query.
Comparing fluency and perceived utility be- tween generative search engines. Comparing fluency and perceived utility ratings between the generative search engines (aggregated over all re- sponses), we see that Bing Chat receives the lowest fluency perceived utility ratings (4.40 / 4.344,, fol- lowed by NeevaAl (4.43 / 4.48), perplexity.ai (4.51 / /4.56), and YouChat (4.59 / 4.62)..
Annotator compensation. Annotators were compensated $1.00 per query-response pair for responses with citations, and $0.38 per query- response pair for responses without citations ($15.00 per hour, by conservative time estimates). On average, annotators took approximately four minutes to complete all three steps for a single query-response pair for responses that contained at least one citation.
Comparing fluency across query distributions. Comparing average fluency ratings across different query distributions, we see similar ratings between NaturalQuestions queries that have a long answer (i.e., an extractive answer of some length exists on Wikipedia) and non- con-NaturalQuestionn distributions (4.50 vs. 4.47, respectively). Comparing average fluency ratings between NaturalQuestions subdistri- butions, we see that generated responses queries that have a short extractive answer are generally more fluent (4.55) than responses to queries with only a long answer (4.46) or those without a long
Annotation agreement. Each query-respone pair is annotated once in the human evaluation process. To measure inter-annotator agreement, we collected three annotations for 250 randomly- sampled query-response pairs, finding high agree- ment rates (greater than 82.0% pairwise agreement and 91.0 F1 for all judgments; see Appendix E)..

---

answer (4.46), perhaps because responses ques- tions with short answers are generally shorter and often only require factoid knowledge.
pear informative and useful.
Comparing citation recall and precision be- tween generative search engines. Citation recall and precision varies dramatically between different generative search engines. perplexity.ai achieves the highest average recall (68.7), compared to Nee- vaAI (67.6), Bing Chat (58.7), and YouChat (11.1). On the other hand, Bing Chat achieves the highest average precision (89.5), followed by perplexity.ai (72.7), NeevaAl (72.0), and YouChat (63.6). A gap of nearly 58% separates the system with the high- est and lowest recall (perplexity.aa vs. YouChat), and the gap between the systems with the highest and lowest precision is almost 25% (Bing Chat vs. YouChat).
A notable outlier distribution is NaturalQues- tions queries with table-type long answers and no short answers, where system responses are dra- matically less fluent (average of 4.36 across sys- tems vs. average of 4.48 across all query distribu- tions). These challenging queries often require ag- gregating information across table cells or retrieved sources, since the lack of a short answer implies that no single Wikipedia table cell directly answers the question (c.g., the query "how many grammys does beyonce have without destiny's child"). When the retrieved webpages do not contain clear ex- tractive answer the query, but contain facts that seem relevant (e.g., information about Destiny's Child's first Grammy, or Beyonce's 's total number career Grammy awards), the generated response is often a stilted agglomeration of statements from various sources, reducing overall fluency.
Comparing citation recall across query distri- butions. Modifying the evaluation query distri- bution appears to affect citation recall more than citation precision. For example, the gap in cita- tion recall between NaturalQuestions queries with long answer and non-NaturalQuessioo queries is nearly 11% (58.5 vs. 47.8, respectively). Simi- larly, the difference in citation recall between Nat- uralQuestions queries with and without short an- swers is nearly 10% (63.4 for queries with a short answer, 53.6 for queries with only a long answer. and 53.4 for queries with no long or short answer).
Comparing perceived utility across query distri- butions. In contrast to fluency, perceived utility can differ substantially between different query dis- tributions. Perceived utility is much higher for NaturalQuestions queries containing a long answer (4.59), as opposed to non-NaturllQuusstion queries (4.43). Comparing between different NaturalQues- tions subdistributtions, we see that perceived util- ity is highest for queries that have short answer (4.62), followed by queries that have only a long an- swer (4.55), and finally by queries that have no long (or short) answer (4.52). Overall, perceived utility decreases as queries require longer-form and less- extractive answers (e.g., factoid NaturalQuestions queries with short answers versus ELI5 queries).
We hypothesize that citation recall is driven by the relevance of retrieved webpages. In the ab- sence of retrieved evidence that directly answers the input user query, systems generate statements that are unsubstantiated by citations, resulting in lower recall. For example, generative search en- gines struggle with citation recall when evaluated on the open-ended AllSouls essay questions (aver- age recall of 44.3), because these queries generally have no extractive answer on the Internet.
4.2 Citation Recall and Precision
Comparing citation precision across query distributions. Precision on NaturalQuestions queries with long answers is higher than non- NaturalQuestions distributions (76.1 vs. 72.3, re- spectively). Precision is highest on NaturalQues tions queries with paragraph answer types (preci- sion of 81.5 when a short answer exists and 78.7 when only long answer exists). On the other hand, citation precision is lowest when systems evalu- ated on AllSouls open-ended essay questions (67.8) and davinci-debate queries (70.3). Comparing be- tween NaturalQuestions subdistributions, average system precision is higher on queries with short answers (77.4) than those with only long answers
See Appendix G for full citation recall and preci- sion results for every generative search engine each of our query distributions.
Existing generative search engines often do not cite comprehensively or correctly. When aver- aging across all systems, mere 51.5% of gener- ated statements are fully supported with citations (recall), and only 74.5% of citations fully support their associated statements (precision). We believe these results are unacceptably low for systems that are quickly becoming a popular tool for answer- ing user queries and already have millions of users, especially given that generated responses often ap-

---

4.6
%ouChat
Is cooperation or competition the driving force guiding the evolution society?
Perceived Utility
perplexity.ai
Bing Chat
(higter chation precision. lower perceheduutin)
4.5
There are different opinions on this topic. Some researchers that cooperation, struggle survival, drives evalution [1]. Others think that there are two driving forces evolution: mutation (change) and competition What do you think?
4.4
Bing Chat
(" ('Some generated atatements may not be fuly aupporied by citation,, whice others are fully supported)
20
40
60
Cited Webpages
Citation F1
[1] te plyy
(0 fully supports to essoeiated statoment) Cooperation, not struggle for survival, drives evolution Researchers balieve compatition tenda differentiate ecological requirements- A new conceptual evolutionrr model ... reviowed the debated mechanism of speciation, suggesting that competition and struggle for existence are not the main drivers of evolution.
Figure 3: Averaged perceived utility plotted against av- eraged citation Fi for each evaluated generative search engine. Different systems make different trade-off's be- tween perceived utility and citation F1. Note that these systems are difficult to directly compare since they may have different abstention rates (Table 1).
M cooperation
fully supports
ststrment)
Game Theory Cooperation cool
Lots biokogiats conaider avalution as a continuous struggle for survival. They think that there are two driving forces of the evolution: mutation (change), competition.
(74.8) or no long answer (73.5).
Figure 4: Citation precision is inversely correlated with perceived utility existing generative search engines. Bing Chat often achieves high citation precision because closely paraphrases from cited webpages (bolded). However, since these citations are largely irrelevant the user's 's input query (biological evolution vs. societal evolution), copying this contents results lower per- ceived utility.
Summary. To summarize our human evaluation results, Figure 3 plots average perceived utility against average citation Fi. Existing systems make different trade-offss between citation recall, citation precision, and perceived utility. See Appendix H for full citation Fi results for every generative search engine on each of our query distributions.
4.3 Citation Precision is Inversely Related to Perceived Utility
This tradeoff is especially apparent on the All- Souls query distribution. which contains open- ended essay questions. AllSouls queries often can- not be answered via extraction from single web- page the Internet. For example, given the query "Is cooperation or competition the driving force guiding the evolution of society?", conventional search engine results focus on biological evolution. rather than societal evolution. Bing Chat simply copies irrelevant statements directly from the cited sources, resulting in high citation precision but low perceived utility (Figure 4).
We find that citation precision is inversely corre- lated with perceived utility in existing generative search engines (r = 0.96). For example, Bing Chat achieves the highest precision, but has the low- est perceived utility. In contrast, YouChat has the lowest citation precision, but its responses attain the highest perceived utility ratings.
This inverse relationship between citation pre- cision and perceived utility is symptomatic of trade-off between faithfulness and abstractiveness (Ladhak al., 2022). In particular, we find that sestem-generated statements often closely para- phrase or directly copy from their associated ci- tations (see $4.4 for further analysis). This results in high citation precision (since extractively copied text is almost always fully supported by the source citation), but lower perceived utility (since the ex- tractive snippets may not actually answer the user's input query). In contrast, systems that frequently deviate from cited content (resulting in low citation precision) may have greater freedom to generate fluent responses that appear relevant and helpful to the user's 's input query.
4.4 Generative Search Engines Closely Paraphrase From Cited Webpages
To better understand how generative search engines use citations to support their responses, we analyze the similarity between generated statements and their supporting cited webpages. For citations that provide full partial support for their associated statement, annotators were asked provide evi- dence by copy-pasting the minimal of sentences from the cited webpage that support their judg- ment (if any such sentences exist). We compute the BLEU (Papineni et al., 2002) and BERTScore

---

BLEU
BERTScore (FI)
(2022) propose the altributed identified sources (AIS) evaluation framework assess whether a par- ticular statement is supported by provided evidence and validate their guidelines on conversational question answering, summarization, and table-to- text systems. Bohnet et al. (2023) introduce the task of attributed question answering, where sys- tems are given input question and must output answer string with pointer evidence text sup- porting the answer, and propose a reproducible eval- uation setup with Natural VatsraoQuestions queries (only paragraph answer type containing long and short answers) with Wikipedia as the evidence corpus.
Bing Chat
44.1
78.8
NeevaAl
30.0
72.9
perplexity.ai
22.3
69.2
YouChat 28.6
72.0
Average
31.3
73.2
Table 2: Existing generative search engines closely para- phrase from cited articles; generated statements have high similarity with their cited webpages.
(Zhang et al., 2020) between each generated state- ment and the annotator-provt amootato-proovided evidence from the associated citation. For statements with multiple as- sociated citations, take the maximum similarity with any associated citation's 's evidence.
In contemporancous work, Peskoff and Stewart (2023) have domain experts evaluate ChatGPT and YouChat responses 100 expert-written questions. They find that generated responses are coherent and concise, but frequently undersourced and inaccu- rate; our results also show that YouChat responses frequently lack citations for generated statements (i.e., low citation recall).
Table presents similarity metries between gen- erated statements and extracted evidence from sup- porting webpages--wen statements are fully or partially supported by their citations, they often copy or closely paraphrase from their cited arti- cles. Furthermore, systems with higher similarity between their generated statements and cited web- pages also have higher average citation precision (r= 0.80 between each of BLEU and BERTScore with average citation precision), indicating that their improved precision may largely be a byprod- uct of their increased tendency to copy or para- phrase from cited webpages.
Conclusion
In this work, we used human evaluation to audit the verifiability of four popular commercial generative search engines-Bing Chat, NeevaA1, perplexity.ai. and YouChat. We find that responses from existing generative search engines are generally fluent and often appear informative, but frequently contain unsupported statements and inaccurate citations (low citation recall and precision) mere 51.5% of generated statements are fully supported by cita- tions (recall), and only 74.5% citations support their associated statements (precision). We believe that existing systems' citation recall and precision are unacceptably low, given that they are quickly becoming popular tool for answering user queries and already have millions of users. Moreover, we find that citation precision is inversely correlated with perceived utility in existing generative search engines-the the responses that seem more helpful are often those with more unsupported statements or inaccurate citations. Analysis suggests that this inverse correlation occurs in existing systems be- cause of their propensity to copy closely para- phrase from cited webpages, which inflates citation precision the cost of lower perceived utility. We hope our results and insights further motivate the development of trustworthy generative search en- gines and help researchers and users better under- stand their current shortcomings.
5 Related Work
Existing work has proposed variety of techniques for building language models that provide refer- ences to support generated text. Nakano et al. (2021) use reinforcement learning from human preferences to train language models to answer questions and provide supporting evidence. Simi- larly, Menick et al. (2022) also use reinforcement learning from human preferences to train language models to answer user questions, but their system generates responses by conditioning on evidence retrieved from a Google search for the given user query. Finally, the LaMDA system of Thoppilan et al. (2022) is trained to provide URLs that sup- port its generated statements. In contrast to the aforementioned line of work on training systems to generate citations, Gao et al. (2022) propose method for post-editing generated output to reflect and cite retrieved evidence.
Existing work has also proposed evaluation pro- tocols and benchmarks for improving verifiabil- ity in language generation systems. Rashkin et al.

---

Acknowledgements
Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roce Aha- roni, Daniel Andor, Livio Baldini Soares, Massimil- iano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui. Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schus- ter, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, and Kellie Webster. 2023. Attributed question answering: Evaluation and modeling for attributed large language models. ArXiv:2212.08037.
We are grateful to the 34 annotators who partici- pated in our human evaluation study-this work would not have been possible without them. We also thank Rishi Bommasani, Ge Gao, Natasha Klein-Atlas, Vivian Lai, Kevin Lin, John Thickstun, Eric Wallace, and Gerben Wierda for feedback and discussions that helped improve this work. We thank Amazon Web Services for providing Ama- zon Mechanical Turk credits that helped support this work. This work was supported in part by the AI2050 program at Schmidt Futures (Grant G-22- 63429).
Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in com- mercial gender classification. In Proc. of FAceT.
Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, and Dan Roth. 2019. Seeing things from different angle: Discovering diverse perspec- tives about claims. In Proc. of NAACL.
Limitations
The primary goal of this work was to assess verifia- bility in generative search engine responses. How- ever, note that verifiability is not factuality-rathee than arbitrating if a generated statement is true (dif- ficult for all but the simplest claims; Rashkin et al., 2022), verifiability enables users to easily check any generated statement's 's source, allowing them to draw their own conclusions about whether to trust the generated statement. Studying the factuality of generative search engines (that may or may not pro- vide citations) is an important direction for future work-use may not necessarily bother to check the sources, especially given that responses often seem helpful and sound confident, and we'd thus like responses to be as factual as possible.
Angela Fan, Yacine Jernite, Ethan Perez, David Grang- ier, Jason Weston, and Michael Auli. 2019. ELIS: Long form question answering. In Proc. of ACL.
Luyu Gao, Zhuyun Dai, Pamupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vin- cent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2022. RARR: Researching and revising what language models say, using language models. ArXiw:2210.08726.
Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslamides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Camphell-Gillnggaall
Jonathan Uesato, Po- Sen Huang, Ramona Comaneseu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soña Mokrá, Nicholas Fernando. Boxi Wu, Rachel Foley, Susannah Young. lason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. 2022. Improving alignment of dialogue agents via targeted human judgements. ArXiv:2209.144 14375.
In our evaluation of verifiability, we consider sentence-level claims. However, sentences often have multiple claims (c.g., "Cats[1] and dogs[2] are common pets."). However, there is currently no clear linguistic definition on what constitutes a claim. As result, we use sentences for simplicity and reproducibility. Proposing a concrete definition of "claim" and performing finer-grained evalua- tion is an interesting direction for future work.
Ben Green and Yiling Chen. 2019. Disparate interac- tions: An algorithm-in innthe--ooot analysis of faimess in risk assessments. In Proc. of FAceT.
Svetlana Kiritchenko and Saif Mohammad. 2018. Ex- amining gender and race bias in two hundred senti- ment analysis systems. In Proc. of *SEM.
References
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti. Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang. Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slay Petrov. 2019. Natu- ral questions: benchmark for question answering research. Transactions of the Association for Compu- lational Linguistics, 7:452-466.
Michiel Bakker, Martin Chadwick, Hannah Sheahan, Michael Tessler, Lucy Camppell- Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matt Botvinick, and Christopher Sum- merfield. 2022. Fine-tuning language models to find agreement among humans with diverse preferences. In Proc. of NewrIPS.
Abeba Birhane, Vinay Uday Prabhu, and John Whaley. 2022. Auditing saliency cropping algorithms. In Proc. of WACV.
Faisal Ladhak, Esin Durmus, He He, Claire Cardie, and Kathleen McKeown, 2022. Faithful or extractive? on

---

mitigating the faithfolness-abstraciit abstractiveness trade-off in abstractive summarization. In Proc. of ACL.
Amin Ghafouri, Marcelo Menegali. Yanping Huang. Maxim Krikun, Dmitry Lepikhin, James Qin. Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung- Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-HHllsterm, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Ale- jandra Molina, Erin HoffimanJoonn Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Co- hen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoe Le. 2022. LaMDA: Language models for dialog ap- plications. ArXiv:2201.0822
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factual- ity abstractive summarization, Proc. of ACL.
Yusuf Mehdi. 2023. The new Bing and Edge - progress from our first month Bing search blog, Accessed on March 28, 2023.
Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslamides, Francis Song, Martin Chadwick, Mia Glaese Susannah Young, Lucy Campbell- Gillingham, Geoffrey Irving. and Nat McAleese. 2022. Teaching language models to support answers with verified quotes. ArXiv:220..1117..
Danaë Metaxa, Joon Sung Park, James A. Landay. and Jeff Hancock. 2019. Search media and elections: longitudinal investigation of political search results. In Proc. of CSCW.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT, In Proc. of ICLR.
Panagiotis Takis Metaxas and Yada Pruksachatkun. 2017. Manipulation of search engine results during the 2016 us congressional elections.
Reichirr Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2021. We- bGPT: Browser-assisted question-answeriin with hu- man feedback. AtXiv:2112.09332.
Kishore Papineni, Salim Roukos. Todd Ward, and Wei- Jing Zhu. 2002. BLEU: method for automatic eval- uation of machine translation. In Proc. of ACI..
Denis Peskoff and Brandon Stewart. 2023. Credible without credit: Domain experts assess generative language models. In Proc. of ACL.
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis. Majid Yazdani, Nicola De Cao, James Thome, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proc. of NAACL.
Hamnah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slay Petrov, Gaurav Singh Tomar, Iulia Ture, and David Reitter. 2022. Measuring attribution natural lan- guage generation models. ArXiv:21 12870.
Ronald E. Robertson, David Lazer, and Christo Wilson. 2018. Auditing the personalization and composition of politically--elated search engine results pages. In Proc. of www.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apporv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng,

---

A
Example queries from each evaluated query distribution
Source
Example Queries
What are the functions of fashion?
AllSouls
Should wealth be inheritable?
Should private companies be allowed to manage public utilities?
davinci-debate
Should controversial opinions be censored on social media?
Why is a circle 360 degrees and not 100 degrees? Why can animals drink dirty water safely but humans can't '1?
ELIS (KILT)
Why jumping into water from great height feels landing concrete? where does deleted data go
EL15 (Live)
age paper using ways stop stressing over exam results
Keyworss
NaturalQuestions
who wrote the song god your mama and me what is the queen of spain's 's name
(paragraph long answer, has short answer)
NaturalQuestions
where did knock on wood superstition come from what is the use of tap and dic
(paragraph long answer, no short answer)
Natural Queutions
what is the most nominated film for the oscars who played guitar on i want you she's so heavy
(list long answer, has short answer)
NaturalQuestions
alicia keys if ain't got you awards of florida the same time zone
(list long answer, no short answer)
NaturalQuestions
how many episodes there in quantum leap what kind music red hot chili peppers
(table long answer, has short answer)
NaturalQuestions
where does copa airlines fly in the united states michael jordan career high against every nba team
(table long answer, short answer)
NaturalQuestions
what does the x card mean uno what changes were made when trinidad and tohago gained independence
(no long or short answer)
Table 3: Example queries from each of the evaluated query distributions. Queries come from diverse sources and require knowledge from a variety of answer types (e.g., short text span, long-form paragraph, list, or table). Each system is evaluated 1450 queries-Ba 150 randomly-samea sampled queries from each of AllSouls, dabanci-debate, ELIS (KILT), ELI5 (Live), and WikiHooweyyoorss, and 100 randomly-sample queries for each of the seven NaturalQuestions subdistributions.
Query Distribution Details
B.1
davinci-debate
We seed the data generation process with 100 debate questions, which are manually transformed propo- sitions propositions taken from the Perspectrum dataset of Chen et al. (2019) (e.g., the proposition "Vaccination must be made compulsory." could be rewritten as the question "Should vaccines be manda- tory?").
To generate debate question, we prompt text-davinci-003 with 10 randomly-sampled seed ques- tions. We repeat this procedure until we have generated 150 unique debate questions that also do not appear in seed set. Finally, generated questions were manually filtered for inappropriate content.
B.2 EL15
To transform EL15 post titles into queries, remove ELI5-specific prefixes (e.g., the post title "ELI5: why can't our brains recall every memory?" becomes the query "Why can't our brains recall every memory?").
B.3 WikiHow
To paraphrase article titles into keyword queries, we prompt text-davinci-003 with "Given a question, write a concise Google search query that would answer the question" and two in-context examples.

---

C Annotation Interface
Figures 5-7 show the annotation interface used for human evaluation.
In the first step, annotators were shown the query and the generated response (without citations) and asked to rate response fluency and pereeived utility on five-point Likert scale.
In the second step, annotators were shown the statements in the generated response (including any generated citations) and asked to filter out statements not verification-wortthy
Finally, in the third step, annotators verification (in the prior step), well as each statement's associated each statement and associated citation, annotators judged whether the citation fully supports, partially supports, or does not support the statement, as interpreted within the broader context of the query and system response. For statements with multiple associated citations, annotators are asked to judge whether the citations, when taken together, fully support the statement; this captures cases where multiple citations support disjoint parts of a statement (e.g., "Health benefits of cycling include improved cardiovascular health[1] and lowered cholesterol levels[2].").
shown the statements that previously judged require
citations. For
User Query and System Response
Query: recycling
mandatory?
mandatory. Recycling helps problerns energy. reducing landfills and conserving natural resources. also has economic benefits, recycling help reduce pollution and reduce greenhouse gas emisssions.
Response: recycling
reduce management Additionally,
response
Strongly Disagree
Disagree
Agree
Strongly Agree
response helpful and informative answer to query.
Strangly Disagree
Disagree
Agroo
Strongly Agroe
Figure First step of the annotation interface, where annotators judge response fluency and perceived utility.

---

User Query and System Response
Yea, recycling should made mandatory.
Encycling helps reduce climate problems and saves energy (1), while reducing the need for landfills and conseniing natural resources.
also has economic benefite, as can create jobs and reduce waste management costs.
Additionally,, recycling can help reduce air pollution and reduce greenhouse gas emissions [2]
Figure 6: Second step of the annotation interface, where annotators uncheck statements that are not verification- worthy. Statements that contain generated citations must be verificationwoorlyy automatically mark them such the interface (greyed-out checkboxes next to the 2nd and 4th sentences above).
User Query and System Response
Query: Should recycling be made mandatory?
Response (with citations): Yes, recycling should be made mandatory. Recycling helps reduce climate problems and saves energy [1], while reducing the need for landfills and conserving natural resources, it also has economic banefits, as it can create jobs and reduce waste management costs. Additionally, recycling can help reduce air pollution and reduce greenhouse gas emissions [2].
Yes, recycling should be made mandatory.
Sentence has associated citations, fool free
Recycling helps reduce climate problems saves energy [1], while reducing the need for landfills conserving natural resources.
(1) wwwinphhycom////eeyllg
fully support
above?
also has economic benefits, as can create jobs and reduce waste management costs.
Sentence has na associated citations, feel free to to the next sentence.
Additionally, recycling help reduce air pollution and reduce greenhouse gas emissions [2].
Does citation (2) calrecydle gov. /commeroal) fully support the entire sentence above?
Figure 7: Third step of the annotation interface, where annotators provide judgments whether each citation supports its associated statement, and whether each statement is supported by the union of its citations (only applicable statement has multiple associated citations).

---

D Annotation Guidelines
Figures 8-12 show the annotation guidelines we used for the task. We ask crowd annotators to read these guidelines as part of the qualification study. Only annotators that demonstrated a thorough understanding of the guidelines and task were permitted to participate in the main round of human evaluation.
In this task, you will evaluate Al system's response to user query. The Al system outputs paragraph contains information relevant the user's query, and we would to evaluate whether the Al system can accurately cite sources statements it makes about the external world.
a high level, this task breaks down into three steps:
1. Evaluating response quality
Filtering sentences that require citation.
3. Judging whether each statement fully supported by its citation(s).
Please carefully read the guidelines below before starting the task. The task compensation accounts for the time needed to guidelines.
Preliminaries: Logging In
When entering site, you will be prompted select a username. Please use your worker ID as the username, so we can keep track of the examples you've annotated. The top of the interface displays worker ID, the total number of examples submitted from this completion code when you finished the task.
username, will
If something is wrong with the example, you may press the "Flag Example" button in the top-right corner to report the error. Please do not submit annotations for such examples.
Your task ends after you've completed 5 responses. A completion code will appear at the top of the thterface--there is no need to complete more than 5 responses to receive credit for the study.
Step 1: Evaluating response quality
You will shown the user's original query, and system's response query-pleasee carefully read both them. Then, you will be asked rate your level of agreement with two questions:
cohesive. 2. The response is a helpful and informative answer to the query.
The response
Figure First page annotation guidelines.

---

Once you have finished selecting a response for each of the two questions, press the "Next Step" button in the top-right corner to continue.
Step 2: Filtering sentences that do not require citation.
The goal of this step is to filter the sentences in the system respanse removing sentences that require citation (unchecking them the interface). We expect the majority of sentences produced by the system to require citation, so don't worry if you find yourself rarely unchecking sentences.
general, we take position even if they are trivially true or "common sense" (since users may differ in their background, which affects their basic beliefs). For example, the following sentences require citation:
statements about external world require citation,
(1a): The House of Lords a topic of ongoing debate the
(1b): However, there is still no consensus on what should replace the Electoral College.
(1c): The sky is blue.
(1d): The moon landing was staged.
(1e): In February 2023, LeBron James took 26 1,960 total breaths.
(1f): Patrick Henry once said "Give me liberty, or give me death".
(1g): Thanksgiving dinners usually taste bad.
(1h): Voting rights are controversial
In particular, note that sentences require citation despite being nearly impossible to verify. Consider example (e) above. It's highly unlikely that anyone knows exactly many breaths LeBron James took in February 2023, let alone that such information could be linked to in a citation. However, it's still statement about the external world, and it's still possible to find out for certain whether the statement is true false. Thus, the statement requires citation.
In contrast, consider the following examples of sentences that do not require citation:
(2a): believe that the moon landing was staged.
Explanation: In general, sentences pertaining to "I" do not require citation.
This statement expresses a belief held by speaker. The speaker is unknown,
so this statement does not require citation. Note that the similar-looking statement "The moon landing was staged" (example 1d) require citation and is verifiable.
(2b): Have you listened that song?
Explanation: Questions do not have information to verify.
(2c): Pick up the ball on the floor.
Explanation: Commands
information verify.
(2d): It is the year 2300. Robots rule the earth.
Figure 9: Second page of the annotation guidelines.

---

sentence "Robots rule the earth." does not require citation, since the context ("It is the year 2300")) specifies that this is hypathetical situation and not statement about the external world.
Explanation:
Carefully read each sentence again and decide whether it requires citation. does not require citation, uncheck its corresponding checkbox. When you have finished, press the "Next Step" button in the top-right corner to proceed.
Step 3: Judging whether each statement is fully supported by its citation(s).
this step, you will evaluate whether each statement supported by corresponding citations. Note that the system responses may appear very fluent and well-formed, but contain slight inaccuracies that are not easy to discem at first glance. Pay close attention to the text. Read carefully as you would when proofreading.
Carefully read the user query and statement. You may also have re-read the full system response to understand the statement in its full context. Given the statement's associated citations, your task is to judge whether all of the information provided the system response is fully supported by the source document.
In particular, this question can be answered by considering: What the information provided the statement? (B): According to citation(s). statement true?
(A): What is the information provided by the statement?
determine the information provided by the statement, you must consider the query, the statement, and the context of the statement within the full response. The citations should be completely ignored when determining "the information provided the statement."
Consider the following example:
Query: Why do many people want to get married?
Response (statement highlighted): People get married for many reasons, including love, companionship. financial security, and to share their lives with partner. Marriage can also be seen way to affirm mutual love or start family.
this case; the statement is stand-alone, and can interpreted without looking at query or the rest of the response. However, this is not always the case. Consider the example below:
Query: wrong to exaggerate a letter recommendation?
Figure 10: Third page of the annotation guidelines.

---

Response (statement highlighted): Yes, it is wrong, Letters of recommendation should reflect the author's honest perspective on the candidate.
The response "Yes, it wrong" is uninterpretable refers However, by using context the query, becomes clear the statement equivalent to "Yes, (exaggerating in letter of recommendation] is wrong".
own, because it is not clear what "it"
For another example, consider:
Query: how many characters are the prologue of canterbury tales Response (statement highlighted): Geoffrey Chaucer's The Canterbury Tales, 32 characters make the journey to Canterbury. This includes the narrator, the host, and the Canon's s yeoman, who join the group later.
The statement 'This includes narrator, host, and Canon's yeoman, who join group later." is uninterpretable on its own, because it not clear what "This" refers to, or what "group" they join. The preceding sentence of the response is essential for realizing that this sentence is equivalent "[The 32 characters that make the journey to Canterbury] include the narrator, the host, the Canon's yeoman, who join the (32 characters] later".
In general, use best judgment to determine the information provided by the system response.
(B): According to the citation(s), is this statement true?
Again, you should your best judgment in determining whether all of information provided by the statement is supported by the associated citation(s).
It may be helpful to ask yourself whether it is accurate say "according the citation" with statement following this phrase. For example, it accurate say "according to citation. in Geoffrey Chaucer's The Canterbury Tales, 32 characters make the journey to Canterbury"?
Be check all of information the statement. You will be given options:
"Full Support": All of the information the statement is supported in document.
"Partial Support": Only some of the information is supported in the document, but other parts of the information
missing from the document.
Support": This document does not support any part the statement. "Article Not Accessible": :: Not able to access the document (e.g.. paywall or the link dead)
The citation has information that supports statement, but also has information that refutes the statement. "Statement is Unclear, Can't Make Judgment": The statement is so incomprehensible that it cannot be determined the citation supports the statement.
"Citation Has Support but also Refutes Statement":
Figure 1I: Fourth page of the annotation guidelines.

---

If the citation offers "full support" or "partial support" of document, you will also be asked capy and paste the minimal set of sentences from the article that support your judgment. cases where you can't localize the judgment particular sentence(s) (e.g., entire article supports the statement, or the support comes from an image or graphic), feel free to leave this input blank.
When statement has more than one associated citation. you will also judge whether the citations. when taken logether, fully support the statement (Yes/No). In other words. if you merged all these citations into big webpage (and became single citation), would this citation fully support statement? the citations contradict each other (e.g.. ane fully supports the statement, whereas another refutes the statement), then select "Citations Contradict Each Other".
Figure 12: Fifth page of the annotation guidelines.

---

E Annotation Quality
Table presents inter-annotator agreement statistics, computed on a random sample of 250 query-response pairs that received annotations each. We measure the pairwise agreement between individual pairs of ratings and an Fl score comparing individual ratings to the majority consensus. We compute agreement on judgments of (i) fluency and perceived utility, (ii) whether statement is verification-worthy (iii) whether a citation supports its associated statement, and (iv) whether a statement is fully supported by the union of its citations (in the case where multiple webpages and perceived utility judgments, we coarsen the 5-point Likert judgments into three options: "Disagree", "Neutral", and "Agree". Agreement rates between annotators are high (pairwise agreement greater than 82.0% and Fl greater than 91.0 for all judgments).
cited). When caleulating agreement fluency
Inter-Anmotator Agreement (1)
Pairwise Agreement
Fluency Perceived Utility
88.5
94.1
86.4
93.1
Verifiability
94.6
97.3
Citation Supports
82.0
91.0
Statement Supported
82.2
91.1
Table 4: Inter-annotator agreement stafisties. Pairwise Agreement % computes the proportion of individual
judgment pairs that agree, and F1 compares individual judgments to the majority consensus judgment. Inter- annotator agreement high (greater than 82.0% pairwise agreement % and 91.0 F1 for all judgments).

---

F Fluency and Perceived Utility
Table 5 presents the fluency of generative search engine responses on each of our query distributions, and
Table 6 presents the perceived utility.
Fluency (1)
Fluency (1)
Average All Queries
ELI5
AllSouls
davinci-debate
WikiHowKeywords
KILT
Live
Bing Chat
4.40
Bing Chat
4.31
4.37
4.36
4.30
4.41
NeevaAl
4.43
NeevaAl
4.50
4.53
4.50
4.42
4.42
perplexity.ai
4.51
perplexity.ai
4.43
4.54
4.55
4.47
4.45
YouChat
4.59
YouChat
4.58
4.65
4.56
4.53
4.52
Average
4.48
Average
4.45
4.52
4.49
4.43
4.45
Fluency (t)
Natural
List Long Answer
Table Long Answer
Paragraph Long Answer
No Answer
Has Short
Short
Has Short
Has Short
Short
Bing Chat
4.49
4.52
4.46
4.30
4.54
4.41
4.39
NeevaAl
4.45
4.40
4.31
4.28
4.41
4.49
4.43
perplexity.ai
4.69
4.54
4.59
4.41
4.73
4.43
4.37
YouChat 4.65
4.56
4.60
4.45
4.66
4.69
4.64
Average
4.57
4.50
4.49
4.36
4.58
4.50
4.46
Table 5: Human evaluation results for generated response fluency (five-point Likert ratings). general, existing generative search engines produce fluent text. Performance is notably lower on NaturalQuestions queries with table-type long answers and no short answers, which often require aggregating information within or across citations.

---

Perceived Utility (1)
Perceived Utility (1)
Average Over All Queries
E1.15
AllSouls
davinci-debate
Keywors
KILT
Live
Bing Chat
4.34
Bing Chat
4.15
4.19
4.19
4.09
4.37
NeevaAl
4.48
NcevaAl
4,44
4.39
4.54
4.46
4.42
perplexity.ai
4.56
perplexity.ai
4.39
4.60
4.54
4.50
4.51
YouChat
4.62
YouChat
4.53
4.54
4.53
4.50
4.63
Average
4.50
Average
4.38
4.43
4.45
4.39
4.48
Perceived Utility (t)
NaturalQuections
Long Answer
Table Long Answer
Paragraph Long Answer
No Answer
Has Short
No Short
Has Short
No Short
Has Short
No Short
Bing Chat
4.63
4.49
4.49
4.47
4.53
4.40
4.38
NeevaAI
4.65
4.57
4.43
4.38
4.43
4.63
4.49
perplexity.ai
4.71
4.61
4.60
4.55
4.77
4.58
4.50
YouChat
4.72
4.64
4.70
4.54
4.77
4.77
4.70
Average
4.68
4.58
4.55
4.49
4.62
4.60
4.52
Table 6: Human evaluation results for perceived utility of generated responses (five-point Likert ratings). In general, responses from existing generative search engines appear informative and useful.

---

G Citation Recall and Precision
Table 7 presents generative search engine citation recall across the evaluated query distributions, and
Table 8 presents citation precision.
Citation Recall (%; f)
Citation Recall (%;
Average All Queries
ELI5
AllSouls
davinci-debate
WikiHowKeywords
KILT
Live
Bing Chat
58.7
Bing Chat
55.6
57.1
59.8
59.9
50.7
NeevaAl
67.6
NeevaAl
55.3
66.3
66.6
61.6
72.5
perplexity.ai
68.7
perplexity.ai
63.0
64.2
64.8
58.1
74.6
YouChat
11.1
YouChat
3.2
3.9
3.0
4.6
12.1
Average
51.5
Average
44.3
47.9
48.5
46.0
52.5
Citation Recall (%;; f)
Naturalluestions
List Long Answer
Table Long Answer
Paragraph Long Answer
No Answer
Has Short
Has Short
Has Short
Short
Bing Chat
74.1
60.6
63.5
49.2
72.1
66.3
61.9
NeevaAl
73.0
64.2
69.5
65.1
75.0
74.8
65.6
perplexity.ai
85.3
74.4
79.6
62.4
84.9
75.9
68.4
YouChat 21.6
16.6
30.6
11.5
31.6
21.8
17.8
Average
63.5
53.9
60.8
47.1
65.9
59.7
53.4
Table 7: Human evaluation results of citation recall existing generative search engines. Citation recall is concerningly low (many generated statements are not fully supported by citations), especially given that these systems already have millions users and may serve as primary tool for fulfilling user information needs.

---

Citation Precision (%;; 1)
Citation Precision (%;; 1)
Average Over All Queries
E1.15
AllSouls
davinci-debate
WikiHowKeywords
KILT
Live
Bing Chat
89.5
Bing Chat
88.8
88.8
87.6
87.2
92.1
NeevaAI
72.0
NcevaAl
69.8
74.1
75.7
73.8
74.0
perplexity.ai
72.7
perplexity.ai
61.7
68.4
64.9
66.3
77.4
YouChat
63.6
YouChat
51.1
50.0
64.7
57.9
71.1
Average
74.5
Average
67.8
70.3
73.2
71.3
78.7
Citation Precision (%; 1)
NaturalQuestionn
List Long Answer
Table Long Answer
Paragraph Long Answer
No Answer
Has Short
Short
Has Short
Short
Has Short
Short
Bing Chat
86.8
86.8
89.0
92.5
92.9
91.3
90.8
NeevaAI
73.2
67.6
67.1
64.2
73.4
76.5
70.8
perplexity.ai
82.1
81.0
76.0
71.7
83.8
79.7
74.0
YouChat
63.3
62.7
64.8
56.1
75.7
67.5
58.6
Average
76.4
74.5
74.2
71.1
81.5
78.7
73.5
Table Human evaluation results of citation precision in existing generative search engines. Citation precision is concerningly low (many generated citations do not support their associated statements), especially given that these systems already have millions users may serve as primary tool fulfilling user information needs.

---

H
Citation Fi
Table 9 presents the citation Fi for every evaluated generative search engine on each query distribution.
Citation Pi (1)
Citation
Average Over All Queries
ELI5
AllSouls
davinci-debate
WikiHowKeywords
KILT
Live
Bing Chat
70.9
Bing Chat
68.4
69.5
71.1
71.0
65.4
NeevaAl
69.8
NeevaAI
61.7
70.0
70.8
67.1
73.2
perplexity.ai
70.6
perplexity.ai
62.3
66.2
64.8
62.0
76.0
YouChat
18.9
YouChat
6.0
7.2
5.6
8.5
20.7
Average
57.6
Average
49.6
53.2
53.1
52.2
58.8
Citation Fi (f)
NaturalQuestions
List Long Answer
Table Long Answer
Paragraph Long Answer
No Answer
Has Short
Short
Has Short
Short
Has Short
Short
Bing Chat
79.9
71.4
74.1
64.2
81.2
76.8
73.6
NeevaAl
73.1
65.9
68.3
64.6
74.2
75.7
68.1
perplexity.ai
83.7
77.5
77.8
66.7
84.3
77.7
71.1
YouChat
32.2
26.2
41.5
19.2
44.6
32.9
27.4
Average
67.2
60.2
65.4
53.7
71.1
65.8
60.0
Table 9: Citation Fi of generated responses.


Reviews:
[{"review_id": "ah3kz7SBIZ", "reviewer": "Reviewer_qPBo", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful review. See below for some inline responses.\n\n## Clearing up how we calculate citation precision / recall\n\nWe think there might be a misunderstanding about how we define and compute the citation precision and recall metrics. Citation precision and recall are designed to behave exactly as you mention---in a nutshell, citation recall is 1.0 when every sentence is fully supported by their associated citations, and citation precision is 1.0 when every sentence contributes support for its associated sentence. Our metrics are calculated for a single response, but they rely on judgements on individual **sentences** in that response and each sentence's corresponding citations---the evaluation is sentence-level, but aggregated to give a response-level score.\n\n> Weaknesses in the definition of citation precision: although all citations for a single response are evaluated by a single worker, the citations are evaluated individually rather than holistically.\n\nWe aren't quite sure what you mean by holistically, but note that we ask for 2 types of judgments in calculating citation precision:\n- For each _citation_, we ask if it (a) fully supports, (b) partially supports, or (c) does not support its associated sentence (L214 - L226)\n- For each _sentence_, we ask if it is (a) fully supported or (b) not fully supported by **the union of its associated citations** (L227 - L233)\n    - We believe that this is what you might be asking for when you say \"holistically\"\n\n> For instance, this does not take into account the redundancy of citations. Let's say a response has 4 aspects that need to be verified and 4 citations that all are duplicates of the same Web page. Although these citations verify only one aspect of the response, they will result in perfect citation recall and precision, according to the definitions of these metrics proposed in the paper.\n\nOur metric does take into account this redundancy. To make your example above concrete, let's say that we have a response with the following 4 sentences, and there are 4 citations that are all duplicates of the same webpage:\n\n- Sentence 1: Mice eat cheese [1].\n- Sentence 2: Mice eat bread [1].\n- Sentence 3: Mice drink coffee [1].\n- Sentence 4: Mice drink water [1].\n\nLet's say that citation [1] simply states \"Mice eat cheese\". Thus, it fully supports sentence 1, but provides no support for sentences 2, 3, 4.\n\nLet's first calculate citation recall. At a high level, citation recall asks: \"what proportion of generated sentences are fully supported by their associated citations?\". All 4 sentences here that are verification-worthy (L175). Now, for each of the 4 sentences, we evaluate \"Evaluating whether a verification-worthy sentence is fully supported by (the union of) its associated citations\". In this case, clearly only Sentence 1 is fully supported by its associated citation [1], while Sentences 2, 3, 4 are not. **Thus, since there is only 1 sentence that is fully supported by the union of its associated citations, the citation recall is 1/4 = 0.25, not 1.0**\n\nNow, let's calculate citation precision. At a high level, citation precision asks: \"when you output a citation, what % of the time does that citation actually support the associated sentence?\" To measure citation precision, we first ask annotators to rate, **for every outputted citation**, whether it fully, partially, or does not support its associated sentence. To be concrete, there are 4 outputted citations here: (a) [1] for sentence 1, (b) [1] for sentence 2, (c) [1] for sentence 3, (d) [1] for sentence 4 (since it's all just the same webpage). Of these 4 citations, only (a) ([1] for sentence 1) fully supports the associated sentence. The other 3 citations do not support their associated sentence. Thus, according to the definition of the metric in L234, $T_{fs} = 1$, $T_{ps} = 0$ (the partial support case does not apply here, since no citation partially supports its associated sentence), and $N = 4$ (since we outputted 4 citations). **Thus, precision is (1+0)/4 = 0.25, not 1.0**\n\n> More fine-grained definitions would be more accurate.\n\nOur evaluation is done at the sentence-level---there is currently no linguistic consensus on a definition for a sub-sentential \"claim\" or \"statement\", so we used sentences for reproducibility.\n\n> For instance, a response can only result in a citation recall of 1 if all aspects of it are covered by citations. Duplicate citations that partially support a response should not be counted towards precision.\n\nThis is indeed how the metric is defined and designed.\n\n> according to the definition in section 2.4, the citation precision for the first response in Figure 2 should be 5/8, not 3/8\n\n**No, actually, according to the definition in section 2.4, the citation precision is indeed 3/8**. Here's a walk-through of the calculation: there are 8 outputted citations in total (3 for the first generated statement, 3 for the second generated statement, and 2 for the third generated statement), so $N = 8$. Now, we need to calculate $T_{fs}$, the proportion of citations that _fully support_ their associated statements:\n\n- Statement 1: citation [1] fully supports, citation [2] does not support, and citation [3] partially supports. So, we add 1 to $T_{fs}$ to get $T_{fs} = 1$\n- Statement 2: citation [1] fully supports, citation [2] does not support, and citation [4] does not support. So, we add 1 to $T_{fs}$ to get $T_{fs} = 1 + 1 = 2$\n- Statement 2: citation [4] fully supports, citation [5] partially supports. So, we add 1 to $T_{fs}$ to get $T_{fs} = 1 + 1 + 1 = 3$.\n\nSo, in total, $T_{fs} = 3$. Now, let's calculate $T_{ps}$. As a reminder, we only count a citation toward $T_{ps}$ if (condition 1) it partially supports the associated statement, (condition 2) the associated statement is fully supported by the union of its associated citations, and (condition 3) the citation associated with this statement singlehandedly fully supports this statement. This might seem convoluted, but for an intuitive explanation, see footnote 4 on page 4 of the paper---the tl;dr is that we don't want to penalize systems for (correctly) aggregating information across multiple webpages. Ok, so let's calculate $T_{ps}$. As a reminder, all 3 conditions above need to hold to count a citation toward $T_{ps}$:\n\n- Statement 1: citation [3] partially supports (condition 1), but citation [1] fully supports the statement by itself (fails condition 2). Thus, citation [3] is actually redundant here---there's no point to having it, since [1] already covers it. So, we don't give the system credit for outputting this citation. $T_{ps} = 0$\n- Statement 2: None of the citations partially support, so $T_{ps} = 0$ remains 0.\n- Statement 3: citation [5] partially supports (condition 1), but citation [4] fully supports the statement by itself (fails condition 2). Thus, like in statement 1, citation [5] is actually redundant here---there's no point to having it, since [4] already covers it. So, we don't give the system credit for outputting this citation. $T_{ps} = 0$\n\nSo, we have that $T_{ps} = 0$. **Thus, precision = $\\frac{T_{fs} + T_{ps}}{N} = \\frac{3 + 0}{8}$ = 3/8**\n\n### Other comments:\n\n> Some far-fetched and misleading conclusions: it seems quite counter-intuitive to me that close paraphrasing or copying of the source necessarily leads to lower fluency and perceived utility.\n\nNote that just because a claim is counter-intuitive does not make it \"far-fetched and misleading\"---we also found these results to be somewhat surprising.\n\n> It is completely unclear why paraphrasing leads to lower fluency (the response of Bing Chat in Figure 3 looks perfectly fluent to me).\n\nIndeed, the correlation with fluency is weaker, and fluency is high overall among all systems. In the camera-ready version of the paper we're planning on focusing only on the relationship between precision and perceived utility and cutting the 3 other comparisons.\n\n> The relationship with perceived utility is indirect and rather depends on the relevance of the retrieved source text rather than whether it was copied or not.\n\nThe (ir)relevance of the retrieved source text directly affects citation precision (increasing it) and perceived utility (decreasing it) when irrelevant text is copied. To be concrete, take Figure 3. The query asks about societal evolution, but one of the retrieved source texts is irrelevant because it talks about biological evolution (citation [1]). Nonetheless, the system copies directly from this citation. Since statement 1 is directly copied from the citation [1], citation [1] is almost guaranteed to fully support statement 1, which increases precision. However, since [1] is irrelevant to the query, this decreases perceived utility. This copying behavior is shown in a variety of responses (Section 4.4 / Table 2), and leads to the observed negative correlation between citation precision and perceived utility.\n\n> This problem is caused by the authors not providing any details or examples of what \"fluent\" and \"cohesive\" response is in the annotator guidelines.\n\nWe're not sure that providing details or examples of what constitutes fluency / cohesive would have fixed this problem---collecting Likert scale judgments for generated text fluency and utility is a fairly standard practice.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* No other existing studies of generative search engines\n* Findings that may help improve the quality of responses by generative search engines", "reasons_to_reject": "* Weaknesses in the definition of citation precision: although all citations for a single response are evaluated by a single worker, the citations are evaluated individually rather than holistically. This may lead to problems. For instance, this does not take into account the redundancy of citations. Let's say a response has 4 aspects that need to be verified and 4 citations that all are duplicates of the same Web page. Although these citations verify only one aspect of the response, they will result in perfect citation recall and precision, according to the definitions of these metrics proposed in the paper. More fine-grained definitions would be more accurate. For instance, a response can only result in a citation recall of 1 if all aspects of it are covered by citations. Duplicate citations that partially support a response should not be counted towards precision.\n* Some far-fetched and misleading conclusions: it seems quite counter-intuitive to me that close paraphrasing or copying of the source necessarily leads to lower fluency and perceived utility. It is completely unclear why paraphrasing leads to lower fluency (the response of Bing Chat in Figure 3 looks perfectly fluent to me). The relationship with perceived utility is indirect and rather depends on the relevance of the retrieved source text rather than whether it was copied or not. This problem is caused by the authors not providing any details or examples of what \"fluent\" and \"cohesive\" response is in the annotator guidelines.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper reports the results of a human audit of the verifiability of responses by popular generative search engines - Bing Chat, NeevaAI, perplexity.ai and YouChat. The authors define new recall and precision metrics for the citations produced by generative search engines. The key finding is that, despite generally high fluency and perceived utility of generative search engine responses, only 51.5% of them are fully supported with citations, whereas 74.5% of citations actually support their associated responses. A secondary finding is that citation recall and precision are inversely correlated with fluency and the perceived utility of responses.", "excitement": 2, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "according to the definition in section 2.4, the citation precision for the first response in Figure 2 should be 5/8, not 3/8\nthe authors should clearly associate numeric scores with different levels of the qualitative Likert scale for fluency and perceived utility, instead of having the readers guess"}}, {"review_id": "cZiMjBOBub", "reviewer": "Reviewer_XbQo", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and thoughtful comments. See below for some inline responses:\n\n> The reason to reject this paper is that the authors fail to provide a clear and convincing explanation for why they consider the results obtained on citation precision and recall metrics as unacceptable for Generative Search Engines. Without a solid justification, the claims made in the paper lack credibility and undermine the significance of the research.\n\nWe will add additional discussion to the paper. To recap, our human evaluation indicates that, \"when averaging across all systems, a mere 51.5% of generated statements are fully supported with citations (recall), and only 74.5% of citations fully support their associated statements (precision)\" (lines 465-469 of the paper).\n\nWe argue that these citation precision and recall numbers are unacceptable for systems that are quickly being deployed to users and have the responsibility of keeping them informed---given that we know that these systems hallucinate and output incorrect statements, it's difficult to trust these systems knowing that only 51% of statements can be cross-referenced with accurate citations, and that 25% of citations flat-out do not support their associated claims.\n\nIn fact, Bing.com recently started to display generated output to users of their conventional search engine by default, massively broadening its usage and exposure. The overwhelming majority of these users have no expertise with language models and thus have no understanding of where they might fail---they see informative-looking text on the internet (with citations, generally a hallmark of well-researched response) and automatically assume that the citations support their corresponding statements.\n\nOur work quantifies generative search engine trustworthiness (via citation precision and recall), which enables researchers to track and improve these systems.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Metric Introduction: The authors introduce novel evaluation metrics specifically designed for assessing the quality of answers generated by Generative Search Engines (GSEs). These metrics are essential for accurately measuring and understanding the performance of GSEs in providing verifiable answers.\n2. Human Annotations Resource: The paper provides a valuable resource by offering human annotations that assess the quality of answers based on four different metrics. This dataset enhances the research community's understanding of GSEs' capabilities and limitations.\n3. Comprehensive Analysis: The authors thoroughly analyze the results of their study, revealing that the answers produced by GSEs exhibit low performance on citation recall and precision metrics. Furthermore, their investigation demonstrates a negative correlation between these metrics and the aspects of fluency and perceived utility in the answers. This analysis yields important insights into the trade-offs between different performance aspects of GSEs' answer generation.\n\nOverall, the introduction of novel evaluation metrics, the provision of a human annotations resource, and the comprehensive analysis of the study's findings make this paper a valuable addition to the field of natural language processing and worthy of acceptance.", "reasons_to_reject": "The reason to reject this paper is that the authors fail to provide a clear and convincing explanation for why they consider the results obtained on citation precision and recall metrics as unacceptable for Generative Search Engines. Without a solid justification, the claims made in the paper lack credibility and undermine the significance of the research.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper delves into the issue of verifiability concerning answers produced by Generative Search Engines (GSEs). The primary focus is on analyzing the current search engines' performance using citation recall and citation precision as evaluation metrics, which are notable contributions of the paper. Additionally, the authors undertake a human evaluation of four search engines, revealing instances where the answers provided include unsupported or inaccurate citations. Furthermore, the paper explores the relationship between fluency and perceived utility, which are two additional performance metrics, and their inverse correlation with the citation recall and precision metric. This observation sheds light on the trade-offs between these aspects of GSE-generated answers. Finally, the authors make their human evaluation annotations accessible.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FSWwlw0wQz", "reviewer": "Reviewer_hMXQ", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for taking the time to review our paper. See below for some inline responses:\n\n> These conclusion may be outdated quickly due to the fact that these generative search engines being tested may updated quickly.\n\nOur goal was not to make conclusions about the absolute performance of these systems, but just to show that these systems (in their current instantiation) do not cite reliably and to provide an evaluation blueprint for measuring and tracking progress in this space.\n\n> Due to the limited access, I understand we know very little about the model details. That being said, I personally would love to see more explanations (or educated guesses) about these models and their performances.\n\nSimilar to the point above, we wanted to avoid speculating about what's going on underneath the hood of these systems---the point of the evaluation numbers is less to make concrete recommendations about model development (since we don't know how these models work), but to make a sociotechnical contribution and underscore that these (widely used) systems, in their current form, do not provide reliable citations with their responses. As more open-source alternatives for these generative search engines emerge, we hope that the evaluation metrics and process laid out in this paper will provide a useful blueprint for measuring system quality.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Systematic metric definition and comprehensive evaluation metrics and generative search engines.\n2. These findings are insightful and shed some lights into developing future generative search engines.\n3. This paper is very well-written and easy-to-follow.", "reasons_to_reject": "1. These conclusion may be outdated quickly due to the fact that these generative search engines being tested may updated quickly.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper makes an important step towards attribution evaluation in generative search engines by defining citation metrics and comprehensive human evaluations.\n\nComprehensive human evaluation on the citation of generative search engines.\n\nImportant insights and conclusions are found in this paper.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "[omitted.link](http://omitted.link) doesn’t work in the footnote of Page 2.\n\nDue to the limited access, I understand we know very little about the model details. That being said, I personally would love to see more explanations (or educated guesses) about these models and their performances."}}]

Author rebuttal:

Meta-review: This paper develops metrics to evaluate citations from generative search engines. They perform a human evaluation to determine the efficacy of four such search engines, and find that many statements of these search engines are not supported by citations. The released annotations will be useful for future research.

The primary concern following the discussion period is the lack of a method to quantify support of individual claims within a sentence, as mentioned by reviewer qPBo. There are additionally remarks from reviewer hMXQ about how the search engines may change; however, even if they change, the metrics can be used to evaluate their progress over time.
Decision: Accept-Findings