Evaluating Verifiability in Generative Search Engines (2023)

Abstract: Generative search engines directly generate responses to user queries, along with in-line citations. A prerequisite trait of a trustworthy generative search engine is verifiability, i.e., systems should cite comprehensively (high citation recall; all statements are fully supported by citations) and accurately (high citation precision; every cite supports its associated statement). We conduct human evaluation to audit four popular generative search engines—Bing Chat, NeevaAI, perplexity.ai, and YouChat—across a diverse set of queries from a variety of sources (e.g., historical Google user queries, dynamically-collected open-ended questions on Reddit, etc.). We find that responses from existing generative search engines are fluent and appear informative, but frequently contain unsupported statements and inaccurate citations: on average, a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence. We believe that these results are concerningly low for systems that may serve as a primary tool for information-seeking users, especially given their facade of trustworthiness. We hope that our results further motivate the development of trustworthy generative search engines and help researchers and users better understand the shortcomings of existing commercial systems.

## introduction generative search engines fulfill user information needs by directly generating responses to input queries, along with in-line citations (figure 1 ). 1 existing generative search engines are rapidly gaining users-in march 2023, microsoft reported that "roughly one third of daily preview users are using what are the latest discoveries from the james webb space telescope? the james webb space telescope is designed to peer into the dusty clouds of gas where stars and planetary systems are born. webb has captured the first direct image of an exoplanet, and the pillars of creation in the eagle nebula @xcite . additionally, the telescope will be used to study the next interstellar interloper studying the next interstellar interloper with webb ...scientists have had only limited ability to study these objects once discovered, but all of that is about to change with nasa's james webb space telescope...the team will use webb's spectroscopic capabilities in both the near-infrared and mid-infrared bands to study two different aspects of the interstellar object. ## evaluation setup in this section, we describe the evaluated generative search engines ( §3.1), the diverse query distributions we use for evaluation ( §3.2), and the details of our human evaluation protocol ( §3.3). ## results and analysis this section presents the results of our human evaluation study and discusses our main observations and analyses. we see that fluency and perceived utility are generally high across different generative search engines ( §4.1), while citation recall and precision are quite low ( §4.2), though performance certainly varies by system and query distribution-the low citation recall and precision, when combined with the facade of trustworthiness from fluency and high perceived utility, increase the potential for existing generative search engines to mislead users. our results also show that citation precision is inversely correlated with perceived utility in existing generative search engines ( §4.3). we hypothesize that this is a byproduct of systems' propensity to copy or closely paraphrase text from cited webpages, which may increase citation precision and decrease perceived utility ( §4.4). ## related work existing work has proposed a variety of techniques for building language models that provide references to support generated text. @xcite use reinforcement learning from human preferences to train language models to answer questions and provide supporting evidence. similarly, @xcite also use reinforcement learning from human preferences to train language models to answer user questions, but their system generates responses by conditioning on evidence retrieved from a google search for the given user query. finally, the lamda system of @xcite is trained to provide urls that support its generated statements. in contrast to the aforementioned line of work on training systems to generate citations, @xcite propose a method for post-editing generated output to reflect and cite retrieved evidence. existing work has also proposed evaluation protocols and benchmarks for improving verifiability in language generation systems. @xcite propose the attributed to identified sources (ais) evaluation framework to assess whether a particular statement is supported by provided evidence and validate their guidelines on conversational question answering, summarization, and table-totext systems. @xcite introduce the task of attributed question answering, where systems are given an input question and must output an answer string with a pointer to evidence text supporting the answer, and propose a reproducible evaluation setup with naturalquestions queries (only paragraph answer type containing long and short answers) with wikipedia as the evidence corpus. in contemporaneous work, peskoff and stewart (2023) have domain experts evaluate chatgpt and youchat responses to 100 expert-written questions. they find that generated responses are coherent and concise, but frequently undersourced and inaccurate; our results also show that youchat responses frequently lack citations for generated statements (i.e., low citation recall). ## conclusion in this work, we used human evaluation to audit the verifiability of four popular commercial generative search engines-bing chat, neevaai, perplexity.ai, and youchat. we find that responses from existing generative search engines are generally fluent and often appear informative, but frequently contain unsupported statements and inaccurate citations (low citation recall and precision)-a mere 51.5% of generated statements are fully supported by citations (recall), and only 74.5% of citations support their associated statements (precision). we believe that existing systems' citation recall and precision are unacceptably low, given that they are quickly becoming a popular tool for answering user queries and already have millions of users. moreover, we find that citation precision is inversely correlated with perceived utility in existing generative search engines-the responses that seem more helpful are often those with more unsupported statements or inaccurate citations. analysis suggests that this inverse correlation occurs in existing systems because of their propensity to copy or closely paraphrase from cited webpages, which inflates citation precision at the cost of lower perceived utility. we hope our results and insights further motivate the development of trustworthy generative search engines and help researchers and users better understand their current shortcomings.