Enabling Large Language Models to Generate Text with Citations (2023)

Abstract: Large language models (LLMs) have emerged as a widely-used tool for information seeking, but their generated outputs are prone to hallucination. In this work, our aim is to allow LLMs to generate text with citations, improving their factual correctness and verifiability. Existing work mainly relies on commercial search engines and human evaluation, making it challenging to reproduce and compare different modeling approaches. We propose ALCE, the first benchmark for Automatic LLMs’ Citation Evaluation. ALCE collects a diverse set of questions and retrieval corpora and requires building end-to-end systems to retrieve supporting evidence and generate answers with citations. We develop automatic metrics along three dimensions—fluency, correctness, and citation quality—and demonstrate their strong correlation with human judgements. Our experiments with state-of-the-art LLMs and novel prompting strategies show that current systems have considerable room for improvement—For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time. Our analyses further highlight promising future directions, including developing better retrievers, advancing long-context LLMs, and improving the ability to synthesize information from multiple sources.

## introduction large language models (llms; brown et al., 2020; openai, 2023) have gained increasing popularity as a tool for information seeking. while they generate engaging and coherent responses, their outputs are prone to hallucination and often contain factually incorrect information @xcite . this makes it harder for users to trust and verify llmgenerated outputs without any supporting evidence. in this work, we study a new generation paradigm for llms, in which we require llms when did the us break away from england? the us took the first step towards gaining independence from gb when it declared independence on july 2, 1776 (although the event is now commemorated on july 4, 1776, the date when the declaration of independence was officially adopted by congress) to provide citations to one or a few text passages for any statement they generate (figure 1 ). incorporating citations brings several benefits: (1) users can easily verify llms' claims with the provided citations; (2) llms can generate text that faithfully follows cited passages, which has the promise to improve correctness and alleviate hallucination. multiple commercial systems have adopted this paradigm: bing chat foot_1 and perplexity.ai 3 respond to user questions in natural language with references to web pages. @xcite we present alce, the first reproducible benchmark for automatically evaluating llms' generations with citations. alce assumes a naturallanguage question and a retrieval corpus, and requires building end-to-end systems to retrieve relevant passages from the corpus, generate a response to the question, and cite corresponding supporting passages. we compile three datasets that cover different types of questions and corpora-asqa @xcite , qampari (rubin et al., 2022), and eli5 @xcite -as shown in we design automatic evaluation methods in three dimensions: fluency, correctness, and citation quality. specifically, we use mauve @xcite to measure fluency, propose tailored correctness metrics for each dataset, and adopt a natural language inference (nli) model @xcite to measure citation quality. we showcase how the three dimensions together contribute to a robust evaluation, preventing systems from exploiting shortcuts. additionally, we conduct human evaluation and demonstrate a strong correlation with our automatic metrics. we experiment on multiple systems with stateof-the-art llms and retrievers and also propose novel prompting strategies to synthesize retrieved text into text generation. although all systems are capable of providing fluent and coherent responses, there remains substantial room for improvement in terms of correctness and citation quality: for example, on the eli5 dataset, around 50% generations of our chatgpt and gpt-4 baselines are not fully supported by the cited passages. additionally, we find that (1) a closed-book model (generating answers without accessing any retrieved documents) with post-hoc citing achieves good correctness but much worse citation quality; (2) although interactive retrieval approaches @xcite al., 2023; schick et al., 2023) offer more flexibility in when/what to retrieve, they do not improve the performance on this challenging benchmark; (3) summarizing the retrieved passages in a shorter text improves correctness but not citation quality; (4) reranking multiple generations boosts citation quality measured by human evaluation; (5) incorporating more retrieved passages in context does not help chat-gpt but improves gpt-4 performance. our extensive analyses highlight three major challenges of building llms to generate text with citations: (1) the retrieval quality is crucial to the final performance and has substantial room for improvement; (2) llms' limited context window restricts the number of passages they can incorporate; ## task setup and datasets our task is formalized as follows: given a query q and a corpus of text passages d, the system is required to return an output s, which consists of n statements s 1 , ..., s n , and each statement s i cites a list of passages c @xmath0,1 , c i,2 , . . .} foot_3 , where c i,@xmath1. in this work, we segment llms' output into statements by sentence boundaries. 5 while llms may include sentences that do not require a citation, such as "i'm happy to help", we observe that almost all sentences that llms output provide valuable information and require citations, similar to findings in liu et al. @xcite . in this work, citations are enclosed by box brackets such as [1][2]. we divide the corpus d into 100-word passages following previous works on open-domain question answering @xcite @xcite , in contrast to commercial systems like bing chat, which cite entire web pages. we take 100-word passages because it is easier for humans to verify, and allows for more retrieved passages to fit in llms' limited context. we choose qa datasets so that (1) they contain factual questions, in which references are important; (2) questions require long-text answers that cover multiple aspects; (3) answering the questions requires synthesizing multiple sources. we select three datasets ( asqa @xcite ) is a long-form factoid dataset. as shown in figure 1 , each question is an ambiguous question from ambigqa @xcite ) that requires multiple short answers to cover different aspects, and the dataset provides a longform answer that covers all short answers. since most questions can be answered by wikipedia, we use the 2018-12-20 wikipedia snapshot as d. qampari (rubin et al., 2022) is a factoid qa dataset constructed from wikipedia, where the answer is a list of entities that are drawn from different passages. same as asqa, we use the 2018-12-20 wikipedia as the corpus. eli5 @xcite is a long-form qa dataset built on the reddit forum "explain like i'm five". 6 we randomly select 1,000 examples from the development set of each dataset for alce. our benchmark primarily assesses the citation capabilities of existing llms and does not provide training data, as there are no available examples that provide supervision for citations in these datasets. ## automatic evaluation our benchmark measures the following three dimensions of system responses: ## modeling in this section, we discuss three major modeling components for an alce system-retrieval, synthesis, and post-editing. ## experiments we describe experiment details in §c. we use chat-gpt (gpt-3.5-turbo-0301) with a 4k context window for most main experiments and ablations. we also report results with chatgpt-16k (gpt-3.5-turbo-16k-0613) and gpt-4 (gpt-4-0613; 8k context window). for open-source models, we test llama @xcite oasst @xcite . they all have a 2k context window. we use short instructions for llama ( ## human evaluation to verify that our automatic evaluation correlates with human judgement, we conduct human evaluation on selected models and request workers to judge model generations on three dimensions similar to liu et al. (2023)-(1) utility: a 1-to-5 score indicating whether the generation helps answer the question; (2) citation recall: the annotator is given a sentence and all passages that the sentence cited, and is asked to judge whether the passages fully support the sentence; (3) citation precision: given a sentence and one of its citations, the annotator is asked to judge whether the citation "fully supports", "partially supports", or "does not support" the sentence. each citation gets a precision score 1 if the output sentence has a citation recall of 1 and this citation at least "partially supports" it. see appendix f for more details. model outputs score high utility. the utility scores do not differ significantly between models, ranging 3.7-3.9 for asqa and 3.5-3.6 for eli5. upon inspection, all tested models are mostly able to output fluent answers that are related to the question, despite differences in factual correctness. our automatic evaluation of citation quality strongly correlates with human judgements. as shown in ## conclusion we propose alce, the first automatic benchmark for evaluating llm generations with citations. we deploy automatic metrics to measure fluency, correctness, and citation quality, and verify their efficacy via human evaluation. we explore a variety of strategies for incorporating citations in llms and demonstrate that current systems have considerable room for improvement on alce. our experiments highlight a number of promising research directions, including (1) enhancing retrieval and refining retrieval integrations in llms, (2) developing long-context llms, and (3) advancing llms' ability to synthesize multiple sources. what's even more intriguing is that these research proposals extend beyond the alce setup (for example, long-context llms have numerous exciting applications), and alce can serve as a valuable testbed for their development.