Active Retrieval Augmented Generation (2023)

Abstract: Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout generation is essential. In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation. We propose Forward-Looking Active REtrieval augmented generation (FLARE), a generic method which iteratively uses a prediction of the upcoming sentence to anticipate future content, which is then utilized as a query to retrieve relevant documents to regenerate the sentence if it contains low-confidence tokens. We test FLARE along with baselines comprehensively over 4 long-form knowledge-intensive generation tasks/datasets. FLARE achieves superior or competitive performance on all tasks, demonstrating the effectiveness of our method.

## introduction generative language models (lms) @xcite @xcite @xcite @xcite have become a foundational component in natural language processing (nlp) systems with their remarkable abilities. although lms have memorized some world knowledge during training @xcite @xcite , they still tend to hallucinate and create imaginary content @xcite . augmenting lms with retrieval components that look up relevant information from external knowledge resources is a promising direction to address hallucination @xcite . retrieval augmented lms commonly use a retrieve-and-generate setup where they retrieve documents based on the user's input, and then generate a complete answer conditioning on the retrieved documents @xcite @xcite @xcite @xcite @xcite @xcite . these single-time retrieval augmented lms outperform purely parametric lms, particularly for shortform knowledge-intensive generation tasks such as factoid question answering (qa) @xcite , where the information needs are clear in the user's input, and it is sufficient to retrieve relevant knowledge once solely based on the input. increasingly powerful large lms have also demonstrated abilities in more complex tasks that involve generating long-form output, such as longform qa @xcite , open-domain summarization @xcite @xcite , and (chain-of-thought; cot) reasoning @xcite @xcite . in contrast to short-form generation, long-form generation presents complex information needs that are not always evident from the input alone. similar to how humans gradually gather information as we create content such as papers, essays, or books, long-form generation with lms would require gathering multiple pieces of knowledge throughout the generation process. for example, to generate a summary about a particular topic, the initial retrieval based on the topic name generate a summary about joe biden. search results: ! ! [1]: … [2]: … joe biden @xcite is the 46th president of the united states. joe @xcite is the 46th president of the united states. he graduated from the university of delaware in 1965 with a bachelor of arts in history and political science. joe biden attended the university of pennsylvania, where he earned a law degree. joe biden announced his candidacy for the 2020 presidential election on april 25, 2019. (e.g., joe biden) may not cover all aspects and details. it is crucial to retrieve extra information as needed during generation, such as when generating a certain aspect (e.g., joe biden's education history) or a specific detail (e.g., the date of joe biden's presidential campaign announcement). ## retrieval augmented generation we formally define single-time retrieval augmented generation and propose the framework of active retrieval augmented generation. ## flare: forward-looking active retrieval augmented generation our intuition is that (1) lms should only retrieve information when they do not have the necessary knowledge to avoid unnecessary or inappropriate retrieval, and (2) the retrieval queries should reflect the intents of future generations. we propose two forward-looking active retrieval augmented generation (flare) methods to implement the active retrieval augmented generation framework. the first method prompts the lm to generate retrieval queries when necessary while generating the answer using retrieval-encouraging instructions, denoted as flare instruct . the second method directly uses the lm's generation as search queries, denoted as flare direct , which iteratively generates the next sentence to gain insight into the future topic, and if uncertain tokens are present, retrieves relevant documents to regenerate the next sentence. ## multi-time retrieval baselines existing passive multi-time retrieval augmented lms can also be formulated using our framework (subsection 2.3). in this section, we formally introduce three baseline categories based on when and what to retrieve. these baselines are not exact reproductions of the corresponding paper because many design choices differ which makes direct comparisons impossible. we implemented them using the same settings, with the only variation being when and what to retrieve. previous-window approaches trigger retrieval every l tokens, where l represents the window size. generated tokens from the previous window are used as the query: some existing methods in this category are @xcite , ic-ralm @xcite , which retrieve every few tokens, and knn-lm @xcite , which retrieves every token. 4 we follow @xcite to use a window size of @xmath0. previous-sentence approaches trigger retrieval every sentence and use the previous sentence as the query, and ircot @xcite belongs to this category: question decomposition approaches manually annotated task-specific exemplars to guide lms to generate decomposed sub-questions while producing outputs. for example, self-ask @xcite , a method in this category, manually inserts sub-questions in exemplars using prompt d.2. for the test case, retrieval is triggered dynamically whenever the model generates a sub-question. the aforementioned approaches can retrieve additional information while generating. however, they have notable drawbacks: (1) using previously generated tokens as queries might not reflect what lms intend to generate in the future. (2) retrieving information at a fixed interval can be inefficient because it might occur at inappropriate points. (3) question decomposition approaches require taskspecific prompt engineering, which restricts their generalizability in new tasks. ## experimental setup we evaluate the effectiveness of flare on 4 diverse knowledge-intensive tasks using few-shot incontext learning @xcite @xcite . we follow previous works @xcite to sub-sample at most 500 examples from each dataset due to the cost of running experiments. datasets, metrics, and settings are summarized in multihop qa the goal of multihop qa is to answer complex questions through information retrieval and reasoning. we use 2wikimultihopqa @xcite ) which contains 2-hop complex questions sourced from wikipedia articles that require composition, comparison, or inference, e.g., "why did the founder of versus die?" we follow @xcite to generate both the chain-ofthought and the final answer. experimental setting details are included in appendix b. we use regular expressions to extract the final answer from the output and compare it with the reference answer using exact match (em), and tokenlevel f 1 , precision, and recall. commonsense reasoning commonsense reasoning requires world and commonsense knowledge to generate answers. we use strategyqa @xcite which is a collection of crowdsourced yes/no questions, e.g., "would a pear sink in water?" we follow @xcite to generate both the chain-of-thought and the final yes/no answer. details are included in appendix b. we extract the final answer and match it against the gold answer using exact match. long-form qa long-form qa aims to generate comprehensive answers to questions seeking complex information @xcite . we use asqa @xcite as our testbed where inputs are ambiguous questions with multiple interpretations, and outputs should cover all of them. for example, "where do the philadelphia eagles play their home games?" could be asking about the city, sports complex, or stadium. we found in many cases it is challenging even for humans to identify which aspect of the question is ambiguous. therefore, we created another setting (asqa-hint) where we provide a brief hint to guide lms to stay on track when generating answers. the hint for the above case is "this question is ambiguous in terms of which specific location or venue is being referred to." experimental setting details are included in appendix b. we use metrics from @xcite , including em, roberta-based qa score (disambig-f 1 ), rouge @xcite , and an overall score combining disambig-f 1 and rouge (dr). ## experimental results we first report overall results across 4 tasks/datasets and compare the performance of flare with all the baselines introduced in section 4. we then run ablation experiments to study the efficacy of various design choices of our method. 6.1 comparison with baselines overall results. the overall performance of flare and baseline across all tasks/datasets are reported in figure 4 . flare outperforms all baseline on all tasks/datasets, indicating that flare is a generic method that can effectively retrieve additional information throughout the generation. among various tasks, multihop qa shows the most significant improvement. this is largely due to the task's clear definition and specific objective of producing the final answer through a 2-hop reasoning process, which makes it easier for lms to generate on-topic output. in contrast, asqa and wikiasp are more open-ended, which increases the difficulty of both generation and evaluation. the improvement on asqa-hint is larger than that of asqa because identifying ambiguous aspects is challenging even for humans in many cases, and providing a generic hint helps lms to stay on topic. thorough comparisons with baselines. the performance of all baselines on 2wikimultihopqa are reported in ## related work we refer to subsection 2.2 and section 4 for extensively discussion on single-time and multi-time retrieval augmented lms, which is the most relevant area to this paper. iterative and adaptive retrieval iterative retrieval and refinement has been studied in both text and code generation tasks @xcite @xcite . flare differs from these methods in the granularity of generation and retrieval strategies. adaptive retrieval has been studied in single-time retrieval scenarios based on either question popularity or generation probabilities @xcite , while we focus on long-form generation requiring active information access. browser-enhanced lms @xcite and webcpm @xcite train lms to interact with browser to enhance factuality using reinforcement learning or supervised training where multiple queries can be triggered before generation. flare is built on text-based retrievers but can be combined with a browser to potentially improve retrieval quality. ## conclusion to aid long-form generation with retrieval augmentation, we propose an active retrieval augmented generation framework that decides when and what to retrieve during generation. we implement this framework with forward-looking active retrieval that iteratively uses the upcoming sentence to retrieve relevant information if it contains lowconfidence tokens and regenerates the next sentence. experimental results on 4 tasks/datasets demonstrate the effectiveness of our methods. future directions include better strategies for active retrieval and developing efficient lm architectures for active information integration.