Adaptive- RAG : Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024)

Abstract: Retrieval-Augmented Large Language Models (LLMs), which incorporate the non-parametric knowledge from external knowledge bases into LLMs, have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA). However, even though there are various approaches dealing with queries of different complexities, they either handle simple queries with unnecessary computational overhead or fail to adequately address complex multi-step queries; yet, not all user requests fall into only one of the simple or complex categories. In this work, we propose a novel adaptive QA framework that can dynamically select the most suitable strategy for (retrieval-augmented) LLMs from the simplest to the most sophisticated ones based on the query complexity. Also, this selection process is operationalized with a classifier, which is a smaller LM trained to predict the complexity level of incoming queries with automatically collected labels, obtained from actual predicted outcomes of models and inherent inductive biases in datasets. This approach offers a balanced strategy, seamlessly adapting between the iterative and single-step retrieval-augmented LLMs, as well as the no-retrieval methods, in response to a range of query complexities. We validate our model on a set of open-domain QA datasets, covering multiple query complexities, and show that ours enhances the overall efficiency and accuracy of QA systems, compared to relevant baselines including the adaptive retrieval approaches. Code is available at: https://github.com/starsuzi/Adaptive-RAG.

## introduction recent large language models (llms) @xcite @xcite have shown overwhelming performances across diverse tasks, including question- performance vs time with gpt-3.5 answering (qa) @xcite . however, they still generate factually incorrect answers since their knowledge solely relies on their parametric memory @xcite . meanwhile, memorizing all the (ever-changing) world knowledge may not be possible. to address this problem, retrievalaugmented llms @xcite @xcite , which incorporate non-parametric knowledge into llms with additional retrieval modules, have gained much increasing attention. specifically, these models access a knowledge base, which serves as an extensive repository of information across various subjects and disciplines, to retrieve information relevant to the given input, and then incorporate the retrieved information into llms, which enables them to stay accurate and current with the world knowledge. a particularly salient application of retrievalaugmented llms is to handling qa tasks, whose goal is to provide correct answers in response to user queries, especially those of high complexity. early work on retrieval-augmented llms focuses primarily on single-hop queries @xcite , whose answers are typically found within a single document; therefore, this approach involves retrieving a relevant document based on the query and subsequently integrating this information into qa models to formulate a response. however, unlike this single-hop qa, some queries require connecting and aggregating multiple documents, which are, furthermore, often not answerable through a single-step process of retrieval-and-response. an example query is 'when did the people who captured malakoff come to the region where philipsburg is located?', which requires four reasoning steps to solve. therefore, to effectively handle such complex queries, recent studies have concentrated largely on multistep and multi-reasoning qa, which requires iterative accesses to both llms and retrievers multiple times @xcite , at the cost of heavy computational overheads. yet, we should rethink: in a real-world scenario, are all the requests from users complex? instead, users might often ask simple and straightforward questions, while only occasionally asking complex ones. specifically, a query such as 'paris is the capital of what?' is likely to be asked more frequently, compared to the aforementioned multistep query, and this simpler query might also be easily answered by the llms themselves, without accessing external knowledge. in other words, a multi-step qa approach could give rise to unnecessary computational overhead for simple queries, even though it would be vital for complex queries (see figure 2 (a) ). on the other hand, handling complex queries with single-step-retrieval or even non-retrieval strategies would be largely insufficient (figure 2 (b) ). this suggests the need for an adaptive qa system, which can dynamically adjust the operational strategies of retrieval-augmented llms based on the query complexity. while some recent approaches are capable of doing this based on the frequency of entities in queries @xcite or on the generated outputs from models for multi-step qa @xcite , they are still suboptimal: the former methods are overly simplistic, failing to consider multihop queries; meanwhile, the latter are excessively complex, terminating answer solving steps after several rounds of module access. in this work, considering diverse complexity levels of real-world queries, we argue that previous one-size-fits-all approaches might be inadequate to cover all of them. instead, we propose to select the most suitable strategy from a range of (retrievalaugmented) llms, each of which is tailored to the specific complexity of the input query. notably, a critical step in this process is pre-defining the query complexity, which is instrumental in determining the most fitting model to it. in this work, we operationalize this process with a novel classifier, which is a smaller model trained to predict the complexity level of incoming queries (see figure 2 (c)). moreover, we automatically collect its training datasets without human labeling, by leveraging the predicted outcomes (i.e., which models accurately respond to which queries) as well as by capitalizing on the inherent biases in existing datasets (i.e., samples in the datasets are designed either for singlestep or for multi-step qa scenarios). this proposed method can offer a robust middle ground among the iterative llm augmentation methods for complex queries, single-step methods for simpler queries, and even no-retrieval-augmented methods for the most straightforward queries (answerable by llms themselves), thus significantly enhancing the overall efficiency and accuracy, as shown in figure 1 . we refer to our framework as adaptive retrieval-augmented generation (adaptive-rag). we validate adaptive-rag using benchmark open-domain qa datasets, covering a wide range of query complexity from single-hop @xcite @xcite to multi-hop @xcite @xcite queries. the experimental results show that ours significantly improves the overall accuracy and efficiency, compared to the prior adaptive strategies, on multiple llms, such as gpt-3.5 @xcite and flan-t5 series @xcite . our contributions and findings are threefold: open-domain qa open-domain qa is the task of accurately answering a query by sourcing for query-relevant documents, and then interpreting them to provide answers @xcite , which, thus, generally involves two modules: a retriever @xcite and a reader @xcite @xcite . along with the emergence of llms with superior reasoning capabilities thanks to their billion-sized parameters @xcite , a synergy between llms and retrievers has led to significant advancements @xcite . specifically, this integration has been shown to enhance open-domain qa by mitigating the hallucination problem from llms through strengthened reasoning abilities of the reader, as well as utilizing the retrieved, external documents @xcite . despite these advancements for single-hop retrieval-augmented llms, however, the complexity of some queries needs a more complex strategy. multi-hop qa multi-hop qa is an extension of conventional open-domain qa, which additionally requires the system to comprehensively gather and contextualize information from multiple documents (often iteratively), to answer more complex queries @xcite . in the realm of multi-hop qa, the approach to iteratively access both llms and the retrieval module is generally employed. specifically, @xcite al. (2022), press et al. (2023), pereira et al. (2023) and khot et al. (2023) proposed to first decompose the multi-hop queries into simpler single-hop queries, repeatedly access the llms and retriever to solve these sub-queries, and merge their solutions to formulate a complete answer. in contrast to this decomposition-based approach, other recent studies, such as yao et al. (2023) and trivedi et al. (2023) , explored the interleaving of chain-of-thought reasoning @xcite ) -a method where a logical sequence of thoughts is generated -with document retrieval, repeatedly applying this process until the reasoning chain generates the answer. in addition, @xcite introduced an approach to repeatedly retrieving new documents if the tokens within generated sentences have low confidence. however, the aforementioned methods overlooked the fact that, in real-world scenarios, queries are of a wide variety of complexities. therefore, it would be largely inefficient to iteratively access llms and retrievers for every query, which might be simple enough with a single retrieval step or even only with an llm itself. adaptive retrieval to handle queries of varying complexities, the adaptive retrieval strategy aims to dynamically decide whether to retrieve documents or not, based on each query's complexity. in this vein, @xcite proposed to decide the query's complexity level based on the frequency of its entities and suggested using the retrieval modules only when the frequency falls below a certain threshold. however, this approach, focusing solely on the binary decision of whether to retrieve or not, may not be sufficient for more complex queries that require multiple reasoning steps. additionally, @xcite proposed an approach that performs a fixed set of operations (retrieving, reading, and reranking) multiple times until the answer is derived for the given query, which is built upon traditional bert-like lms. however, unlike our adaptive-rag which pre-determines the query complexity and adapts the operational behavior of any off-the-shelf llms accordingly, this approach applies the same fixed operations to every query regardless of its complexity but also necessitates additional specific training to lms. concurrent to our work, @xcite suggested training a sophisticated model to dynamically retrieve, critique, and generate the text. nevertheless, we argue that all the aforementioned adaptive retrieval methods that rely on a single model might be suboptimal in handling a variety of queries of a range of different complexities since they tend to be either overly simple or complex for all the input queries, which demands a new approach that can select the most suitable strategy of retrieval-augmented llms tailored to the query complexity. ## method in this section, we describe our approach to adapting retrieval-augmented llms, by pre-determining the query complexity and then selecting the most fitting strategies for retrieval-augmented llms. ## experimental setups in this section, we explain datasets, models, metrics, and implementation details. we provide additional details in appendix a. ## experimental results and analyses in this section, we show the overall experimental results and offer in-depth analyses of our method. ## conclusion in this work, we proposed the adaptive retrieval-augmented generation framework, referred to as adaptive-rag, to handle queries of various complexities. specifically, adaptive-rag is designed to dynamically adjust its query handling strategies in the unified retrieval-augmented llm based on the complexity of queries that they encounter, which spans across a spectrum of the nonretrieval-based approach for the most straightforward queries, to the single-step approach for the queries of moderate complexity, and finally to the multi-step approach for the complex queries. the core step of our adaptive-rag lies in determining the complexity of the given query, which is instrumental in selecting the most suitable strategy for its answer. to operationalize this process, we trained a smaller language model with querycomplexity pairs, which are automatically annotated from the predicted outcomes and the inductive biases in datasets. we validated our adaptive-rag on a collection of open-domain qa datasets, covering the multiple query complexities including both the single-and multi-hop questions. the results demonstrate that our adaptive-rag enhances the overall accuracy and efficiency of qa systems, allocating more resources to handle complex queries while efficiently handling simpler queries, compared to the existing one-size-fits-all approaches that tend to be either minimalist or maximalist over varying query complexities.