Assessing “Implicit” Retrieval Robustness of Large Language Models (2024)

Abstract: Retrieval-augmented generation has gained popularity as a framework to enhance large language models with external knowledge. However, its effectiveness hinges on the retrieval robustness of the model. If the model lacks retrieval robustness, its performance is constrained by the accuracy of the retriever, resulting in significant compromises when the retrieved context is irrelevant. In this paper, we evaluate the “implicit” retrieval robustness of various large language models, instructing them to directly output the final answer without explicitly judging the relevance of the retrieved context. Our findings reveal that fine-tuning on a mix of gold and distracting context significantly enhances the model’s robustness to retrieval inaccuracies, while still maintaining its ability to extract correct answers when retrieval is accurate. This suggests that large language models can implicitly handle relevant or irrelevant retrieved context by learning solely from the supervision of the final answer in an end-to-end manner. Introducing an additional process for explicit relevance judgment can be unnecessary and disrupts the end-to-end approach.

## introduction large language models (llms) have brought about a paradigm shift in the field of natural language processing, enabling remarkable advancements in various tasks @xcite @xcite . however, their static nature imposes limitations, preventing them from fully encompassing all specialized knowledge or maintaining its currency @xcite . to mitigate this constraint, a prevailing trend involves the adoption of retrieval-augmented generation (rag) methodologies @xcite elling the relevance of retrieved context. the explicit approach evaluates whether the retrieved context is relevant and then calls different functions based on this assessment. in contrast, the implicit approach directly generates the final answer in an end-to-end manner. @xcite . through bringing extra context from the retriever, these models can tap into external knowledge reservoirs, refining their outputs with heightened precision and contextually fitting information @xcite @xcite . nevertheless, acquiring a reliable retriever is challenging. since the number of candidate documents for retrieval is typically much larger than the vocabulary size of llms, it is often easier to generate the correct answer from the knowledge stored in the model parameters rather than retrieving it @xcite @xcite . when the retriever is imperfect, the quality of llm generations can be significantly compromised, which often leads to poorer performance compared to scenarios where no retriever is employed at all @xcite . the main reason that influences the quality of rag is their retrieval robustness @xcite . ideally, a retrieval-robust model should possess two key capabilities: i properly incorporate helpful retrieved information to provide an accurate answer. ii ignore distracting information and rely on its own internal knowledge as a fallback. 2 capability i pertains to scenarios where the retrieved information aids in deriving the answer, while capability ii pertains to scenarios where the retriever only returns distracting information. a wide range of approaches have been proposed to improve the retrieval robustness of llms, which can be classified into two categories: the first category explicitly decouples capability i and ii by injecting an intermediate process to judge the relevance of retrieved information, based on which different functions are called @xcite . the second category, on the contrary, relies on the model itself to implicitly judge the relevance of the retrieved information and generate the right answer directly @xcite . figure 1 depicts the difference between explicit and implict approaches. despite being finer-grained, explicit approaches increase runtime latency and the risk of error propagation. they also require annotations regarding the relevance of retrieved information, which can be costly to obtain on a large scale. 3 in this paper, we conduct a thorough analysis in a controlled setting to evaluate the "implicit" retrieval robustness of llms. more concretely, we aim to determine the extent to which we can uphold the retrieval robustness without requiring explicit judgment of the retrieval's relevance. to conduct this analysis, we run extensive experiments with 5 question-answering tasks spanning different domains and scenarios; 5 open-source llms (vicuna-7/13/33b and llama 2-7/13b); 2 closed-source models (gpt-3.5 and gpt-4) and 3 testing scenarios (zero-shot with prompting, full fine-tuning and lora fine-tuning). for each experiment, we run controlled tests to evaluate capability i and ii of the models separately. our findings can be summarized as follows: overall, we suggest that llms are notably robust at noisy retrievals during fine-tuning. with a high noise ratio, the "implicit" retrieval robustness of llms can be remarkably effective. for most question-answering tasks that do not involve sophisticated multi-hop or multi-turn inference, relying on the model's implicit retrieval robustness may already suffice. ## related work retrieval-augmented generation due to the static nature of the knowledge stored within their parameters, large language models encounter difficulties in tasks that require extensive knowledge or have temporal dependencies @xcite . retrieval-augmented generation has emerged as a valuable approach to address these limitations by enabling models to retrieve and integrate information from external sources during the generation process @xcite @xcite . the external sources may include knowledge bases, search engines, multi-turn histories, or private databases, depending on the specific knowledge needed for the task @xcite . various studies have explored the integration of retrieval mechanisms into generative models to enhance the quality and relevance of generated text from llms @xcite @xcite . the retrieval-augmented mechanism not only improves performance but also offers a cost-effective approach to adapting the model for diverse domains by dynamically adjusting external knowledge sources @xcite . although improvement has been observed, the quality of generations is strongly affected by the accuracy of retrievers. inaccuracies in retrievers can lead to the incorporation of irrelevant or misleading information, resulting in lower-quality generated content @xcite . retrieval-robust large language model recognizing that the quality of text generations from llms is significantly influenced by the retriever's quality, various research works have been proposed to enhance the retrieval robustness of llms, i.e. , the model should effectively utilize accurate retrieved information while also disregarding distracting information in cases where the retriever is inaccurate @xcite . the first line of research introduces an intermediary step to assess the relevance of retrieved information, aligning with conventional methods of step-by-step planning in text generation @xcite @xcite . when the information is detected to be unhelpful, the model will simply fall back to use its own parameterized knowledge to answer the question. this helpfulness label is usually obtained by manual annotation @xcite , chain-of-thought prompting on a powerful llm @xcite @xcite , or inspecting its effect on the model generation @xcite . although this step-by-step approach provides finergrained signals, it also leads to increased runtime latency and training costs, with potential risks of error propagation @xcite . conversely, the alternative line of research employs an endto-end approach to train models to autonomously discern the relevance of retrieved information from without extra helpfulness labels. the key to achieving successful end-to-end learning is to incorporate noisy retrievals, allowing the model to adjust to distracting information @xcite . nonetheless, existing studies lack quantitative analysis on how the retrieval robustness is influenced by factors such as the model, fine-tuning method, data, and noise ratio. our research seeks to address this gap in the literature. ## definition of retrieval robustness let q, c, a denote the question, context retrieved from an external source, and answer respectively. the variable p denotes the probability estimator from the llm generator. in retrieval-augmented generation, the retriever retrieves some context c foot_2 from external sources where c can be either helpful or unhelpful depending on the accuracy of the retriever. the answer is generated from p(a|q, c) by conditioning on q and c. an llm is considered retrieval-robust if the probability estimation p(a|q, c) remains effective regardless of the helpfulness of c. it corresponds to two different capabilities that the llm should possess: i when c is helpful, i.e. , the correct answer a * can be derived from the information contained in c, then it should return a * . ii when c is not helpful, it should discard the information in c and rely on its own parameterized knowledge p(a|q) to answer the question. equation 1 illustrates the ideal p robust (a|q, c) from a retrieval-robust llm mathematically, where δ is the dirac-delta function. p robust (a|q, c) = δ(aa * ), if a * ∈ c p(a|q), otherwise (1) 4 experiment setup model we test 5 open-source llms: vicuna-1.3-7/13/33b @xcite and llama 2chat-7b/13b @xcite , as well as two closed-source llms gpt-3.5 and gpt-4 @xcite . 5 for open-source llms, we test their performance with zero-shot prompting, lora and full fine-tuning on task-specific datasets. for closed-source llms, we only report their performance by prompting them with instructions. ## results and analysis we evaluate how retrieval robust different llms are in three scenatios: when directly prompting the original llms without fine-tuning them; when fine-tuning them only on gold context, and when fine-tuning them on mixed gold and distracting context. the results are presented in this order. full results tables are in appendix c ## conclusion retrieval robustness is the key to determine the quality of model generations in rag. in this paper, we conduct an extensive assessment of the "implicit" retrieval robustness of llms without explicitly letting models judge the relevance of the retrieved context. our findings indicate that llms are remarkably adept at handling context with varied retrieval accuracy, without needing explicit relevance annotations. by incorporating a certain ratio of distracting context into the fine-tuning dataset, llms can maintain their ability to extract correct answers from relevant context while hardly being misled by irrelevant information.