Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training (2024)

Abstract: Large Language Models (LLMs) exhibit substantial capabilities yet encounter challenges including hallucination, outdated knowledge, and untraceable reasoning processes. Retrieval-augmented generation (RAG) has emerged as a promising solution, integrating knowledge from external databases to mitigate these challenges. However, inappropriate retrieved passages can potentially hinder the LLMs’ capacity to generate comprehensive and high-quality responses. Prior RAG studies on the robustness of retrieval noises often confine themselves to a limited set of noise types, deviating from real-world retrieval environments and limiting practical applicability. In this study, we initially investigate retrieval noises and categorize them into three distinct types, reflecting real-world environments. We analyze the impact of these various retrieval noises on the robustness of LLMs. Subsequently, we propose a novel RAG approach known as Retrieval-augmented Adaptive Adversarial Training (RAAT). RAAT leverages adaptive adversarial training to dynamically adjust the model’s training process in response to retrieval noises. Concurrently, it employs multi-task learning to ensure the model’s capacity to internally recognize noisy contexts. Extensive experiments demonstrate that the LLaMA-2 7B model trained using RAAT exhibits significant improvements in F1 and EM scores under diverse noise conditions. For reproducibility, we will release our code and data upon acceptance.

## introduction large language models (llms) have garnered substantial attention in both academic and industrial research within the domain of artificial intelligence due to their remarkable capabilities @xcite . despite their immense power, llms face challenges such as hallucinations and outdated knowledge @xcite . moreover, a lack of domain knowledge may hinder their performance on domain-specific tasks @xcite . to mitigate these challenges, recent studies improve llms by retrieving passages from external databases and pretending them in context, constituting a framework known as retrieval-augmented language models (ralms) @xcite . however, ralms also present significant limitations. previous studies @xcite @xcite have empirically demonstrated that retrieved noisy passages are problematic for llms, resulting in performance degradation. we term this issue as the noise robustness problem of ralms. as illustrated in figure 1 , the model can provide correct answers when the re-trieving context is accurate and related to the query. however, when the retrieved context contains misleading or inaccurate information, the model may yield incorrect answers. as the retriever inherently cannot achieve complete accuracy, the presence of noise in the retrieved context is inevitable. therefore, designing robust algorithms against retrieved noises is of great practical importance. recently, several studies @xcite have attempted to enhance the noise robustness of ralms through noisy training, which involves incorporating retrieved noisy contexts into fine-tuning data. while noisy training exhibits promise, its effectiveness heavily relies on the composition of the training dataset. incorrectly introducing noises to the training data can lead to model overfitting, adversely affecting generalization. in practice, meticulous adjustment of the type and intensity of noises is essential to ensure the model's proficiency across various tasks and datasets. this demands significant experimentation and tuning, adding complexity to the development process. moreover, the lack of clear classification for retrieval noises in current studies stands in contrast to the diverse range of noises encountered in real retrieval environments. this paper systematically explores three types of retrieval noises: (i) contexts that are superficially related to the query but lack the correct answer (relevant retrieval noise), (ii) contexts that are irrelevant to the query (irrelevant retrieval noise), and (iii) contexts that are topically related to the query but contain incorrect information (counterfactual retrieval noise). our empirical study indicates that llms exhibit varying robustness to these three types of noise. compared to entirely irrelevant texts, texts that are superficially related to the query or those containing counterfactual details often lead to more misinformation. in response to diverse types of noises, we propose a novel approach named retrieval-augmented adaptive adversarial training (raat), which employs adaptive adversarial training to dynamically regulate the model's training process in response to retrieved noisy texts. concretely, raat generates adversarial samples (noises) by considering the model's sensitivity to different types of noises during training, which aligns with the minmax paradigm of adversarial training @xcite . moreover, raat utilizes multi-task learning @xcite to encourage the llms to generate tokens that are aware of noises, thereby enabling the model to internally recognize retrieved noisy contexts and improve the overall generation performance. the main contributions of this paper can be summarized as follows: retrieval-augmented generation with noisy context retrieval-augmented language models (ralms) have shown impressive performance in various nlp tasks @xcite . however, limited by the capabilities of the retriever, retrieval-augmented systems inevitably introduce irrelevant or partially relevant knowledge to the models @xcite . recent studies @xcite @xcite have increasingly focused on the impact of noisy information on retrieval-augmented generation. for example, @xcite ; @xcite observed that adding irrelevant noise to the context could detrimentally affect model performance. @xcite demonstrated that as the proportion of noise in the retrieval context increases, the performance of llms experiences a notable decline. similar phenomena have been reported by @xcite and @xcite . adversarial training adversarial training is recognized as a crucial method for enhancing model robustness, initially proposed by @xcite . early studies widely investigated adversarial training in the computer vision domain @xcite . in the nlp domain, @xcite applied perturbations to word embeddings, making the model less prone to overfitting. similarly, perturbations on different granularities have been extensively studied, encompassing various aspects of nlp tasks @xcite @xcite @xcite . recently, several studies have concentrated on generating adversarial examples designed to induce llms to generate harmful or non-factual content @xcite instead of merely causing the model to make inaccurate predictions. @xcite employed decisionbased perturbation at different levels to craft adversarial examples, revealing vulnerabilities in chat-gpt to both sentence-level and character-level adversarial attacks. @xcite added irrelevant context to an arithmetic reasoning dataset, finding that including irrelevant information distracted the model's predictions. @xcite proposed a method that could reliably generate adversarial attack suffixes, yielding adversarial prompts that exhibit high transferability. in this study, we investigate adversarial training concerning llms in response to various retrieval noises, aiming to efficiently obtain adversarial examples that enhance model robustness while reducing training overhead. we construct noisy adversarial examples by sampling or paraphrasing the original dataset. this approach ensures more dependable and precise outputs even when confronted with imperfect retrieved contexts. ## conclusion this work initially investigated retrieval noises in ralms and categorized them into three distinct types, reflecting real-world environments. in addition, we introduced raat as a solution to address the noise robustness challenges faced by ralms, which leveraged adaptive adversarial learning and multi-task learning to enhance the model's capability. moreover, we established a benchmark to verify the effectiveness of raat based on three open-domain qa datasets. experimental results demonstrate substantial improvements in f1 and em scores for the llama2 7b model fine-tuned with raat across diverse noise conditions.