Toward Robust RALM s: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models (2024)

Abstract: Retrieval Augmented Language Models (RALMs) have gained significant attention for their ability to generate accurate answers and improve efficiency. However, RALMs are inherently vulnerable to imperfect information due to their reliance on the imperfect retriever or knowledge source. We identify three common scenarios—unanswerable, adversarial, conflicting—where retrieved document sets can confuse RALMs with plausible real-world examples. We present the first comprehensive investigation to assess how well RALMs detect and handle such problematic scenarios. Among these scenarios, to systematically examine adversarial robustness we propose a new adversarial attack method, Generative model-based ADVersarial attack (GenADV) and a novel metric Robustness under Additional Document (RAD). Our findings reveal that RALMs often fail to identify the unanswerability or contradiction of a document set, which frequently leads to hallucinations. Moreover, we show that the addition of an adversary significantly degrades RALM’s performance, with the model becoming even more vulnerable when the two scenarios overlap (adversarial+ unanswerable). Our research identifies critical areas for assessing and enhancing the robustness of RALMs, laying the foundation for the development of more robust models.1

## introduction large language models (llms) are becoming the foundation for various nlp tasks @xcite @xcite . notably, in open-domain question answering (qa) tasks @xcite that require substantial knowledge, retrieval augmented language models (ralms) have proven to be highly effective @xcite @xcite @xcite . ralms generate answers based on external knowledge and show competitive performance with simple in-context setting without additional training. @xcite however, ralms are known to be sensitive to the quality of external information due to their reliance on it. common issues such as imperfect retriever or contaminated knowledge sources can affect the robustness of ralm @xcite @xcite . for example, figure 1 shows different types of real-world scenarios, illustrating both incorrect responses by a ralm and their ideal responses. if a query ''what is the tallest building in the world?'' retrieves documents that do not contain the answer, the ralm should classify it as ''unanswerable'' rather than parroting incorrect answer in the document (unanswerable scenario). furthermore, ralm should ignore documents that do not contain the answer even if it appears to be related to the question (e.g., ''the tallest mountain in the world is mount everest'') and instead extract the answer from documents that do (adversarial scenario). in cases where documents provide conflicting answers (e.g., ''the tallest building in the world is burj khalifa. . . the taipei 101 is known for the tallest building in the world''), the model should respond with ''conflict'' as the ralm cannot surely identify which is the correct answer (conflicting scenario). previous studies have primarily focused on one of these three scenarios @xcite @xcite or on inconsistencies within individual documents @xcite , addressed methods for mitigating problems @xcite @xcite or examined the relationship between parametric knowledge and documents rather than interactions across multiple retrieved documents @xcite . in contrast, our work systematically analyzes the robustness of ralms for open-domain qa in scenarios of imperfect information, which is a critical factor of ralm's robustness. we define each type of scenario and develop a perturbation method to generate imperfect documents, particularly for simulating adversarial and conflict scenarios in open-domain qa. we also introduce appropriate robustness metrics for each experiment. for the unanswerable scenario, we categorize examples into answerable and unanswerable based on whether the answer strings appear in the retrieved documents, and then measure the accuracy for each subset. our findings show the challenges that ralms encounter in accurately identifying unanswerability. this often leads to hallucination or parroting an incorrect answer in the documents instead of saying unanswerable. for the adversarial scenario ( our results indicate that ralms are vulnerable to adversarial information, particularly genadv. also, they show higher vulnerability in adversarial-unanswerable situations where an unanswerable example contain adversarial information in the retrieved documents. concerning conflicting documents ( in-context ralms traditionally, retrieval-augmented language models (ralms) involved training a generator to generate answers based on the retrieved documents @xcite @xcite . however, recent discoveries show that llms can be used as generators for ralms in an in-context setting, without additional training, by simply concatenating retrieved documents to the query @xcite @xcite . since this method is highly efficient and promising in open-domain qa, we will use the in-context ralms. ## conclusion in this study, we conducted a comprehensive evaluation of the robustness of ralms under various imperfect retrieval conditions. our findings revealed significant challenges faced by these models in handling unanswerability, as well as adversarial and conflicting information. through extensive experiments, we demonstrated that ralms struggle to identify unanswerable scenarios, often hallucinating responses even when the retrieved documents do not contain the answer. additionally, we introduced a new method, genadv, for generating adversarial information, which proved highly effective in distracting the models and causing them to abandon correct answers. furthermore, our results highlighted the vulnerability of ralms to conflicting information, as they exhibited poor performance in both detecting conflicts within the retrieved documents and generating accurate responses in the presence of such conflicts. our study provides a foundation for evaluating the robustness of ralms, crucial for their safe use. based on this foundation, further exploration into developing robust models is necessary.