Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models (2024)

Abstract: Retrieval-augmented language model (RALM) represents a significant advancement in mitigating factual hallucination by leveraging external knowledge sources. However, the reliability of the retrieved information is not always guaranteed, and the retrieval of irrelevant data can mislead the response generation. Moreover, standard RALMs frequently neglect their intrinsic knowledge due to the interference from retrieved information. In instances where the retrieved information is irrelevant, RALMs should ideally utilize their intrinsic knowledge or, in the absence of both intrinsic and retrieved knowledge, opt to respond with “unknown” to avoid hallucination. In this paper, we introduces Chain-of-Note (CoN), a novel approach to improve robustness of RALMs in facing noisy, irrelevant documents and in handling unknown scenarios. The core idea of CoN is to generate sequential reading notes for each retrieved document, enabling a thorough evaluation of their relevance to the given question and integrating this information to formulate the final answer. Our experimental results show that GPT-4, when equipped with CoN, outperforms the Chain-of-Thought approach. Besides, we utilized GPT-4 to create 10K CoN data, subsequently trained on smaller models like OPT and LLaMa-2. Our experiments across four open-domain QA benchmarks show that fine-tuned RALMs equipped with CoN significantly outperform standard fine-tuned RALMs.

## introduction retrieval-augmented language models (ralms) represent a novel framework that significantly advances large language models @xcite by addressing key limitations such as reducing factual hallucinations @xcite , injecting up-to-date knowledge in a plug-and-play manner @xcite , and enhancing domainspecific expertise @xcite . wikipedia: … "it must have been love" is a song written by per gessle and performed by the swedish pop duo roxette. the power ballad became the duo's third number one hit in united states. wikipedia: … "it must be love" is a song written and originally recorded in 1971 by labi siffre. it was also recorded by ska/pop band madness in 1981. the answer is labi siffre. the first passage confirms that roxette is the performer of "it must have been love." however, the second passage refers to "it must be love," which is a different song and irrelevant to the question. thus, the answer is roxette. ## related work retrieval-augmented language models (ralms) represent a significant advancement in natural language processing, combining the power of large language models with the specificity and detail provided by external knowledge sources @xcite @xcite . recent studies highlight the impact of context relevance on language model performance @xcite @xcite . notably, @xcite demonstrated that incorporating random or irrelevant contexts could adversely affect qa performance. in contrast, @xcite discovered that adding irrelevant context to exemplars or task-specific instructions can sometimes enhance model performance, implying that models might intrinsically possess capabilities, developed during pre-training, to manage such scenarios. most pertinent to our research is the study by @xcite , which focused on training ralms to disregard irrelevant contexts. this approach, while distinct from our proposed solution, underscores the importance of context relevance in enhancing the effectiveness of ralms. besides, we present more related chain-of-xs approaches (e.g., chain-of-thought (cot) @xcite ) in the appendix a.1.1 and a.2. ## conclusion in this paper, we introduce the chain-of-note (con) framework, a novel methodology designed to enhance the robustness of ralms. the central concept of con revolves around the generation of sequential reading notes for each retrieved document. this process allows for an in-depth assessment of document relevance to the posed question and aids in synthesizing this information to craft the final answer. our experiments show that gpt-4, when equipped with con, outperforms the chain-of-thought approach. besides, we utilized gpt-4 to create 10k con data, subsequently trained on a llama-2 7b model. our experiments across four open-domain qa benchmarks show that ralms equipped with con significantly outperform standard fine-tuned ralms.