Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
Authors: Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, Sercan O Arik
Accepted at: acl25

ASTUTE RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

Fei Wang12*   Xingchen Wan1   Ruoxi Sun1   Jiefeng Chen1   Sercan Ö. Arik1
1Google Cloud AI Research   2University of Southern California
fwang598@usc.edu   {xingchenw,ruoxis,jiefengc,soarik}@google.com

Abstract
Retrieval-augmented generation (RAG), while effective in integrating external knowledge to enhance large language models (LLMs), can be undermined by imperfect retrieval, which may introduce irrelevant, misleading, or even malicious information. Despite its importance, previous studies have rarely explored the behavior of RAG with errors from imperfect retrieval, and how potential conflicts arise between the LLMs’ internal knowledge and external sources. We show that imperfect retrieval augmentation might be inevitable and quite harmful, through controlled analysis under realistic conditions. Knowledge conflicts between LLM-internal and external knowledge from retrieval is a bottleneck to overcome in the post-retrieval stage of RAG. To render LLMs resilient to imperfect retrieval, we propose ASTUTE RAG, a novel RAG approach that adaptively elicits essential information from LLMs’ internal knowledge, iteratively consolidates internal and external knowledge with source-awareness, and finalizes the answer according to information reliability. Our experiments with Gemini and Claude demonstrate that ASTUTE RAG significantly outperforms previous robustness-enhanced RAG methods. Notably, ASTUTE RAG is the only approach that matches or exceeds the performance of LLMs without RAG under worst-case scenarios. ASTUTE RAG effectively resolves knowledge conflicts, improving the reliability and trustworthiness of RAG systems.

1 Introduction
Retrieval-augmented generation (RAG) is commonly used for large language models (LLMs) to tackle knowledge-intensive tasks (Guu et al., 2020; Lewis et al., 2020). Prior works mainly leverage RAG to address the inherent knowledge limitations of LLMs, effectively integrating missing information and grounding to reliable sources.

However, recent research has highlighted a significant drawback that RAG might rely on imperfect retrieval, including irrelevant, misleading, or even malicious information (Fig. 1), which eventually leads to inaccurate LLM responses (Chen et al., 2024a; Zou et al., 2024). Moreover, recent studies have shown that retrieval augmentation can confuse LLMs when retrieved passages are conflicting with LLMs’ parametric knowledge (Tan et al., 2024; Xie et al., 2024; Jin et al., 2024). These pose significant challenges to the trustworthiness of RAG.

To address imperfect retrieval, earlier work seeks to improve the retrieval approaches, such as dynamic and iterative retrieval (Jiang et al., 2023; Asai et al., 2023; Yan et al., 2024). However, the occurrence of imperfect retrieval is still inevitable, due to corpus quality limitations (Shao et al., 2024), the reliability of retrievers (Dai et al., 2024), and the complexity of queries (Su et al., 2024). Consequently, recent work shifts the focus to the generation stage, seeking to reduce the negative impact of noisy retrieved passages (Xiang et al., 2024; Wei et al., 2024). Another line of research at generation stage, motivated by knowledge conflicts, has explored complementing retrieved passages with LLM-generated passages (Yu et al., 2023a; Zhang et al., 2023) or deactivating RAG when the retrieved passages are of insufficient quality (Xu et al., 2024; Mullen et al., 2023; Jeong et al., 2024).

Despite the previous work on the impact of imperfect retrieval and knowledge conflicts at RAG generation stage, quantitative analyses lack on two crucial real-world aspects: (i) the relation between retrieval quality and occurrence of knowledge conflicts, and (ii) the extent to which retrieved passages and LLMs’ parametric knowledge can correct each other. Method-wise, existing approaches for mitigating RAG failures caused by imperfect retrieval and knowledge conflicts have not yet yielded a training-free method capable of explicitly analyzing conflicting knowledge across various internal
Figure 1: Knowledge conflicts between the LLMs’ internal knowledge and retrieved knowledge from external sources. We report the overall results with Claude under the setting in Sec. 5.1.

and external sources, and achieving worst-case robustness for black-box LLMs.

In this paper, we first conduct comprehensive analyses to investigate the relation between imperfect retrieval and knowledge conflicts, and examine the frequency of external and LLMs’ internal knowledge mutually correcting each other (Sec. 3). On a diverse range of general, domain-specific, and long-tail questions from NQ (Kwiatkowski et al., 2019), TriviaQA (Joshi et al., 2017), BioASQ (Tsatsaronis et al., 2015), and PopQA (Mallen et al., 2023), we observe that imperfect retrieval is widespread even with an adept real-world search engine, leading to the impeded performance of RAG.\footnote{such as Google Search with Web as corpus} Retrieval precision is tightly correlated with the knowledge conflict rate. Mutual correction between the LLM’s knowledge and external knowledge is crucial for recovering from RAG failures. Our findings underscore the potential severity of imperfect retrieval in real-world RAG and highlight the widespread existence of knowledge conflicts as the bottleneck.

We propose ASTUTE RAG, a novel RAG approach designed for resilience to imperfect retrieval augmentation, while preserving RAG grounding effect when retrieval is reliable (Sec. 4). ASTUTE RAG effectively differentiates between consistent and conflicting information from the LLM’s internal knowledge and the externally retrieved passages, assesses their reliability, and ensures proper integration of trustworthy information. ASTUTE RAG first adaptively elicits LLMs’ knowledge and then conducts source-aware knowledge consolidation. The desiderata is combining consistent information, identifying conflicting information, and filtering out irrelevant information. Finally, ASTUTE RAG proposes answers based on consistent information and compares them to determine the final answer. Our experiments with various LLMs (Claude, Gemini and Mistral), demonstrate superior performance of ASTUTE RAG compared to previous RAG approaches designed for robustness (Sec. 5). ASTUTE RAG consistently outperforms baselines across different retrieval quality levels. Notably, ASTUTE RAG is the only RAG method that achieves performance comparable to or even surpassing retrieval-free mode of LLMs under the worst-case scenario where all retrieved passages are unhelpful. Further analysis reveals the effectiveness of ASTUTE RAG in resolving knowledge conflicts.

In summary, our core contributions are threefold. First, we provide quantitative analyses and novel insights for the connection among imperfect retrieval, knowledge conflicts, and RAG failures under real-world conditions. Second, we propose ASTUTE RAG, which explicitly analyzes LLM-internal and external knowledge in-context, assesses their reliability, and recovers from RAG failures with black-box access. Third, with experiments with various LLMs and datasets, we demonstrate the effectiveness of ASTUTE RAG in improving robustness and trustworthiness, even in the most challenging scenarios.

2 Related Work

RAG aims to address the inherent knowledge limitation of LLMs with passages retrieved from external sources of information such as private corpora or public knowledge bases (Guu et al., 2020; Lewis et al., 2020; Borgeaud et al., 2022). Given the
Figure 2: Imperfect retrieval (samples with low retrieval precision) is prevalent in real-world RAG.

Figure 3: Conflicting rate between answers from LLMs with and without RAG on different retrieval precision.

widespread real-world adoption of RAG, including risk-sensitive domains, the negative impact of noisy information within retrieved passages has garnered increasing attention (Cuconasu et al., 2024). Recent work explored enhancing the robustness of RAG systems against noise from various perspectives, including training LLMs with noisy context (Yu et al., 2023b; Yoran et al., 2024; Pan et al., 2024; Fang et al., 2024), training small models to filter out irrelevant passages (Wang et al., 2023b; Xu et al., 2023), passage reranking (Yu et al., 2024; Glass et al., 2022), dynamic and iterative retrieval (Jiang et al., 2023; Asai et al., 2023; Yan et al., 2024), query rewriting (Ma et al., 2023), and speculative drafting (Wang et al., 2024). These focus on distinct modules or stages of RAG systems and are orthogonal to our work.

Our work focuses on enhancing RAG robustness at the post-retrieval stage, after retrieved passages have been provided. On this, RobustRAG (Xiang et al., 2024) aggregates answers from each independent passage to provide certifiable robustness. InstructRAG (Wei et al., 2024) instructs the LLM to provide a rationale connecting the answer with information in passages. MADRA (Wang et al., 2023a) applies multi-agent debate to select helpful evidence. However, these do not explicitly incorporate internal knowledge to recover from RAG failures and therefore might severely suffer when the majority of retrieved passages have issues. For emphasizing internal knowledge of LLMs in RAG, recent work explored using LLM-generated passage as context (Yu et al., 2023a), training models to match generated and retrieved passages (Zhang et al., 2023), adaptively switching between LLMs with and without RAG (Xu et al., 2024; Mailen et al., 2023; Jeong et al., 2024), and combining answers through contrastive decoding (Zhao et al., 2024; Jin et al., 2024). Different from prior work, we provide a systematic framework on connecting imperfect retrieval, knowledge conflicts, and RAG failures. Specifically focusing on the imperfect context setting, our method is training-free and applicable to black-box LLMs, explicitly analyzes internal and external knowledge in-context, and offers broader usability and adaptability.

3 The Pitfall of RAG

To better showcase common real-world challenges and motivate improved methodological designs, we evaluate retrieval quality, the occurrence of knowledge conflicts, their relationship, and the mutual correction between external and internal knowledge using a controlled dataset derived from NQ, TriviaQA, BioASQ, and PopQA, datasets widely used for RAG in prior work (Xiang et al., 2024; Wei et al., 2024; Asai et al., 2023). Different from prior work, our analysis is based on real-world retrieval results with Google Search2 as the retriever and the Web as the corpus. Overall, we sample 1K instances, each with 10 retrieved passages.

Imperfect retrieval and knowledge conflicts are common and harmful. Our initial observations

2https://developers.google.com/custom-search/v1/overview
Figure 4: Overview of the ASTUTE RAG framework. ASTUTE RAG is designed to better combine the information from the external sources (e.g. web, domain-specific corpora) and internal knowledge of the LLMs by employing a consolidation mechanism to address the conflicts, which eventually leads to superior generation quality.

are consist with prior work. As shown in Fig. 2, the retrieval precision3 is generally low - roughly 70% retrieved passages do not directly contain true answers, consistent with prior work demonstrating the often imperfect nature of retrieval results (Thakur et al., 2024; Su et al., 2024). With Claude 3.5 Sonnet as the LLM, Fig. 1 shows that 19.2% of the overall data exhibit knowledge conflicts, consistent with prior work demonstrating the prevalence of such conflicts across various scenarios (Pham et al., 2024; Xie et al., 2024; Longpre et al., 2021). Moreover, we observe strong correlations between retrieval precision and RAG performance (Fig. 7) and between the occurrence of knowledge conflicts and RAG performance (Fig. 8), findings consistent with prior work on these respective topics (Chen et al., 2024a; Xie et al., 2024).

Lower retrieval precision increases knowledge conflicts in general. As shown in Fig. 3, most advanced LLMs exhibit the highest conflict rates when retrieval precision is as low as 10%. Subsequently, the conflict rate generally decreases as precision increases, although some fluctuations may occur. This trend is generally applicable to the studied LLMs with different training processes. Notably, when retrieval precision is 0%, conflict rates tend to be significantly lower. This suggests that limited external knowledge for the query results in more irrelevant passages rather than incorrect ones.

Internal and external knowledge can correct each other to a comparable extent. Among the conflicting cases, the internal knowledge is correct on 47.4% of them, while the external knowledge is correct on the remaining 52.6%. These results emphasize the importance of effectively combining the internal and external knowledge to overcome the inherent limitation of relying solely on either source. However, previous work (Tan et al., 2024; Xie et al., 2024; Jin et al., 2024) shows that LLMs often select knowledge based on unreliable shortcuts, so simply presenting LLM-generated passages in the context may not help.

4 ASTUTE RAG

We first provide an overview of ASTUTE RAG (Sec. 4.1). Subsequently, we delve into the three major steps of ASTUTE RAG, including adaptive generation of internal knowledge (Sec. 4.2), source-aware knowledge consolidation (Sec. 4.3), and answer finalization (Sec. 4.4).

4.1 Overview

Our objective is to mitigate the effects of imperfect retrieval augmentation, resolve knowledge conflicts between the LLM’s internal knowledge and external sources (such as custom/public corpora and knowledge bases), and ultimately produce more accurate and reliable responses from LLMs. Given a set of retrieved passages from external sources \( E = [e_1, \ldots, e_n] \), a pre-trained LLM \( \mathcal{M} \) (accessible through prediction-only APIs, encompassing commercial black-box ones), and a query \( q \), the task is to generate the corresponding correct an-

3Ratio of passages directly contain true answers.
Algorithm 1 ASTUTE RAG

Require: Query q, Retrieved Passages \( E = [e_1, \ldots, e_n] \), Large Language Model \( \mathcal{M} \), Number of Iteration t, Max Number of Generated Passages \( \hat{m} \), Prompt Templates \( p_{gen}, p_{con}, p_{ans} \)
1: Adaptively generate passages: \( I \leftarrow \mathcal{M}(p_{gen}, q, \hat{m}) \)  ▷ Sec. 4.2
2: Combine internal and external passages: \( D_0 \leftarrow E \oplus I \)
3: Assign passage sources: \( S_0 \leftarrow [1_{\{d \in E\}} \text{for } d \text{ in } D_0] \)
4: if \( t > 1 \) then
    5:     for \( j = 1, \ldots, t - 1 \) do
        6:         Consolidate knowledge: \( \langle D_{j+1}, S_{j+1} \rangle \leftarrow \mathcal{M}(p_{con}, q, \langle D_0, S_0 \rangle, \langle D_j, S_j \rangle) \)  ▷ Sec. 4.3
    7:     end for
    8:     Finally consolidate and answer: \( a \leftarrow \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle, \langle D_{t-1}, S_{t-1} \rangle) \)  ▷ Sec. 4.4
9: else
10:     Consolidate knowledge and finalize the answer: \( a \leftarrow \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle) \)
11: end if
12: return \( a \)

swer \( a^* \). Notably, this setting is orthogonal to prior work on improving the retriever, training LLMs, or conducting adaptive retrieval, which are mainly preliminary steps.

ASTUTE RAG is designed to better leverage collective knowledge from both internal knowledge of LLMs and external corpus, for more reliable responses. As shown in Fig. 4 and Alg. 1, ASTUTE RAG starts from acquiring the most accurate, relevant, and thorough passage set from the LLMs’ internal knowledge. Then, internal and external knowledge are consolidated in an iterative way, by comparing the generated and retrieved passages. Finally, the reliability of conflicting information is compared and the final output is generated according to the most reliable knowledge.

4.2 Adaptive Generation of Internal Knowledge

In the first step, we elicit internal knowledge from LLMs. This LLM-internal knowledge, reflecting the consensus from extensive pre-training and instruction-tuning data, can supplement any missing information from the limited set of retrieved passages and enable mutual confirmation between LLM-internal and external knowledge. This is especially valuable when the majority of retrieved passages might be irrelevant or misleading. Specifically, we prompt LLMs to generate passages based on the given question \( q \), following Yu et al. (2023a). While Yu et al. (2023a) primarily focused on generating diverse internal passages, we emphasize the importance of reliability and trustworthiness of generated passages. To achieve this goal, we enhance the original method with constitutional principles and adaptive generation.

Inspired by Bai et al. (2022), we provide constitutional principles indicating the desired properties of internal passages in the prompt \( p_{gen} \) (see Appx. A for details) to guide their generation, emphasizing that the generated passages should be accurate, relevant, and hallucination-free. Moreover, we allow the LLM to perform adaptive generation of passages in its internal knowledge. The LLM can decide how many passages to generate by itself. Rather generating a fix number of passages, we request the LLM to generate at most \( \hat{m} \) passages, each covering distinct information, and to directly indicate if no more reliable information is available. This adaptive approach allows the LLM to generate fewer passages (or even no passages at all) when the useful information within internal knowledge is limited and more passages when there are multiple feasible answers in the internal knowledge. In this step, the LLM generates \( m \leq \hat{m} \) passages based on its internal knowledge:

\[
I = [i_1, \ldots, i_m] = \mathcal{M}(p_{gen}, q, \hat{m}).
\]

4.3 Iterative Source-aware Knowledge Consolidation

In the second step, we employ the LLM to explicitly consolidate information from both passages generated from its internal knowledge and passages retrieved from external sources. Initially, we combine passages from both internal and external knowledge sources \( D_0 = E \oplus I \).

We additionally ensure source-awareness by providing the source of each passage to LLMs when consolidating knowledge. The source information (internal or external, such as a website) is helpful in assessing the reliability of passages. Here, we provide the passage source as \( S_0 = [1_{\{d \in E\}} \text{for } d \text{ in } D_0] \). To consolidate knowledge, we prompt the LLM (with \( p_{con} \) in Appx. A) to identify consistent information across passages, detect conflicting information between each group
of consistent passages, and filter out irrelevant information. This step would regroup the unreliable knowledge in input passages into fewer refined passages. The regrouped passages also attribute their source to the corresponding input passages:

\[
(D_{j+1}, S_{j+1}) = \mathcal{M}(p_{con}, q, \langle D_0, S_0 \rangle, \langle D_j, S_j \rangle).
\]

We find that this is especially helpful in comparing the reliability of conflicting knowledge and addressing knowledge conflicts. This knowledge consolidation process can run iteratively for \( t \) times to improve better utilization of the retrieved context.

4.4 Answer Finalization

In the last step, we prompt the LLM (with \( p_{ans} \) in Appx. A) to generate one answer based on each group of passages (\( \langle D_t, S_t \rangle \)), and then compare their reliability and select the most reliable one as the final answer. This comparison allows the LLM to comprehensively consider knowledge source, cross-source confirmation, frequency, and information thoroughness when making the final decision. Notably, this step can be merged into the last knowledge consolidation step to reduce the inference complexity (the amount of prediction API calls) using a combined prompt:

\[
a = \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle, \langle D_t, S_t \rangle).
\]

When \( t = 1 \), the initial passages will be input to the model directly for knowledge consolidation and subsequent answering: \( a = \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle) \).

5 Experiments

We evaluate the effectiveness of ASTUTE RAG on overcoming imperfect retrieval augmentation and addressing knowledge conflicts. In this section, we introduce the experiment setting (Sec. 5.1), compare the performance of ASTUTE RAG with various baselines on diverse datasets (Sec. 5.2), and provide in-depth analyses (Sec. 5.3).

5.1 Experimental Settings

Datasets and metrics. We consider datasets encompass general questions, domain-specific questions, long-tail questions, as well as both short-form and long-form formats, following prior work (Xiang et al., 2024; Wei et al., 2024). On NQ, TriviaQA, BioASQ, and PopQA, we provide 10 passages collected with Google Search from the Web for each instance. For long-form QA, we use ASQA (Stelmakh et al., 2022). We also evaluate on RGB (Chen et al., 2024a). We choose the English subset (refined version) focusing on noise robustness. For each instance, we select five top negative passages to form a worst-case scenario. Following prior work, we report the accuracy by string match. More details are in Appx. B.

Models and General Settings. We conduct experiments on advanced proprietary and open-source LLMs of different scales, including Claude 3.5 Sonnet (claude-3-5-sonnet@20240620),5 Gemini 1.5 Pro (gemini-1.5-pro-002),6 Mistral-Large (128B; version 2407), and Mistral-Nemo (12B; version 2407). The generation temperature is set to 0 and the maximum output tokens is set to 1,024. All experiments are under the zero-shot setting for controlled evaluation.

Baselines. We compare ASTUTE RAG with various RAG methods designed for enhanced robustness. USC (Chen et al., 2024b) is a self-consistency method that samples multiple LLM responses and aggregates the answers. It provides a reference of naive improvements using additional API calls. Genread (Yu et al., 2023a) augments retrieved passages with LLM-generated passages without explicit consolidation process. RobustRAG (Xiang et al., 2024) aggregates answers from independent passages to provide certifiable robustness. We use the best-performing keyword aggregation variant. InstructRAG (Wei et al., 2024) instructs the LLM to provide a rationale connecting the answer with information in passages. For a fair comparison, no training is applied. Self-Route (Xu et al., 2024) adaptively switches between LLMs with and without RAG.7 It provides a reference of switching between LLMs’ internal and external knowledge.

Implementation Details. The prompt templates for ASTUTE RAG can be found in Appx. A. By default, we set \( t = 1 \) and \( \hat{m} = 1 \) to limit the number of additional tokens used. Results with larger \( t \) and \( \hat{m} \) are discussed in Sec. 5.3.

5.2 Main Results

Performance under real-world retrieval. Tab. 1 presents the results with real-world retrieval aug-

5https://www.anthropic.com/news/claude-3-5-sonnet
6https://deepmind.google/technologies/gemini/pro/
7The original Self-Route switches between RAG and long-context LLMs, while our implementation switches between RAG and No RAG according to our problem formulation.
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="5">Claude 3.5 Sonnet (20240620)</th>
    <th colspan="5">Gemini 1.5 Pro (002)</th>
  </tr>
  <tr>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>47.1</td>
    <td>82.0</td>
    <td>50.4</td>
    <td>29.8</td>
    <td>54.5</td>
    <td>44.8</td>
    <td>80.2</td>
    <td>45.8</td>
    <td>25.3</td>
    <td>51.3</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>44.4</td>
    <td>76.7</td>
    <td>58.0</td>
    <td>36.0</td>
    <td>55.5</td>
    <td>42.7</td>
    <td>76.0</td>
    <td>55.2</td>
    <td>33.7</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td><b>USC (Chen et al., 2024b)</b></td>
    <td>48.1</td>
    <td>80.2</td>
    <td><b>61.5</b></td>
    <td>37.6</td>
    <td>58.7</td>
    <td>46.4</td>
    <td>76.7</td>
    <td><b>58.4</b></td>
    <td>37.6</td>
    <td>56.4</td>
  </tr>
  <tr>
    <td>GenRead (Yu et al., 2023a)</td>
    <td>42.0</td>
    <td>74.2</td>
    <td>57.0</td>
    <td>34.3</td>
    <td>53.6</td>
    <td>45.1</td>
    <td>77.4</td>
    <td>54.9</td>
    <td>34.3</td>
    <td>54.7</td>
  </tr>
  <tr>
    <td>RobustRAG (Xiang et al., 2024)</td>
    <td>47.8</td>
    <td>78.1</td>
    <td>56.3</td>
    <td>37.1</td>
    <td>56.5</td>
    <td>34.2</td>
    <td>67.5</td>
    <td>44.1</td>
    <td>32.0</td>
    <td>45.6<sup>a</sup></td>
  </tr>
  <tr>
    <td>InstructRAG (Wei et al., 2024)</td>
    <td>47.1</td>
    <td>83.0</td>
    <td>58.0</td>
    <td>41.0</td>
    <td>58.8</td>
    <td>46.8</td>
    <td>80.6</td>
    <td>54.9</td>
    <td>34.8</td>
    <td>56.1</td>
  </tr>
  <tr>
    <td>Self-Route (Xu et al., 2024)</td>
    <td>47.5</td>
    <td>78.8</td>
    <td>59.1</td>
    <td>41.0</td>
    <td>58.1</td>
    <td>47.5</td>
    <td>79.9</td>
    <td>58.0</td>
    <td>38.2</td>
    <td>57.6</td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td><b>52.2</b></td>
    <td><b>84.1</b></td>
    <td>60.1</td>
    <td><b>44.4</b></td>
    <td><b>61.7</b></td>
    <td><b>50.2</b></td>
    <td><b>81.6</b></td>
    <td>58.0</td>
    <td><b>40.5</b></td>
    <td><b>59.2</b></td>
  </tr>
  <tr>
    <th colspan="11">Mistral-Large (2407), 128B</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>46.8</td>
    <td>79.5</td>
    <td>43.7</td>
    <td>24.7</td>
    <td>51.1</td>
    <td>29.8</td>
    <td>67.8</td>
    <td>34.3</td>
    <td>23.0</td>
    <td>40.2</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>43.1</td>
    <td>77.4</td>
    <td>55.9</td>
    <td>36.0</td>
    <td>54.7</td>
    <td>39.3</td>
    <td>66.8</td>
    <td>49.0</td>
    <td>32.6</td>
    <td>48.3</td>
  </tr>
  <tr>
    <td><b>USC (Chen et al., 2024b)</b></td>
    <td><b>51.2</b></td>
    <td>80.9</td>
    <td><b>61.5</b></td>
    <td>36.0</td>
    <td>59.5</td>
    <td>29.5</td>
    <td>66.1</td>
    <td>36.0</td>
    <td>20.2</td>
    <td>39.6</td>
  </tr>
  <tr>
    <td>GenRead (Yu et al., 2023a)</td>
    <td>40.7</td>
    <td>73.1</td>
    <td>55.6</td>
    <td>35.4</td>
    <td>52.7</td>
    <td>38.6</td>
    <td>68.9</td>
    <td>48.3</td>
    <td><b>33.7</b></td>
    <td>48.7</td>
  </tr>
  <tr>
    <td>RobustRAG (Xiang et al., 2024)</td>
    <td>42.7</td>
    <td>77.7</td>
    <td>50.4</td>
    <td>34.8</td>
    <td>53.0</td>
    <td>35.6</td>
    <td>71.7</td>
    <td>44.1</td>
    <td>27.5</td>
    <td>46.4</td>
  </tr>
  <tr>
    <td>InstructRAG (Wei et al., 2024)</td>
    <td>45.4</td>
    <td>80.6</td>
    <td>57.3</td>
    <td>36.5</td>
    <td>56.7</td>
    <td>38.3</td>
    <td>61.8</td>
    <td>50.4</td>
    <td>23.6</td>
    <td>45.5</td>
  </tr>
  <tr>
    <td>Self-Route (Xu et al., 2024)</td>
    <td>45.4</td>
    <td>77.7</td>
    <td>57.3</td>
    <td>38.2</td>
    <td>56.2</td>
    <td>41.4</td>
    <td>73.5</td>
    <td><b>51.8</b></td>
    <td>30.9</td>
    <td>51.2</td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td>50.2</td>
    <td><b>82.7</b></td>
    <td>58.4</td>
    <td><b>42.1</b></td>
    <td><b>59.9</b></td>
    <td><b>42.7</b></td>
    <td><b>73.9</b></td>
    <td>49.3</td>
    <td>32.6</td>
    <td><b>51.3</b></td>
  </tr>
</table>

Table 1: Main results on Claude 3.5 Sonnet, Gemini 1.5 Pro, Mistral-Large, and Mistral-Nemo under zero-shot setting, showing the accuracy of benchmarked alternatives vs. ASTUTE RAG. Best scores are in bold. Note that USC consumes approximately three times more tokens than other RAG methods, and is not directly comparable.

Figure 5: Performance on ASQA.

Figure 6: Worst-case performance of Claude on RGB. ASTUTE RAG reaches a performance close to No RAG, while other RAG systems are far behind.

mentation of various LLMs. We find that retrieved passages might not always bring benefits – on NQ and TriviaQA, RAG performance lags behind No RAG for advanced LLMs. We attribute this questions being covered by the LLM’s internal knowledge and the noise in retrieval results misleading the LLM. In contrast, on BioASQ and PopQA, which focus on domain-specific and ‘long-tail’ questions, RAG significantly improves the LLM performance. Due to imperfect retrieval augmentation, however, the absolute performance still remains to be unsatisfactory. Among all baselines, no single method consistently outperforms others across all datasets and LLMs. This observation highlights these baselines being tailored to distinct settings and not being universally applicable. Overall, InstructRAG and Self-Route demonstrate relatively superior performance among other alternatives. ASTUTE RAG consistently outperforms baselines across all LLMs in terms of overall accuracy. The relative improvement compared to the best baseline is 6.85% for Claude and 4.13% for Gemini, with the improvements in domain-specific questions being much higher. These highlight the effectiveness of ASTUTE RAG in overcoming imperfect retrieval augmentation and knowledge conflicts. Additionally, we observe consistent improvements on the open-source Mistral models. The results demonstrate that ASTUTE RAG generalizes well to LLMs of smaller sizes.

Performance on long-form QA. We conduct additional experiments on the long-form QA dataset, ASQA. Fig. 5 demonstrates that ASTUTE RAG consistently achieves significant improvements, re-
inforcing its effectiveness across diverse scenarios.

Worst-case performance on RGB. Fig. 6 presents the results under the worst-case setting on RGB where all retrieved documents are negative, to demonstrate robustness. The performance gap between RAG and No RAG exceeds 50 points, highlighting the detrimental impact of imperfect retrieval results and emphasizing the importance of providing robust safeguards against worst-case scenarios. While the baseline RAG methods outperform the original RAG, they still obviously fall behind ‘No RAG’. ASTUTE RAG is the only RAG method that reaches a performance close to ‘No RAG’, further supporting its effectiveness in addressing imperfect retrieval augmentation.

5.3 Analyses

We conduct in-depth analyses using Claude following the setting of Tab. 1.

The impact of retrieval precision. As shown in Fig. 7, ASTUTE RAG achieves consistently better performance across different retrieval precision regimes, indicating its effectiveness in improving RAG trustworthiness in broad scenarios. Notably, ASTUTE RAG does not sacrifice performance gain under high retrieval quality in exchange for improvement under low retrieval quality. When the retrieval quality is extremely low (close to zero precision), all other RAG variants underperform the ‘No RAG’ baseline, except for ASTUTE RAG.

Addressing knowledge conflicts. We split our collected data into three subset according to the answers with and without RAG: the answers from two can be (i) both correct, (ii) both incorrect, or (iii) conflicting with one being correct. The results are shown in Fig. 8. On the conflicting subset, ASTUTE RAG successfully chooses the correct answer in approximately 80% of cases, being the most effective one in addressing knowledge conflicts. Notably, ASTUTE RAG even brings performance improvement on the subset where neither internal nor external knowledge alone leads to the correct answer. This indicates that ASTUTE RAG can effectively combine partially-correct information from LLM-internal and external knowledge.

Benefits of more consolidation iteration. For efficiency, we employ a single iteration of knowledge consolidation in our main experiments. However, incorporating multiple iterations has the potential to further enhance model performance as shown in Fig. 9. The magnitude of this improvement diminishes as \( t \) increases, indicating that the knowledge has been better presented and less improvement space left after each iteration.

Efficiency in tokens consumed and API calls. As a proxy to overall prediction cost and latency, we present the average number of tokens and API calls used per instance in Fig. 10 and Fig. 11. ASTUTE RAG incurs only a marginal cost increase, <5%, while delivering substantial improvement, >11%, compared to the RAG baseline.

Effectiveness of adaptive generation. The results in Tab. 2 illustrate the model’s performance when varying the maximum number of passages generated. The design of adaptive generation has been effectively reflected, as the number of generated passages is dynamically adjusted leading to \( m < \hat{m} \). Notably, the number of generated passages can be controlled by \( \hat{m} \), and results show that the system does not generate passages excessively.

Figure 7: Performance across different retrieval precision buckets. ASTUTE RAG is consistently better.

![Line chart showing performance across different retrieval precision buckets for Astute RAG, Self-Route, Instruct RAG, RAG, and No RAG](page_120_120_480_384.png)

Figure 8: Performance on conflicting and consistent instances between No RAG and RAG.

![Bar chart showing accuracy (%) for Instruct RAG, Self-Route, and Astute RAG on Both Incorrect, Conflicting, Both Correct](page_670_120_480_384.png)
Figure 9: Accuracy improvement when increasing \( t \).

Impact of Source-Awareness. To evaluate the impact of source-awareness, we conducted an ablation study where source labels (own memory and external retrieval) were removed during the consolidation and answer generation process. The results are shown in Tab. 3. The comparison shows that providing source information (ASTUTE RAG) leads to better performance overall compared to omitting it (ASTUTE RAG (No Source)), particularly on NQ, PopQA, and BioASQ, suggesting that awareness of information origin aids the consolidation process.

Accuracy of intermediate steps. To investigate the performance of intermediate steps, including knowledge consolidation and confidence assignment, we use LLM-as-a-judge with the instruction in Appx. A. Our experimental results show that the accuracy for knowledge consolidation is 98.2%, and for confidence assignment, it is 95.0%. These results demonstrate the effectiveness of the proposed framework in the intermediate stages.

Qualitative examples. In Fig. 12, we present two representative examples showing the intermediate outputs of ASTUTE RAG. In the first example, LLM without RAG generates a wrong answer, while RAG returns a correct answer. ASTUTE RAG successfully identified the incorrect information in its generated passage and an external passage, avoiding confirmation bias (Tan et al., 2024). In the second example, LLM is correct but RAG is incorrect due to imperfect retrieval. ASTUTE RAG detected the correct answer from imperfect context leveraging internal knowledge.

6 Conclusion

We investigate the impact of imperfect retrieval on the performance of RAG systems and identify knowledge conflicts as a key challenge. To address this, we introduce ASTUTE RAG, a novel approach that leverages the internal knowledge of LLMs and iteratively refines the generated responses by consolidating internal and external knowledge in a source way. We demonstrate the effectiveness of ASTUTE RAG in mitigating the negative effects of imperfect retrieval and improving the robustness of RAG, particularly in challenging scenarios with unreliable external sources.

Acknowledgement

We would like to thank Jinsung Yoon for valuable discussions and insights that helped to improve this paper. We would also like to thank all other colleagues from Google Cloud AI Research for their valuable feedback.

Limitations

ASTUTE RAG’s effectiveness hinges on the capabilities of advanced LLMs with strong instruction-following and reasoning abilities, hence potentially more limited applicability with less sophisticated LLMs. As an important future direction, extending the experimental setup to include longer inputs would be important, where the challenges of imperfect retrieval and knowledge conflicts may be even more pronounced.

References

Marco Alessio, Guglielmo Faggioli, Nicola Ferro, Franco Maria Nardini, Raffaele Perego, et al. 2024. Improving rag systems via sentence clustering and reordering. In RAG@ SIGIR 2024 workshop: The Information Retrieval’s Role in RAG Systems, ACM, pages 1–10.

Akari Asai, Zequi Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.

Figure 10: Efficiency in terms of tokens consumed.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Milligan, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206–2240. PMLR.

Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024a. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17754–17762.

Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Ke-fan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. 2024b. Universal self-consistency for large language models. In ICML 2024 Workshop on In-Context Learning.

Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The power of noise: Redefining retrieval for rag systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 719–729.

Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. 2024. Unifying bias and unfairness in information retrieval: A survey of challenges and opportunities with large language models. arXiv preprint arXiv:2404.11457.

Feiting Fang, Yuelin Bai, Shiwen Ni, Min Yang, Xiaojun Chen, and Ruifeng Xu. 2024. Enhancing noise robustness of retrieval-augmented language models with adaptive adversarial training. arXiv preprint arXiv:2405.20978.

Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengcheng Cai, and Alfio Gliozzo. 2022. Re2g: Retrieve, rerank, generate. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2701–2715.

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR.

Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C Park. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7029–7043.

Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992.

Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jieixin Xu, Li Qiuxia, and Jun Zhao. 2024. Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16867–16878.

Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303–5315.

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9802–9822.

Ruoqiong Pan, Boxi Cao, Hongyu Lin, Xianpei Han, Jia Zheng, Sirui Wang, Xunliang Cai, and Le Sun. 2024. Not all contexts are equal: Teaching llms credibility-aware generation. arXiv preprint arXiv:2404.06809.
Quang Hieu Pham, Hoang Ngo, Anh Tuan Luu, and Dat Quoc Nguyen. 2024. Who’s who: Large language models meet knowledge conflicts in practice. arXiv preprint arXiv:2410.15737.

Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, and Pang Wei Koh. 2024. Scaling retrieval-based language models with a trillion-token datastore. arXiv preprint arXiv:2407.12854.

Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. Asqa: Factoid questions long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8273–8288.

Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muenmighoff, Han-yu Wang, Haisu Liu, Quan Shi, Zachary S Siegel, Michael Tang, et al. 2024. Bright: A realistic and challenging benchmark for reasoning-intensive retrieval. arXiv preprint arXiv:2407.12883.

Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang, Qi Cao, and Xueqi Cheng. 2024. Blinded by generated contexts: How language models merge generated and retrieved contexts for open-domain qa? arXiv preprint arXiv:2401.11911.

Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2024. Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).

George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics, 16:1–28.

Haotian Wang, Xiyuan Du, Weijiang Yu, Qianglong Chen, Kun Zhu, Zheng Chu, Lian Yan, and Yi Guan. 2023a. Apollo’s oracle: Retrieval-augmented reasoning in multi-agent debates. arXiv preprint arXiv:2312.04854.

Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, and Graham Neubig. 2023b. Learning to filter context for retrieval-augmented generation. arXiv preprint arXiv:2311.08377.

Zilong Wang, Zifeng Wang, Long Le, Huaxiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al. 2024. Speculative rag: Enhancing retrieval augmented generation through drafting. arXiv preprint arXiv:2407.08223.

Zhepei Wei, Wei-Lin Chen, and Yu Meng. 2024. Instructrag: Instructing retrieval-augmented generation with explicit denoising. arXiv preprint arXiv:2406.13629.

Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. 2024. Certifiably robust rag against retrieval corruption. arXiv preprint arXiv:2405.15556.

Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2024. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In The Twelfth International Conference on Learning Representations.

Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. Recomp: Improving retrieval-augmented lms with compression and selective augmentation. arXiv preprint arXiv:2310.04408.

Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Retrieval meets long context large language models. In The Twelfth International Conference on Learning Representations.

Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884.

Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, et al. 2024. Crag–comprehensive rag benchmark. arXiv preprint arXiv:2406.04744.

Chanwoong Yoon, Taewho Lee, Hyeon Hwang, Min-byul Jeong, and Jaewoo Kang. 2024. Compact: Compressing retrieved documents actively for question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21424–21439.

Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. Making retrieval-augmented language models robust to irrelevant context. In The Twelfth International Conference on Learning Representations.

Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023a. Generate rather than retrieve: Large language models are strong context generators. In The Eleventh International Conference on Learning Representations.

Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu. 2023b. Chain-of-note: Enhancing robustness in retrieval-augmented language models. arXiv preprint arXiv:2311.09210.

Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. arXiv preprint arXiv:2407.02485.
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, and Lu Wang. 2023. Merging generated and retrieved knowledge for open-domain qa. In The 2023 Conference on Empirical Methods in Natural Language Processing.

Zheng Zhao, Emilio Monti, Jens Lehmann, and Haytham Assem. 2024. Enhancing contextual understanding in large language models through contrastive decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4225–4237.

Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. 2024. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models. arXiv preprint arXiv:2402.07867.
<table>
  <tr>
    <th></th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
    <th>\( m \)</th>
  </tr>
  <tr>
    <td>\( \hat{m}=1 \)</td>
    <td>52.20</td>
    <td>84.10</td>
    <td>60.14</td>
    <td>44.38</td>
    <td>61.71</td>
    <td>0.69</td>
  </tr>
  <tr>
    <td>\( \hat{m}=2 \)</td>
    <td>52.20</td>
    <td>85.16</td>
    <td>60.84</td>
    <td>43.26</td>
    <td>62.00</td>
    <td>1.24</td>
  </tr>
</table>

Table 2: Performance and average number of generated passages using different \( \hat{m} \).

![Scatter plot showing overall accuracy vs. average API calls for different models](page_563_670_482_246.png)

Figure 11: Efficiency in terms of API calls.

<table>
  <tr>
    <th>Model</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td><b>50.2</b></td>
    <td><b>81.6</b></td>
    <td><b>58.0</b></td>
    <td><b>40.5</b></td>
    <td><b>59.2</b></td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG (No Source)</b></td>
    <td>48.1</td>
    <td>82.3</td>
    <td>57.7</td>
    <td>39.9</td>
    <td>58.6</td>
  </tr>
</table>

Table 3: Ablation study on source-awareness.
LLM: Zachary Levi
RAG: Jason Lee

Question
Who plays David in Alvin and the Chipmunks?

Generate Passage
The character Dave Seville in all of the Alvin and the Chipmunks movies, both live-action and animated, is played by Zachary Levi.

Consolidate Information
Group A (External Retrieval):
Documents 2, 6, 9, 11
Jason Lee plays David Seville in the live-action Alvin and the Chipmunks movies ...

Group B (Own Memory/External Retrieval):
Documents 1, 7
Zachary Levi plays Toby Seville (who becomes David Seville) in the live-action movies ...

Group C (External Retrieval):
Documents 3, 4, 5, 8, 10
... These are irrelevant to the question of who plays David Seville.

Propose Answer and Assign Confidence
Answer A (from Group A):
Jason Lee.
**Confidence: High (95%).**
Multiple sources confirm this information.

Answer B (from Group B):
Zachary Levi.
**Confidence: Low (25%).**
My own memory conflicts with external sources, and this group also introduces a different character (Ian) played by David Cross, creating confusion.

LLM: Jack Nicholson
RAG: No Answer

Question
Who won the Oscar for best actor when Titanic was nominated?

Generate Passage
Titanic was nominated for ... Jack Nicholson won the Best Actor Oscar at the 70th Academy Awards in 1998 for his role in As Good as It Gets.

Consolidate Information
Group 1 (Memory & External):
Documents: 1, 6
Jack Nicholson won the Best Actor Oscar ...

Group 2 (External):
Documents: 2, 7
Walt Disney holds the record for most Oscar ...

Group 3 (External):
Documents: 3, 5, 6
Titanic (1997) received 14 Oscar nominations ...

Group 4 (External):
Documents: 4
Leonardo DiCaprio was not nominated ...

Group 5 (External):
Documents: 8, 9, 10, 11
... irrelevant to the question.

Propose Answer and Assign Confidence
Answer from Group 1:
Jack Nicholson.
**Confidence: High (100%)**
Consistent information from memory and a reliable external source.

Groups 2, 3, 4, 5:
Irrelevant to the question.

Figure 12: Qualitative examples. Top: ASTUTE RAG identified the error in internal knowledge (i.e., generated passage) by confirming with external sources. Bottom: ASTUTE RAG detected the correct answer from imperfect retrieval by checking with its internal knowledge. Standard RAG does not provide an answer because the retrieved passages are too noisy.
A  Prompt Template for ASTUTE RAG

Prompt for Adaptive Passage Generation (\( p_{gen} \))

Generate a document that provides accurate and relevant information to answer the given question. If the information is unclear or uncertain, explicitly state 'I don’t know' to avoid any hallucinations.

Question: {question} Document:

Prompt for Iterative Knowledge Consolidation (\( p_{con} \))

Task: Consolidate information from both your own memorized documents and externally retrieved documents in response to the given question.

* For documents that provide consistent information, cluster them together and summarize the key details into a single, concise document.
* For documents with conflicting information, separate them into distinct documents, ensuring each captures the unique perspective or data.
* Exclude any information irrelevant to the query.
For each new document created, clearly indicate:
* Whether the source was from memory or an external retrieval.
* The original document numbers for transparency.

Initial Context: {context}
Last Context: {context}
Question: {question}
New Context:
Prompt for Knowledge Consolidation and Answer Finalization (\( p_{ans} \))

Task: Answer a given question using the consolidated information from both your own memorized documents and externally retrieved documents.

Step 1: Consolidate information
* For documents that provide consistent information, cluster them together and summarize the key details into a single, concise document.
* For documents with conflicting information, separate them into distinct documents, ensuring each captures the unique perspective or data.
* Exclude any information irrelevant to the query.
For each new document created, clearly indicate:
* Whether the source was from memory or an external retrieval.
* The original document numbers for transparency.

Step 2: Propose Answers and Assign Confidence
For each group of documents, propose a possible answer and assign a confidence score based on the credibility and agreement of the information.

Step 3: Select the Final Answer
After evaluating all groups, select the most accurate and well-supported answer.
Highlight your exact answer within <ANSWER> your answer </ANSWER>.

Initial Context: {context_init}
[Consolidated Context: {context}] # optional
Question: {question}
Answer:

Prompt for Intermediate Step Evaluation

**Task:** You are provided with the following:
1. A question.
2. The correct answer.
3. The input context.
4. The model’s response, which contains:
- Consolidated context.
- Confidence scores for candidate answers.
Your task is to:
- Evaluate the **quality of the consolidated context** in the model’s response and provide a label: '<consolidation> correct </consolidation>' or '<consolidation> incorrect </consolidation>'.
This evaluation is only about whether the consolidation is correct given the input context.
- Evaluate the **accuracy of the confidence score** (whether it aligns with the confidence of the supporting context) and provide a label: '<confidence> correct </confidence>' or '<confidence> incorrect </confidence>'. The evaluation is only based on the consolidated context.
Note that correct consolidation and confidence do not necessarily indicate the correct answer.
Question: {query}
Correct Answer: {answer}
Input Context: {input}
Model Response: {response}
Evaluation:
B Data Collection

Encompassing a diverse range of natural questions, our benchmark consists of realistic retrieval results with Google Search8 as the retriever and the Web as the corpus. Notably, we do not select questions or annotate answers based on the retrieval results. This setting allows us to analyze the severity of imperfect retrieval in real-world RAG. It distinguishes our benchmark from previous ones that employ synthetic retrieval corruptions or that unintentionally reduce the frequency of imperfect retrieval with biased construction protocols (Chen et al., 2024a; Yang et al., 2024). Overall, our benchmark contains 1,042 short-form question-answer pairs, each paired with 10 retrieved passages. When collecting the passages, we retrieve the top 30 results and select the first 10 accessible websites. From each retrieved website, we extract the paragraph corresponding to the snippet provided in the search results as the retrieved passage. Retrieved results might contain natural noise with irrelevant or misleading information. We do not consider enhancements to the retrieval side, such as query rewriting, as such enhancements are typically already incorporated into commercial information retrieval systems. All of these datasets are short-form QA. Following previous work (Xiang et al., 2024; Wei et al., 2024; Mallen et al., 2023), a model response is considered correct if it contains the ground-truth answer. To enhance evaluation reliability, we prompt LLMs to enclose the exact answer within special tokens, extracting them as the final responses.

Question-answer pairs. We consider question-answer pairs from four datasets of different properties spanning across general questions, domain-specific questions, and long-tail questions. NQ (Kwiatkowski et al., 2019) and TriviaQA (Joshi et al., 2017) are two widely-studied question-answering (QA) datasets in general domains. BioASQ (Tsatsaronis et al., 2015) is from biomedical domain that has demonstrated significant benefits from RAG when general-purpose LLMs are considered. PopQA (Mallen et al., 2023) focuses on long-tail knowledge and has been shown to be challenging for even advanced LLMs to solve without external knowledge. All these datasets contain questions with short-form answers and most of them list all valid answer variants. This format can support automatic verification of answer appearance in retrieved passages and model responses, leading to more precise evaluations.

Retrieval process. For each question in our benchmark, we query Google Search to retrieve the top 30 results and select the first 10 accessible websites. From each retrieved website, we extract the paragraph corresponding to the snippet provided in Google Search results as the retrieved passage. We do not consider enhancements to the retrieval side, such as query rewriting, as such enhancements are typically already incorporated into commercial information retrieval systems.

C Comparison with Answer Refinement

In Tab. 4, we further compare ASTUTE RAG with Answer Refinement, where the LLM is prompted to refine its initial answer by reconsidering external context. Notably, this baseline performs nearly identically to Self-Route, which is expected, because both approaches rely on the model itself to determine whether to revise its initial answer based on external knowledge. The comparison underscores that simply enabling LLMs to self-correct does not yield significant improvements beyond existing routing strategies. In contrast, ASTUTE RAG continues to outperform all baselines across datasets, reinforcing the benefit of explicitly structured consolidation mechanisms.

D Comparison with Context Filtering

To isolate the effect of identifying irrelevant information separate from consolidation, we conducted an ablation study introducing two Context Filtering baselines. The first baseline filters only the retrieved documents, while the second filters both retrieved and generated documents prior to answer generation, both without applying the consolidation step. As shown in Tab. 5, while context filtering improves performance over the basic RAG baseline, it falls short of the performance achieved by ASTUTE RAG. This supports our hypothesis that consolidating diverse information, including consistent, conflicting, and relevant content, rather than merely filtering out irrelevant parts, is critical to the performance gains observed with ASTUTE RAG.

E Comparison with Context Compression

Context compression (Wang et al., 2023b; Yoon et al., 2024) is also a related direction. We further
<table>
  <tr>
    <th>Model</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>Self-Route</td>
    <td>47.5</td>
    <td>79.9</td>
    <td>58.0</td>
    <td>38.2</td>
    <td>57.6</td>
  </tr>
  <tr>
    <td>Answer Refinement</td>
    <td>47.1</td>
    <td>79.9</td>
    <td>58.0</td>
    <td>38.2</td>
    <td>57.5</td>
  </tr>
  <tr>
    <td><b>Astute RAG</b></td>
    <td>50.2</td>
    <td>81.6</td>
    <td>58.0</td>
    <td>40.5</td>
    <td>59.2</td>
  </tr>
</table>

Table 4: Comparison of Astute RAG with routing and refinement-based baselines on Gemini.

<table>
  <tr>
    <th>Model</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>RAG</td>
    <td>42.7</td>
    <td>76.0</td>
    <td>55.2</td>
    <td>33.7</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td>Context Filtering (retrieved)</td>
    <td>43.7</td>
    <td>77.0</td>
    <td>57.0</td>
    <td>34.3</td>
    <td>54.8</td>
  </tr>
  <tr>
    <td>Context Filtering (generated and retrieved)</td>
    <td>49.2</td>
    <td>79.2</td>
    <td>56.6</td>
    <td>40.4</td>
    <td>57.9</td>
  </tr>
  <tr>
    <td><b>Astute RAG</b></td>
    <td>50.2</td>
    <td>81.6</td>
    <td>58.0</td>
    <td>40.5</td>
    <td>59.2</td>
  </tr>
</table>

Table 5: Comparison with context filtering baselines.

conduct experiments comparing our method with CompAct (Yoon et al., 2024). The results in Tab. 6 and Tab. 7 show that context compression is ineffective in handling the challenges of imperfect context and knowledge conflicts. Notably, it even performs worse than the No RAG and RAG baselines in this context. The primary reason for this underperformance lies in the limitations of context compression. It struggles to effectively identify incorrect information when there are conflicts in context and often removes or reduces the appearance of helpful information in the process. This reinforces the importance of our approach, which does not rely solely on compression but instead integrates both internal and external knowledge while handling conflicts in a more nuanced manner.

F Influence of passage ordering.

We apply different ordering strategies (Alessio et al., 2024) on RAG and ASTUTE RAG. As shown in Tab. 8, we find that the improvement with ASTUTE RAG is significantly larger than the gap between different ordering strategies. Moreover, the consolidation process makes ASTUTE RAG less sensitive to it.
<table>
  <tr>
    <th>Method</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>47.1</td>
    <td>82.0</td>
    <td>50.4</td>
    <td>29.8</td>
    <td>54.5</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>44.4</td>
    <td>76.7</td>
    <td>58.0</td>
    <td>36.0</td>
    <td>55.5</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>38.6</td>
    <td>68.9</td>
    <td>49.3</td>
    <td>30.3</td>
    <td>48.4</td>
  </tr>
  <tr>
    <td>ASTUTE RAG</td>
    <td>52.2</td>
    <td>84.1</td>
    <td>60.1</td>
    <td>44.4</td>
    <td>61.7</td>
  </tr>
</table>

Table 6: Comparison with context compression on Claude.

<table>
  <tr>
    <th>Method</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>44.8</td>
    <td>80.2</td>
    <td>45.8</td>
    <td>25.3</td>
    <td>51.3</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>42.7</td>
    <td>76.0</td>
    <td>55.2</td>
    <td>33.7</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>35.3</td>
    <td>65.0</td>
    <td>47.6</td>
    <td>30.9</td>
    <td>46.0</td>
  </tr>
  <tr>
    <td>ASTUTE RAG</td>
    <td>50.2</td>
    <td>81.6</td>
    <td>58.0</td>
    <td>40.5</td>
    <td>59.2</td>
  </tr>
</table>

Table 7: Comparison with context compression on Gemini.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th rowspan="2">Ordering Strategy</th>
    <th colspan="5">NQ TriviaQA BioASQ PopQA Overall</th>
  </tr>
  <tr>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td rowspan="5">RAG</td>
    <td>Random</td>
    <td>43.39</td>
    <td>76.33</td>
    <td>56.99</td>
    <td>34.83</td>
    <td>54.61</td>
  </tr>
  <tr>
    <td>Ascending</td>
    <td>43.05</td>
    <td>75.62</td>
    <td>57.69</td>
    <td>34.83</td>
    <td>54.51</td>
  </tr>
  <tr>
    <td>Descending</td>
    <td>44.41</td>
    <td>76.68</td>
    <td>58.04</td>
    <td>35.96</td>
    <td>55.47</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Top-to-bottom</td>
    <td>44.75</td>
    <td>77.39</td>
    <td>57.69</td>
    <td>35.96</td>
    <td>55.66</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Bottom-to-top</td>
    <td>44.41</td>
    <td>75.62</td>
    <td>58.04</td>
    <td>35.96</td>
    <td>55.18</td>
  </tr>
  <tr>
    <td rowspan="5">AstuteRAG</td>
    <td>Random</td>
    <td>51.86</td>
    <td>84.81</td>
    <td>61.19</td>
    <td>41.57</td>
    <td>61.61</td>
  </tr>
  <tr>
    <td>Ascending</td>
    <td>51.86</td>
    <td>85.51</td>
    <td>59.79</td>
    <td>42.13</td>
    <td>61.52</td>
  </tr>
  <tr>
    <td>Descending</td>
    <td>52.20</td>
    <td>84.10</td>
    <td>60.14</td>
    <td>44.38</td>
    <td>61.71</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Top-to-bottom</td>
    <td>52.20</td>
    <td>84.45</td>
    <td>59.09</td>
    <td>43.82</td>
    <td>61.42</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Bottom-to-top</td>
    <td>51.19</td>
    <td>85.16</td>
    <td>61.54</td>
    <td>43.82</td>
    <td>62.00</td>
  </tr>
</table>

Table 8: Performance by Ordering Strategies.

Reviewer paper summaries: The paper introduces Astute RAG, a novel approach to enhance RAG by resolving knowledge conflicts between LLMs’ internal knowledge and external sources. Astute RAG integrates adaptive internal knowledge generation, iterative source-aware consolidation, and reliability-based answer finalization to explicitly compare and reconcile conflicting information, thereby recovering from RAG failures. Experiments across diverse LLMs and datasets demonstrate its superior robustness over baseline methods, particularly in worst-case scenarios with unreliable retrievals.

This work introduces ASTUTE RAG, a novel RAG framework designed to tackle the challenges of imperfect retrieval and knowledge conflicts between LLMs' internal knowledge and external sources. ASTUTE RAG follows a three-step process: it first adaptively generates internal knowledge from the LLM, then iteratively consolidates internal and external knowledge with source awareness, and finalizing answers based on reliability assessments. In addition, this work provides quantitative analyses and novel insights for the connection among imperfect retrieval, knowledge conflicts, and RAG failures under real-world conditions. Experiments conducted across diverse datasets and LLMs show that ASTUTE RAG outperforms existing robustness-focused RAG methods. It also effectively mitigates knowledge conflicts, achieving performance comparable to or even exceeding that of retrieval-free LLMs in such scenarios.
Summary of strengths: - The paper empirically demonstrates that imperfect retrieval is inevitable in real-world settings and identifies knowledge conflicts between LLMs’ parametric knowledge and external sources as a critical bottleneck.
- The paper proposes a simple, training-free post-retrieval framework that systematically compares, regroups, and consolidates internal and external knowledge through source-aware iterative refinement, significantly improving accuracy under noisy retrievals.
- Comprehensive evaluations leverage realistic retrieval settings (Google Search as the retriever) and diverse datasets (NQ, TriviaQA, BioASQ), ensuring practical applicability to real-world RAG deployments.

1. This work provides quantitative analyses and novel insights for the connection among imperfect retrieval, knowledge conflicts, and RAG failures under real-world conditions.

2. This work proposes a systematic, training-free framework that explicitly integrates LLMs’ internal knowledge with external sources, resolving conflicts through iterative consolidation and source-aware reliability assessments.

3. Extensive experiments have been conducted to validate the reliability of ASTUTE RAG.
Summary of weaknesses: - The method’s effectiveness heavily relies on the LLM’s ability to accurately elicit internal knowledge and discern nuanced differences between sources. This is evident in the performance gap between smaller models (e.g., Mistral-Nemo) and larger ones (e.g., Mistral-Large).
- While iterative consolidation is central to the framework, increasing the number of iterations does not consistently improve results. Providing qualitative examples of intermediate consolidation steps could clarify how conflicting knowledge evolves during refinement.
- Figure 8 highlights Astute RAG’s ability to outperform both RAG and non-RAG baselines even when neither internal nor external knowledge is fully correct. A deeper analysis of how the framework synthesizes partial or indirect signals in such scenarios would strengthen the discussion.

1. Only a single retriever (Google Search) was used in the experiments. Why not include more retrievers, e.g., Bing, etc.? We would like to see more comparisons.

2. The experiments are limited to the 10 documents retrieved by Google Search. I believe it would be more informative to conduct comparisons using a larger number of retrieved documents (e.g., 20 or 50), especially in light of the fact that current LLMs support substantially larger input lengths.

3. The experiments lack analysis of certain scenarios — for instance, when both the internal knowledge and the retrieved external knowledge are factually correct, but appear to conflict due to temporal differences. Since an LLM’s internal knowledge only reflects information up to a certain cutoff date, such discrepancies might be mistakenly treated as conflicts. In such cases, during the answer generation stage, does the LLM assign higher confidence to the more recent external knowledge or to its own internal knowledge?
Comments, suggestions and typos: See weakness

1. In the *Prompt for Iterative Knowledge Consolidation* ($p_{con}$), when internal and external knowledge are merged into a new document, how is the source of this consolidated document determined? Is it considered internal or external?

2. Regarding *source-awareness* — based on the prompts, it seems that source-awareness plays a role primarily during the final answer generation, rather than during the consolidation step. Is it truly effective? What would happen if source information were omitted entirely throughout the process? It would be helpful to see an ablation study on this.

3. How does the model handle documents that are neither conflicting nor consistent? Taking Figure 4 as an example, why are documents that neither conflict with nor align with the others combined to produce Answer A?
Overall assessment scores: 4

3.5
Metareview score: 3
Metareview: This paper proposes a new (chain of) prompting technique to improve RAG performance. In particular, they prompt an LLM to generate its internal knowledge about the query as a passage. Then, they prompt LLM to group retrieved documents plus the model-generated passage based on their knowledge similarity. Eventually, they prompt the LLM to generate an answer from each group and pick one final answer. 

They show that this prompting technique improves the RAG performance on 6 RAG QA datasets for 4 LLMs.
Summary of reasons to publish: 1. How to deal with noisy retrieved content (especially dealing with the potential knowledge conflicts among retrieved content and between the model and retrieved content) is an important and meaningful research question.

2. The proposed method is intuitive and easy to implement. 

3. The paper includes a lot of analyses from different perspectives.