Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
Authors: Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, Sercan O Arik
Accepted at: acl25

ASTUTE RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

Fei Wang12*   Xingchen Wan1   Ruoxi Sun1   Jiefeng Chen1   Sercan Ö. Arik1
1Google Cloud AI Research   2University of Southern California
fwang598@usc.edu   {xingchenw,ruoxis,jiefengc,soarik}@google.com

Abstract
Retrieval-augmented generation (RAG), while effective in integrating external knowledge to enhance large language models (LLMs), can be undermined by imperfect retrieval, which may introduce irrelevant, misleading, or even malicious information. Despite its importance, previous studies have rarely explored the behavior of RAG with errors from imperfect retrieval, and how potential conflicts arise between the LLMs’ internal knowledge and external sources. We show that imperfect retrieval augmentation might be inevitable and quite harmful, through controlled analysis under realistic conditions. Knowledge conflicts between LLM-internal and external knowledge from retrieval is a bottleneck to overcome in the post-retrieval stage of RAG. To render LLMs resilient to imperfect retrieval, we propose ASTUTE RAG, a novel RAG approach that adaptively elicits essential information from LLMs’ internal knowledge, iteratively consolidates internal and external knowledge with source-awareness, and finalizes the answer according to information reliability. Our experiments with Gemini and Claude demonstrate that ASTUTE RAG significantly outperforms previous robustness-enhanced RAG methods. Notably, ASTUTE RAG is the only approach that matches or exceeds the performance of LLMs without RAG under worst-case scenarios. ASTUTE RAG effectively resolves knowledge conflicts, improving the reliability and trustworthiness of RAG systems.

1 Introduction
Retrieval-augmented generation (RAG) is commonly used for large language models (LLMs) to tackle knowledge-intensive tasks (Guu et al., 2020; Lewis et al., 2020). Prior works mainly leverage RAG to address the inherent knowledge limitations of LLMs, effectively integrating missing information and grounding to reliable sources.

However, recent research has highlighted a significant drawback that RAG might rely on imperfect retrieval, including irrelevant, misleading, or even malicious information (Fig. 1), which eventually leads to inaccurate LLM responses (Chen et al., 2024a; Zou et al., 2024). Moreover, recent studies have shown that retrieval augmentation can confuse LLMs when retrieved passages are conflicting with LLMs’ parametric knowledge (Tan et al., 2024; Xie et al., 2024; Jin et al., 2024). These pose significant challenges to the trustworthiness of RAG.

To address imperfect retrieval, earlier work seeks to improve the retrieval approaches, such as dynamic and iterative retrieval (Jiang et al., 2023; Asai et al., 2023; Yan et al., 2024). However, the occurrence of imperfect retrieval is still inevitable, due to corpus quality limitations (Shao et al., 2024), the reliability of retrievers (Dai et al., 2024), and the complexity of queries (Su et al., 2024). Consequently, recent work shifts the focus to the generation stage, seeking to reduce the negative impact of noisy retrieved passages (Xiang et al., 2024; Wei et al., 2024). Another line of research at generation stage, motivated by knowledge conflicts, has explored complementing retrieved passages with LLM-generated passages (Yu et al., 2023a; Zhang et al., 2023) or deactivating RAG when the retrieved passages are of insufficient quality (Xu et al., 2024; Mullen et al., 2023; Jeong et al., 2024).

Despite the previous work on the impact of imperfect retrieval and knowledge conflicts at RAG generation stage, quantitative analyses lack on two crucial real-world aspects: (i) the relation between retrieval quality and occurrence of knowledge conflicts, and (ii) the extent to which retrieved passages and LLMs’ parametric knowledge can correct each other. Method-wise, existing approaches for mitigating RAG failures caused by imperfect retrieval and knowledge conflicts have not yet yielded a training-free method capable of explicitly analyzing conflicting knowledge across various internal
Figure 1: Knowledge conflicts between the LLMs’ internal knowledge and retrieved knowledge from external sources. We report the overall results with Claude under the setting in Sec. 5.1.

and external sources, and achieving worst-case robustness for black-box LLMs.

In this paper, we first conduct comprehensive analyses to investigate the relation between imperfect retrieval and knowledge conflicts, and examine the frequency of external and LLMs’ internal knowledge mutually correcting each other (Sec. 3). On a diverse range of general, domain-specific, and long-tail questions from NQ (Kwiatkowski et al., 2019), TriviaQA (Joshi et al., 2017), BioASQ (Tsatsaronis et al., 2015), and PopQA (Mallen et al., 2023), we observe that imperfect retrieval is widespread even with an adept real-world search engine, leading to the impeded performance of RAG.\footnote{such as Google Search with Web as corpus} Retrieval precision is tightly correlated with the knowledge conflict rate. Mutual correction between the LLM’s knowledge and external knowledge is crucial for recovering from RAG failures. Our findings underscore the potential severity of imperfect retrieval in real-world RAG and highlight the widespread existence of knowledge conflicts as the bottleneck.

We propose ASTUTE RAG, a novel RAG approach designed for resilience to imperfect retrieval augmentation, while preserving RAG grounding effect when retrieval is reliable (Sec. 4). ASTUTE RAG effectively differentiates between consistent and conflicting information from the LLM’s internal knowledge and the externally retrieved passages, assesses their reliability, and ensures proper integration of trustworthy information. ASTUTE RAG first adaptively elicits LLMs’ knowledge and then conducts source-aware knowledge consolidation. The desiderata is combining consistent information, identifying conflicting information, and filtering out irrelevant information. Finally, ASTUTE RAG proposes answers based on consistent information and compares them to determine the final answer. Our experiments with various LLMs (Claude, Gemini and Mistral), demonstrate superior performance of ASTUTE RAG compared to previous RAG approaches designed for robustness (Sec. 5). ASTUTE RAG consistently outperforms baselines across different retrieval quality levels. Notably, ASTUTE RAG is the only RAG method that achieves performance comparable to or even surpassing retrieval-free mode of LLMs under the worst-case scenario where all retrieved passages are unhelpful. Further analysis reveals the effectiveness of ASTUTE RAG in resolving knowledge conflicts.

In summary, our core contributions are threefold. First, we provide quantitative analyses and novel insights for the connection among imperfect retrieval, knowledge conflicts, and RAG failures under real-world conditions. Second, we propose ASTUTE RAG, which explicitly analyzes LLM-internal and external knowledge in-context, assesses their reliability, and recovers from RAG failures with black-box access. Third, with experiments with various LLMs and datasets, we demonstrate the effectiveness of ASTUTE RAG in improving robustness and trustworthiness, even in the most challenging scenarios.

2 Related Work

RAG aims to address the inherent knowledge limitation of LLMs with passages retrieved from external sources of information such as private corpora or public knowledge bases (Guu et al., 2020; Lewis et al., 2020; Borgeaud et al., 2022). Given the
Figure 2: Imperfect retrieval (samples with low retrieval precision) is prevalent in real-world RAG.

Figure 3: Conflicting rate between answers from LLMs with and without RAG on different retrieval precision.

widespread real-world adoption of RAG, including risk-sensitive domains, the negative impact of noisy information within retrieved passages has garnered increasing attention (Cuconasu et al., 2024). Recent work explored enhancing the robustness of RAG systems against noise from various perspectives, including training LLMs with noisy context (Yu et al., 2023b; Yoran et al., 2024; Pan et al., 2024; Fang et al., 2024), training small models to filter out irrelevant passages (Wang et al., 2023b; Xu et al., 2023), passage reranking (Yu et al., 2024; Glass et al., 2022), dynamic and iterative retrieval (Jiang et al., 2023; Asai et al., 2023; Yan et al., 2024), query rewriting (Ma et al., 2023), and speculative drafting (Wang et al., 2024). These focus on distinct modules or stages of RAG systems and are orthogonal to our work.

Our work focuses on enhancing RAG robustness at the post-retrieval stage, after retrieved passages have been provided. On this, RobustRAG (Xiang et al., 2024) aggregates answers from each independent passage to provide certifiable robustness. InstructRAG (Wei et al., 2024) instructs the LLM to provide a rationale connecting the answer with information in passages. MADRA (Wang et al., 2023a) applies multi-agent debate to select helpful evidence. However, these do not explicitly incorporate internal knowledge to recover from RAG failures and therefore might severely suffer when the majority of retrieved passages have issues. For emphasizing internal knowledge of LLMs in RAG, recent work explored using LLM-generated passage as context (Yu et al., 2023a), training models to match generated and retrieved passages (Zhang et al., 2023), adaptively switching between LLMs with and without RAG (Xu et al., 2024; Mailen et al., 2023; Jeong et al., 2024), and combining answers through contrastive decoding (Zhao et al., 2024; Jin et al., 2024). Different from prior work, we provide a systematic framework on connecting imperfect retrieval, knowledge conflicts, and RAG failures. Specifically focusing on the imperfect context setting, our method is training-free and applicable to black-box LLMs, explicitly analyzes internal and external knowledge in-context, and offers broader usability and adaptability.

3 The Pitfall of RAG

To better showcase common real-world challenges and motivate improved methodological designs, we evaluate retrieval quality, the occurrence of knowledge conflicts, their relationship, and the mutual correction between external and internal knowledge using a controlled dataset derived from NQ, TriviaQA, BioASQ, and PopQA, datasets widely used for RAG in prior work (Xiang et al., 2024; Wei et al., 2024; Asai et al., 2023). Different from prior work, our analysis is based on real-world retrieval results with Google Search2 as the retriever and the Web as the corpus. Overall, we sample 1K instances, each with 10 retrieved passages.

Imperfect retrieval and knowledge conflicts are common and harmful. Our initial observations

2https://developers.google.com/custom-search/v1/overview
Figure 4: Overview of the ASTUTE RAG framework. ASTUTE RAG is designed to better combine the information from the external sources (e.g. web, domain-specific corpora) and internal knowledge of the LLMs by employing a consolidation mechanism to address the conflicts, which eventually leads to superior generation quality.

are consist with prior work. As shown in Fig. 2, the retrieval precision3 is generally low - roughly 70% retrieved passages do not directly contain true answers, consistent with prior work demonstrating the often imperfect nature of retrieval results (Thakur et al., 2024; Su et al., 2024). With Claude 3.5 Sonnet as the LLM, Fig. 1 shows that 19.2% of the overall data exhibit knowledge conflicts, consistent with prior work demonstrating the prevalence of such conflicts across various scenarios (Pham et al., 2024; Xie et al., 2024; Longpre et al., 2021). Moreover, we observe strong correlations between retrieval precision and RAG performance (Fig. 7) and between the occurrence of knowledge conflicts and RAG performance (Fig. 8), findings consistent with prior work on these respective topics (Chen et al., 2024a; Xie et al., 2024).

Lower retrieval precision increases knowledge conflicts in general. As shown in Fig. 3, most advanced LLMs exhibit the highest conflict rates when retrieval precision is as low as 10%. Subsequently, the conflict rate generally decreases as precision increases, although some fluctuations may occur. This trend is generally applicable to the studied LLMs with different training processes. Notably, when retrieval precision is 0%, conflict rates tend to be significantly lower. This suggests that limited external knowledge for the query results in more irrelevant passages rather than incorrect ones.

Internal and external knowledge can correct each other to a comparable extent. Among the conflicting cases, the internal knowledge is correct on 47.4% of them, while the external knowledge is correct on the remaining 52.6%. These results emphasize the importance of effectively combining the internal and external knowledge to overcome the inherent limitation of relying solely on either source. However, previous work (Tan et al., 2024; Xie et al., 2024; Jin et al., 2024) shows that LLMs often select knowledge based on unreliable shortcuts, so simply presenting LLM-generated passages in the context may not help.

4 ASTUTE RAG

We first provide an overview of ASTUTE RAG (Sec. 4.1). Subsequently, we delve into the three major steps of ASTUTE RAG, including adaptive generation of internal knowledge (Sec. 4.2), source-aware knowledge consolidation (Sec. 4.3), and answer finalization (Sec. 4.4).

4.1 Overview

Our objective is to mitigate the effects of imperfect retrieval augmentation, resolve knowledge conflicts between the LLM’s internal knowledge and external sources (such as custom/public corpora and knowledge bases), and ultimately produce more accurate and reliable responses from LLMs. Given a set of retrieved passages from external sources \( E = [e_1, \ldots, e_n] \), a pre-trained LLM \( \mathcal{M} \) (accessible through prediction-only APIs, encompassing commercial black-box ones), and a query \( q \), the task is to generate the corresponding correct an-

3Ratio of passages directly contain true answers.
Algorithm 1 ASTUTE RAG

Require: Query \( q \), Retrieved Passages \( E = [e_1, \ldots, e_n] \), Large Language Model \( \mathcal{M} \), Number of Iteration \( t \), Max Number of Generated Passages \( \hat{m} \), Prompt Templates \( p_{gen}, p_{con}, p_{ans} \)
1: Adaptively generate passages: \( I \leftarrow \mathcal{M}(p_{gen}, q, \hat{m}) \) \( \triangleright \) Sec. 4.2
2: Combine internal and external passages: \( D_0 \leftarrow E \oplus I \)
3: Assign passage sources: \( S_0 \leftarrow [1_{\{d \in E\}} \text{for } d \text{ in } D_0] \)
4: if \( t > 1 \) then
    5:     for \( j = 1, \ldots, t - 1 \) do
        6:         Consolidate knowledge: \( \langle D_{j+1}, S_{j+1} \rangle \leftarrow \mathcal{M}(p_{con}, q, \langle D_0, S_0 \rangle, \langle D_j, S_j \rangle) \) \( \triangleright \) Sec. 4.3
    7:     end for
    8:     Finally consolidate and answer: \( a \leftarrow \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle, \langle D_{t-1}, S_{t-1} \rangle) \) \( \triangleright \) Sec. 4.4
9: else
10:     Consolidate knowledge and finalize the answer: \( a \leftarrow \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle) \)
11: end if
12: return \( a \)

swer \( a^* \). Notably, this setting is orthogonal to prior work on improving the retriever, training LLMs, or conducting adaptive retrieval, which are mainly preliminary steps.

ASTUTE RAG is designed to better leverage collective knowledge from both internal knowledge of LLMs and external corpus, for more reliable responses. As shown in Fig. 4 and Alg. 1, ASTUTE RAG starts from acquiring the most accurate, relevant, and thorough passage set from the LLMs’ internal knowledge. Then, internal and external knowledge are consolidated in an iterative way, by comparing the generated and retrieved passages. Finally, the reliability of conflicting information is compared and the final output is generated according to the most reliable knowledge.

4.2 Adaptive Generation of Internal Knowledge

In the first step, we elicit internal knowledge from LLMs. This LLM-internal knowledge, reflecting the consensus from extensive pre-training and instruction-tuning data, can supplement any missing information from the limited set of retrieved passages and enable mutual confirmation between LLM-internal and external knowledge. This is especially valuable when the majority of retrieved passages might be irrelevant or misleading. Specifically, we prompt LLMs to generate passages based on the given question \( q \), following Yu et al. (2023a). While Yu et al. (2023a) primarily focused on generating diverse internal passages, we emphasize the importance of reliability and trustworthiness of generated passages. To achieve this goal, we enhance the original method with constitutional principles and adaptive generation.

Inspired by Bai et al. (2022), we provide constitutional principles indicating the desired properties of internal passages in the prompt \( p_{gen} \) (see Appx. A for details) to guide their generation, emphasizing that the generated passages should be accurate, relevant, and hallucination-free. Moreover, we allow the LLM to perform adaptive generation of passages in its internal knowledge. The LLM can decide how many passages to generate by itself. Rather generating a fix number of passages, we request the LLM to generate at most \( \hat{m} \) passages, each covering distinct information, and to directly indicate if no more reliable information is available. This adaptive approach allows the LLM to generate fewer passages (or even no passages at all) when the useful information within internal knowledge is limited and more passages when there are multiple feasible answers in the internal knowledge. In this step, the LLM generates \( m \leq \hat{m} \) passages based on its internal knowledge:

\[
I = [i_1, \ldots, i_m] = \mathcal{M}(p_{gen}, q, \hat{m}).
\]

4.3 Iterative Source-aware Knowledge Consolidation

In the second step, we employ the LLM to explicitly consolidate information from both passages generated from its internal knowledge and passages retrieved from external sources. Initially, we combine passages from both internal and external knowledge sources \( D_0 = E \oplus I \).

We additionally ensure source-awareness by providing the source of each passage to LLMs when consolidating knowledge. The source information (internal or external, such as a website) is helpful in assessing the reliability of passages. Here, we provide the passage source as \( S_0 = [1_{\{d \in E\}} \text{for } d \text{ in } D_0] \). To consolidate knowledge, we prompt the LLM (with \( p_{con} \) in Appx. A) to identify consistent information across passages, detect conflicting information between each group
of consistent passages, and filter out irrelevant information. This step would regroup the unreliable knowledge in input passages into fewer refined passages. The regrouped passages also attribute their source to the corresponding input passages:

\[
\langle D_{j+1}, S_{j+1} \rangle = \mathcal{M}(p_{con}, q, \langle D_0, S_0 \rangle, \langle D_j, S_j \rangle).
\]

We find that this is especially helpful in comparing the reliability of conflicting knowledge and addressing knowledge conflicts. This knowledge consolidation process can run iteratively for \( t \) times to improve better utilization of the retrieved context.

4.4 Answer Finalization

In the last step, we prompt the LLM (with \( p_{ans} \) in Appx. A) to generate one answer based on each group of passages (\( \langle D_t, S_t \rangle \)), and then compare their reliability and select the most reliable one as the final answer. This comparison allows the LLM to comprehensively consider knowledge source, cross-source confirmation, frequency, and information thoroughness when making the final decision. Notably, this step can be merged into the last knowledge consolidation step to reduce the inference complexity (the amount of prediction API calls) using a combined prompt:

\[
a = \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle, \langle D_t, S_t \rangle).
\]

When \( t = 1 \), the initial passages will be input to the model directly for knowledge consolidation and subsequent answering: \( a = \mathcal{M}(p_{ans}, q, \langle D_0, S_0 \rangle) \).

5 Experiments

We evaluate the effectiveness of ASTUTE RAG on overcoming imperfect retrieval augmentation and addressing knowledge conflicts. In this section, we introduce the experiment setting (Sec. 5.1), compare the performance of ASTUTE RAG with various baselines on diverse datasets (Sec. 5.2), and provide in-depth analyses (Sec. 5.3).

5.1 Experimental Settings

Datasets and metrics. We consider datasets encompass general questions, domain-specific questions, long-tail questions, as well as both short-form and long-form formats, following prior work (Xiang et al., 2024; Wei et al., 2024). On NQ, TriviaQA, BioASQ, and PopQA, we provide 10 passages collected with Google Search from the Web for each instance. For long-form QA, we use ASQA (Stelmakh et al., 2022). We also evaluate on RGB (Chen et al., 2024a). We choose the English subset (refined version) focusing on noise robustness. For each instance, we select five top negative passages to form a worst-case scenario. Following prior work, we report the accuracy by string match. More details are in Appx. B.

Models and General Settings. We conduct experiments on advanced proprietary and open-source LLMs of different scales, including Claude 3.5 Sonnet (claude-3-5-sonnet@20240620),\footnote{https://www.anthropic.com/news/claude-3-5-sonnet} Gemini 1.5 Pro (gemini-1.5-pro-002),\footnote{https://deepmind.google/technologies/gemini/pro/} Mistral-Large (128B; version 2407), and Mistral-Nemo (12B; version 2407). The generation temperature is set to 0 and the maximum output tokens is set to 1,024. All experiments are under the zero-shot setting for controlled evaluation.

Baselines. We compare ASTUTE RAG with various RAG methods designed for enhanced robustness. *USC* (Chen et al., 2024b) is a self-consistency method that samples multiple LLM responses and aggregates the answers. It provides a reference of naive improvements using additional API calls. *Genread* (Yu et al., 2023a) augments retrieved passages with LLM-generated passages without explicit consolidation process. *RobustRAG* (Xiang et al., 2024) aggregates answers from independent passages to provide certifiable robustness. We use the best-performing keyword aggregation variant. *InstructRAG* (Wei et al., 2024) instructs the LLM to provide a rationale connecting the answer with information in passages. For a fair comparison, no training is applied. *Self-Route* (Xu et al., 2024) adaptively switches between LLMs with and without RAG.\footnote{The original Self-Route switches between RAG and long-context LLMs, while our implementation switches between RAG and No RAG according to our problem formulation.} It provides a reference of switching between LLMs’ internal and external knowledge.

Implementation Details. The prompt templates for ASTUTE RAG can be found in Appx. A. By default, we set \( t = 1 \) and \( \hat{m} = 1 \) to limit the number of additional tokens used. Results with larger \( t \) and \( \hat{m} \) are discussed in Sec. 5.3.

5.2 Main Results

Performance under real-world retrieval. Tab. 1 presents the results with real-world retrieval aug-
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="6">Claude 3.5 Sonnet (20240620)</th>
    <th colspan="6">Gemini 1.5 Pro (002)</th>
  </tr>
  <tr>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>47.1</td>
    <td>82.0</td>
    <td>50.4</td>
    <td>29.8</td>
    <td>54.5</td>
    <td>44.8</td>
    <td>80.2</td>
    <td>45.8</td>
    <td>25.3</td>
    <td>51.3</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>44.4</td>
    <td>76.7</td>
    <td>58.0</td>
    <td>36.0</td>
    <td>55.5</td>
    <td>42.7</td>
    <td>76.0</td>
    <td>55.2</td>
    <td>33.7</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td><b>USC (Chen et al., 2024b)</b></td>
    <td>48.1</td>
    <td>80.2</td>
    <td><b>61.5</b></td>
    <td>37.6</td>
    <td>58.7</td>
    <td>46.4</td>
    <td>76.7</td>
    <td><b>58.4</b></td>
    <td>37.6</td>
    <td>56.4</td>
  </tr>
  <tr>
    <td>GenRead (Yu et al., 2023a)</td>
    <td>42.0</td>
    <td>74.2</td>
    <td>57.0</td>
    <td>34.3</td>
    <td>53.6</td>
    <td>45.1</td>
    <td>77.4</td>
    <td>54.9</td>
    <td>34.3</td>
    <td>54.7</td>
  </tr>
  <tr>
    <td>RobustRAG (Xiang et al., 2024)</td>
    <td>47.8</td>
    <td>78.1</td>
    <td>56.3</td>
    <td>37.1</td>
    <td>56.5</td>
    <td>34.2</td>
    <td>67.5</td>
    <td>44.1</td>
    <td>32.0</td>
    <td>45.6<sup>a</sup></td>
  </tr>
  <tr>
    <td>InstructRAG (Wei et al., 2024)</td>
    <td>47.1</td>
    <td>83.0</td>
    <td>58.0</td>
    <td>41.0</td>
    <td>58.8</td>
    <td>46.8</td>
    <td>80.6</td>
    <td>54.9</td>
    <td>34.8</td>
    <td>56.1</td>
  </tr>
  <tr>
    <td>Self-Route (Xu et al., 2024)</td>
    <td>47.5</td>
    <td>78.8</td>
    <td>59.1</td>
    <td>41.0</td>
    <td>58.1</td>
    <td>47.5</td>
    <td>79.9</td>
    <td>58.0</td>
    <td>38.2</td>
    <td>57.6</td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td><b>52.2</b></td>
    <td><b>84.1</b></td>
    <td>60.1</td>
    <td><b>44.4</b></td>
    <td><b>61.7</b></td>
    <td><b>50.2</b></td>
    <td><b>81.6</b></td>
    <td>58.0</td>
    <td><b>40.5</b></td>
    <td><b>59.2</b></td>
  </tr>
  <tr>
    <th colspan="11">Mistral-Large (2407), 128B</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>46.8</td>
    <td>79.5</td>
    <td>43.7</td>
    <td>24.7</td>
    <td>51.1</td>
    <td>29.8</td>
    <td>67.8</td>
    <td>34.3</td>
    <td>23.0</td>
    <td>40.2</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>43.1</td>
    <td>77.4</td>
    <td>55.9</td>
    <td>36.0</td>
    <td>54.7</td>
    <td>39.3</td>
    <td>66.8</td>
    <td>49.0</td>
    <td>32.6</td>
    <td>48.3</td>
  </tr>
  <tr>
    <td><b>USC (Chen et al., 2024b)</b></td>
    <td><b>51.2</b></td>
    <td>80.9</td>
    <td><b>61.5</b></td>
    <td>36.0</td>
    <td>59.5</td>
    <td>29.5</td>
    <td>66.1</td>
    <td>36.0</td>
    <td>20.2</td>
    <td>39.6</td>
  </tr>
  <tr>
    <td>GenRead (Yu et al., 2023a)</td>
    <td>40.7</td>
    <td>73.1</td>
    <td>55.6</td>
    <td>35.4</td>
    <td>52.7</td>
    <td>38.6</td>
    <td>68.9</td>
    <td>48.3</td>
    <td><b>33.7</b></td>
    <td>48.7</td>
  </tr>
  <tr>
    <td>RobustRAG (Xiang et al., 2024)</td>
    <td>42.7</td>
    <td>77.7</td>
    <td>50.4</td>
    <td>34.8</td>
    <td>53.0</td>
    <td>35.6</td>
    <td>71.7</td>
    <td>44.1</td>
    <td>27.5</td>
    <td>46.4</td>
  </tr>
  <tr>
    <td>InstructRAG (Wei et al., 2024)</td>
    <td>45.4</td>
    <td>80.6</td>
    <td>57.3</td>
    <td>36.5</td>
    <td>56.7</td>
    <td>38.3</td>
    <td>61.8</td>
    <td>50.4</td>
    <td>23.6</td>
    <td>45.5</td>
  </tr>
  <tr>
    <td>Self-Route (Xu et al., 2024)</td>
    <td>45.4</td>
    <td>77.7</td>
    <td>57.3</td>
    <td>38.2</td>
    <td>56.2</td>
    <td>41.4</td>
    <td>73.5</td>
    <td><b>51.8</b></td>
    <td>30.9</td>
    <td>51.2</td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td>50.2</td>
    <td><b>82.7</b></td>
    <td>58.4</td>
    <td><b>42.1</b></td>
    <td><b>59.9</b></td>
    <td><b>42.7</b></td>
    <td><b>73.9</b></td>
    <td>49.3</td>
    <td>32.6</td>
    <td><b>51.3</b></td>
  </tr>
</table>

Table 1: Main results on Claude 3.5 Sonnet, Gemini 1.5 Pro, Mistral-Large, and Mistral-Nemo under zero-shot setting, showing the accuracy of benchmarked alternatives vs. ASTUTE RAG. Best scores are in bold. Note that USC consumes approximately three times more tokens than other RAG methods, and is not directly comparable.

![Bar chart comparing performance on ASQA](page_1012_1012_340_246.png)
Figure 5: Performance on ASQA.

![Bar chart comparing worst-case performance on RGB](page_1342_1012_340_246.png)
Figure 6: Worst-case performance of Claude on RGB. ASTUTE RAG reaches a performance close to No RAG, while other RAG systems are far behind.

mentation of various LLMs. We find that retrieved passages might not always bring benefits – on NQ and TriviaQA, RAG performance lags behind No RAG for advanced LLMs. We attribute this questions being covered by the LLM’s internal knowledge and the noise in retrieval results misleading the LLM. In contrast, on BioASQ and PopQA, which focus on domain-specific and ‘long-tail’ questions, RAG significantly improves the LLM performance. Due to imperfect retrieval augmentation, however, the absolute performance still remains to be unsatisfactory. Among all baselines, no single method consistently outperforms others across all datasets and LLMs. This observation highlights these baselines being tailored to distinct settings and not being universally applicable. Overall, InstructRAG and Self-Route demonstrate relatively superior performance among other alternatives. ASTUTE RAG consistently outperforms baselines across all LLMs in terms of overall accuracy. The relative improvement compared to the best baseline is 6.85% for Claude and 4.13% for Gemini, with the improvements in domain-specific questions being much higher. These highlight the effectiveness of ASTUTE RAG in overcoming imperfect retrieval augmentation and knowledge conflicts. Additionally, we observe consistent improvements on the open-source Mistral models. The results demonstrate that ASTUTE RAG generalizes well to LLMs of smaller sizes.

Performance on long-form QA. We conduct additional experiments on the long-form QA dataset, ASQA. Fig. 5 demonstrates that ASTUTE RAG consistently achieves significant improvements, re-
inforcing its effectiveness across diverse scenarios.

Worst-case performance on RGB. Fig. 6 presents the results under the worst-case setting on RGB where all retrieved documents are negative, to demonstrate robustness. The performance gap between RAG and No RAG exceeds 50 points, highlighting the detrimental impact of imperfect retrieval results and emphasizing the importance of providing robust safeguards against worst-case scenarios. While the baseline RAG methods outperform the original RAG, they still obviously fall behind ‘No RAG’. ASTUTE RAG is the only RAG method that reaches a performance close to ‘No RAG’, further supporting its effectiveness in addressing imperfect retrieval augmentation.

5.3 Analyses

We conduct in-depth analyses using Claude following the setting of Tab. 1.

The impact of retrieval precision. As shown in Fig. 7, ASTUTE RAG achieves consistently better performance across different retrieval precision regimes, indicating its effectiveness in improving RAG trustworthiness in broad scenarios. Notably, ASTUTE RAG does not sacrifice performance gain under high retrieval quality in exchange for improvement under low retrieval quality. When the retrieval quality is extremely low (close to zero precision), all other RAG variants underperform the ‘No RAG’ baseline, except for ASTUTE RAG.

Addressing knowledge conflicts. We split our collected data into three subset according to the answers with and without RAG: the answers from two can be (i) both correct, (ii) both incorrect, or (iii) conflicting with one being correct. The results are shown in Fig. 8. On the conflicting subset, ASTUTE RAG successfully chooses the correct answer in approximately 80% of cases, being the most effective one in addressing knowledge conflicts. Notably, ASTUTE RAG even brings performance improvement on the subset where neither internal nor external knowledge alone leads to the correct answer. This indicates that ASTUTE RAG can effectively combine partially-correct information from LLM-internal and external knowledge.

Benefits of more consolidation iteration. For efficiency, we employ a single iteration of knowledge consolidation in our main experiments. However, incorporating multiple iterations has the potential to further enhance model performance as shown in Fig. 9. The magnitude of this improvement diminishes as \( t \) increases, indicating that the knowledge has been better presented and less improvement space left after each iteration.

Efficiency in tokens consumed and API calls. As a proxy to overall prediction cost and latency, we present the average number of tokens and API calls used per instance in Fig. 10 and Fig. 11. ASTUTE RAG incurs only a marginal cost increase, <5%, while delivering substantial improvement, >11%, compared to the RAG baseline.

Effectiveness of adaptive generation. The results in Tab. 2 illustrate the model’s performance when varying the maximum number of passages generated. The design of adaptive generation has been effectively reflected, as the number of generated passages is dynamically adjusted leading to \( m < \hat{m} \). Notably, the number of generated passages can be controlled by \( \hat{m} \), and results show that the system does not generate passages excessively.

Figure 7: Performance across different retrieval precision buckets. ASTUTE RAG is consistently better.

![Line chart showing performance across different retrieval precision buckets for Astute RAG, Self-Route, Instruct RAG, RAG, and No RAG.](page_120_120_480_320.png)

Figure 8: Performance on conflicting and consistent instances between No RAG and RAG.

![Bar chart comparing accuracy (%) for Instruct RAG, Self-Route, and Astute RAG across Both Incorrect, Conflicting, and Both Correct subsets.](page_720_120_480_320.png)
Figure 9: Accuracy improvement when increasing \( t \).

Impact of Source-Awareness. To evaluate the impact of source-awareness, we conducted an ablation study where source labels (own memory and external retrieval) were removed during the consolidation and answer generation process. The results are shown in Tab. 3. The comparison shows that providing source information (ASTUTE RAG) leads to better performance overall compared to omitting it (ASTUTE RAG (No Source)), particularly on NQ, PopQA, and BioASQ, suggesting that awareness of information origin aids the consolidation process.

Accuracy of intermediate steps. To investigate the performance of intermediate steps, including knowledge consolidation and confidence assignment, we use LLM-as-a-judge with the instruction in Appx. A. Our experimental results show that the accuracy for knowledge consolidation is 98.2%, and for confidence assignment, it is 95.0%. These results demonstrate the effectiveness of the proposed framework in the intermediate stages.

Qualitative examples. In Fig. 12, we present two representative examples showing the intermediate outputs of ASTUTE RAG. In the first example, LLM without RAG generates a wrong answer, while RAG returns a correct answer. ASTUTE RAG successfully identified the incorrect information in its generated passage and an external passage, avoiding confirmation bias (Tan et al., 2024). In the second example, LLM is correct but RAG is incorrect due to imperfect retrieval. ASTUTE RAG detected the correct answer from imperfect context leveraging internal knowledge.

6 Conclusion

We investigate the impact of imperfect retrieval on the performance of RAG systems and identify knowledge conflicts as a key challenge. To address this, we introduce ASTUTE RAG, a novel approach that leverages the internal knowledge of LLMs and iteratively refines the generated responses by consolidating internal and external knowledge in a source way. We demonstrate the effectiveness of ASTUTE RAG in mitigating the negative effects of imperfect retrieval and improving the robustness of RAG, particularly in challenging scenarios with unreliable external sources.

Acknowledgement

We would like to thank Jinsung Yoon for valuable discussions and insights that helped to improve this paper. We would also like to thank all other colleagues from Google Cloud AI Research for their valuable feedback.

Limitations

ASTUTE RAG’s effectiveness hinges on the capabilities of advanced LLMs with strong instruction-following and reasoning abilities, hence potentially more limited applicability with less sophisticated LLMs. As an important future direction, extending the experimental setup to include longer inputs would be important, where the challenges of imperfect retrieval and knowledge conflicts may be even more pronounced.

References

Marco Alessio, Guglielmo Faggioli, Nicola Ferro, Franco Maria Nardini, Raffaele Perego, et al. 2024. Improving rag systems via sentence clustering and reordering. In RAG@ SIGIR 2024 workshop: The Information Retrieval’s Role in RAG Systems, ACM, pages 1–10.

Akari Asai, Zequi Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.

Figure 10: Efficiency in terms of tokens consumed.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Milligan, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206–2240. PMLR.

Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024a. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17754–17762.

Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Ke-fan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. 2024b. Universal self-consistency for large language models. In ICML 2024 Workshop on In-Context Learning.

Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The power of noise: Redefining retrieval for rag systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 719–729.

Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. 2024. Unifying bias and unfairness in information retrieval: A survey of challenges and opportunities with large language models. arXiv preprint arXiv:2404.11457.

Feiting Fang, Yuelin Bai, Shiwen Ni, Min Yang, Xiaojun Chen, and Ruifeng Xu. 2024. Enhancing noise robustness of retrieval-augmented language models with adaptive adversarial training. arXiv preprint arXiv:2405.20978.

Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengcheng Cai, and Alfio Gliozzo. 2022. Re2g: Retrieve, rerank, generate. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2701–2715.

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR.

Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong C Park. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7029–7043.

Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992.

Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiejin Xu, Li Qiuxia, and Jun Zhao. 2024. Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16867–16878.

Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kötter, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303–5315.

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9802–9822.

Ruoqiong Pan, Boxi Cao, Hongyu Lin, Xianpei Han, Jia Zheng, Sirui Wang, Xunliang Cai, and Le Sun. 2024. Not all contexts are equal: Teaching llms credibility-aware generation. arXiv preprint arXiv:2404.06809.
Quang Hieu Pham, Hoang Ngo, Anh Tuan Luu, and Dat Quoc Nguyen. 2024. Who’s who: Large language models meet knowledge conflicts in practice. arXiv preprint arXiv:2410.15737.

Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, and Pang Wei Koh. 2024. Scaling retrieval-based language models with a trillion-token datastore. arXiv preprint arXiv:2407.12854.

Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. Asqa: Factoid questions long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8273–8288.

Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muenmighoff, Han-yu Wang, Haisu Liu, Quan Shi, Zachary S Siegel, Michael Tang, et al. 2024. Bright: A realistic and challenging benchmark for reasoning-intensive retrieval. arXiv preprint arXiv:2407.12883.

Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang, Qi Cao, and Xueqi Cheng. 2024. Blinded by generated contexts: How language models merge generated and retrieved contexts for open-domain qa? arXiv preprint arXiv:2401.11911.

Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2024. Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).

George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, et al. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics, 16:1–28.

Haotian Wang, Xiyuan Du, Weijiang Yu, Qianglong Chen, Kun Zhu, Zheng Chu, Lian Yan, and Yi Guan. 2023a. Apollo’s oracle: Retrieval-augmented reasoning in multi-agent debates. arXiv preprint arXiv:2312.04854.

Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, and Graham Neubig. 2023b. Learning to filter context for retrieval-augmented generation. arXiv preprint arXiv:2311.08377.

Zilong Wang, Zifeng Wang, Long Le, Huaxiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al. 2024. Speculative rag: Enhancing retrieval augmented generation through drafting. arXiv preprint arXiv:2407.08223.

Zhepei Wei, Wei-Lin Chen, and Yu Meng. 2024. Instructrag: Instructing retrieval-augmented generation with explicit denoising. arXiv preprint arXiv:2406.13629.

Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. 2024. Certifiably robust rag against retrieval corruption. arXiv preprint arXiv:2405.15556.

Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2024. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In The Twelfth International Conference on Learning Representations.

Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. Recomp: Improving retrieval-augmented lms with compression and selective augmentation. arXiv preprint arXiv:2310.04408.

Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Retrieval meets long context large language models. In The Twelfth International Conference on Learning Representations.

Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884.

Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, et al. 2024. Crag–comprehensive rag benchmark. arXiv preprint arXiv:2406.04744.

Chanwoong Yoon, Taewho Lee, Hyeon Hwang, Min-byul Jeong, and Jaewoo Kang. 2024. Compact: Compressing retrieved documents actively for question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21424–21439.

Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. Making retrieval-augmented language models robust to irrelevant context. In The Twelfth International Conference on Learning Representations.

Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023a. Generate rather than retrieve: Large language models are strong context generators. In The Eleventh International Conference on Learning Representations.

Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu. 2023b. Chain-of-note: Enhancing robustness in retrieval-augmented language models. arXiv preprint arXiv:2311.09210.

Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. arXiv preprint arXiv:2407.02485.
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, and Lu Wang. 2023. Merging generated and retrieved knowledge for open-domain qa. In The 2023 Conference on Empirical Methods in Natural Language Processing.

Zheng Zhao, Emilio Monti, Jens Lehmann, and Haytham Assem. 2024. Enhancing contextual understanding in large language models through contrastive decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4225–4237.

Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. 2024. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models. arXiv preprint arXiv:2402.07867.
<table>
  <tr>
    <th></th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
    <th>\( m \)</th>
  </tr>
  <tr>
    <td>\( \hat{m}=1 \)</td>
    <td>52.20</td>
    <td>84.10</td>
    <td>60.14</td>
    <td>44.38</td>
    <td>61.71</td>
    <td>0.69</td>
  </tr>
  <tr>
    <td>\( \hat{m}=2 \)</td>
    <td>52.20</td>
    <td>85.16</td>
    <td>60.84</td>
    <td>43.26</td>
    <td>62.00</td>
    <td>1.24</td>
  </tr>
</table>

Table 2: Performance and average number of generated passages using different \( \hat{m} \).

![Scatter plot showing overall accuracy vs. average API calls for various models](page_573_671_495_246.png)

Figure 11: Efficiency in terms of API calls.

<table>
  <tr>
    <th>Model</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td><b>50.2</b></td>
    <td><b>81.6</b></td>
    <td><b>58.0</b></td>
    <td><b>40.5</b></td>
    <td><b>59.2</b></td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG (No Source)</b></td>
    <td>48.1</td>
    <td>82.3</td>
    <td>57.7</td>
    <td>39.9</td>
    <td>58.6</td>
  </tr>
</table>

Table 3: Ablation study on source-awareness.
LLM: Zachary Levi
RAG: Jason Lee
Question
Who plays David in Alvin and the Chipmunks?
Generate Passage
The character Dave Seville in all of the Alvin and the Chipmunks movies, both live-action and animated, is played by Zachary Levi.

Consolidate Information
Group A (External Retrieval):
Documents 2, 6, 9, 11
Jason Lee plays David Seville in the live-action Alvin and the Chipmunks movies ...

Group B (Own Memory/External Retrieval):
Documents 1, 7
Zachary Levi plays Toby Seville (who becomes David Seville) in the live-action movies ...

Group C (External Retrieval):
Documents 3, 4, 5, 8, 10
... These are irrelevant to the question of who plays David Seville.

Propose Answer and Assign Confidence
Answer A (from Group A):
Jason Lee.
**Confidence: High (95%).**
Multiple sources confirm this information.

Answer B (from Group B):
Zachary Levi.
**Confidence: Low (25%).**
My own memory conflicts with external sources, and this group also introduces a different character (Ian) played by David Cross, creating confusion.

LLM: Jack Nicholson
RAG: No Answer
Question
Who won the Oscar for best actor when Titanic was nominated?
Generate Passage
Titanic was nominated for ... Jack Nicholson won the Best Actor Oscar at the 70th Academy Awards in 1998 for his role in As Good as It Gets.

Consolidate Information
Group 1 (Memory & External):
Documents: 1, 6
Jack Nicholson won the Best Actor Oscar ...

Group 2 (External):
Documents: 2, 7
Walt Disney holds the record for most Oscar ...

Group 3 (External):
Documents: 3, 5, 6
Titanic (1997) received 14 Oscar nominations ...

Group 4 (External):
Documents: 4
Leonardo DiCaprio was not nominated ...

Group 5 (External):
Documents: 8, 9, 10, 11
... irrelevant to the question.

Propose Answer and Assign Confidence
Answer from Group 1:
Jack Nicholson.
**Confidence: High (100%)**
Consistent information from memory and a reliable external source.

Groups 2, 3, 4, 5:
Irrelevant to the question.

Figure 12: Qualitative examples. Top: ASTUTE RAG identified the error in internal knowledge (i.e., generated passage) by confirming with external sources. Bottom: ASTUTE RAG detected the correct answer from imperfect retrieval by checking with its internal knowledge. Standard RAG does not provide an answer because the retrieved passages are too noisy.
A  Prompt Template for ASTUTE RAG

Prompt for Adaptive Passage Generation (\( p_{gen} \))

Generate a document that provides accurate and relevant information to answer the given question. If the information is unclear or uncertain, explicitly state 'I don’t know' to avoid any hallucinations.

Question: {question} Document:

Prompt for Iterative Knowledge Consolidation (\( p_{con} \))

Task: Consolidate information from both your own memorized documents and externally retrieved documents in response to the given question.

* For documents that provide consistent information, cluster them together and summarize the key details into a single, concise document.
* For documents with conflicting information, separate them into distinct documents, ensuring each captures the unique perspective or data.
* Exclude any information irrelevant to the query.
For each new document created, clearly indicate:
* Whether the source was from memory or an external retrieval.
* The original document numbers for transparency.

Initial Context: {context}
Last Context: {context}
Question: {question}
New Context:
Prompt for Knowledge Consolidation and Answer Finalization (\( p_{ans} \))

Task: Answer a given question using the consolidated information from both your own memorized documents and externally retrieved documents.

Step 1: Consolidate information
* For documents that provide consistent information, cluster them together and summarize the key details into a single, concise document.
* For documents with conflicting information, separate them into distinct documents, ensuring each captures the unique perspective or data.
* Exclude any information irrelevant to the query.
For each new document created, clearly indicate:
* Whether the source was from memory or an external retrieval.
* The original document numbers for transparency.

Step 2: Propose Answers and Assign Confidence
For each group of documents, propose a possible answer and assign a confidence score based on the credibility and agreement of the information.

Step 3: Select the Final Answer
After evaluating all groups, select the most accurate and well-supported answer.
Highlight your exact answer within <ANSWER> your answer </ANSWER>.

Initial Context: {context_init}
[Consolidated Context: {context}] # optional
Question: {question}
Answer:

Prompt for Intermediate Step Evaluation

**Task:** You are provided with the following:
1. A question.
2. The correct answer.
3. The input context.
4. The model’s response, which contains:
- Consolidated context.
- Confidence scores for candidate answers.
Your task is to:
- Evaluate the **quality of the consolidated context** in the model’s response and provide a label: '<consolidation> correct </consolidation>' or '<consolidation> incorrect </consolidation>'.
This evaluation is only about whether the consolidation is correct given the input context.
- Evaluate the **accuracy of the confidence score** (whether it aligns with the confidence of the supporting context) and provide a label: '<confidence> correct </confidence>' or '<confidence> incorrect </confidence>'. The evaluation is only based on the consolidated context.
Note that correct consolidation and confidence do not necessarily indicate the correct answer.
Question: {query}
Correct Answer: {answer}
Input Context: {input}
Model Response: {response}
Evaluation:
B Data Collection

Encompassing a diverse range of natural questions, our benchmark consists of realistic retrieval results with Google Search8 as the retriever and the Web as the corpus. Notably, we do not select questions or annotate answers based on the retrieval results. This setting allows us to analyze the severity of imperfect retrieval in real-world RAG. It distinguishes our benchmark from previous ones that employ synthetic retrieval corruptions or that unintentionally reduce the frequency of imperfect retrieval with biased construction protocols (Chen et al., 2024a; Yang et al., 2024). Overall, our benchmark contains 1,042 short-form question-answer pairs, each paired with 10 retrieved passages. When collecting the passages, we retrieve the top 30 results and select the first 10 accessible websites. From each retrieved website, we extract the paragraph corresponding to the snippet provided in the search results as the retrieved passage. Retrieved results might contain natural noise with irrelevant or misleading information. We do not consider enhancements to the retrieval side, such as query rewriting, as such enhancements are typically already incorporated into commercial information retrieval systems. All of these datasets are short-form QA. Following previous work (Xiang et al., 2024; Wei et al., 2024; Mallen et al., 2023), a model response is considered correct if it contains the ground-truth answer. To enhance evaluation reliability, we prompt LLMs to enclose the exact answer within special tokens, extracting them as the final responses.

Question-answer pairs. We consider question-answer pairs from four datasets of different properties spanning across general questions, domain-specific questions, and long-tail questions. NQ (Kwiatkowski et al., 2019) and TriviaQA (Joshi et al., 2017) are two widely-studied question-answering (QA) datasets in general domains. BioASQ (Tsatsaronis et al., 2015) is from biomedical domain that has demonstrated significant benefits from RAG when general-purpose LLMs are considered. PopQA (Mallen et al., 2023) focuses on long-tail knowledge and has been shown to be challenging for even advanced LLMs to solve without external knowledge. All these datasets contain questions with short-form answers and most of them list all valid answer variants. This format can support automatic verification of answer appearance in retrieved passages and model responses, leading to more precise evaluations.

Retrieval process. For each question in our benchmark, we query Google Search to retrieve the top 30 results and select the first 10 accessible websites. From each retrieved website, we extract the paragraph corresponding to the snippet provided in Google Search results as the retrieved passage. We do not consider enhancements to the retrieval side, such as query rewriting, as such enhancements are typically already incorporated into commercial information retrieval systems.

C Comparison with Answer Refinement

In Tab. 4, we further compare ASTUTE RAG with Answer Refinement, where the LLM is prompted to refine its initial answer by reconsidering external context. Notably, this baseline performs nearly identically to Self-Route, which is expected, because both approaches rely on the model itself to determine whether to revise its initial answer based on external knowledge. The comparison underscores that simply enabling LLMs to self-correct does not yield significant improvements beyond existing routing strategies. In contrast, ASTUTE RAG continues to outperform all baselines across datasets, reinforcing the benefit of explicitly structured consolidation mechanisms.

D Comparison with Context Filtering

To isolate the effect of identifying irrelevant information separate from consolidation, we conducted an ablation study introducing two Context Filtering baselines. The first baseline filters only the retrieved documents, while the second filters both retrieved and generated documents prior to answer generation, both without applying the consolidation step. As shown in Tab. 5, while context filtering improves performance over the basic RAG baseline, it falls short of the performance achieved by ASTUTE RAG. This supports our hypothesis that consolidating diverse information, including consistent, conflicting, and relevant content, rather than merely filtering out irrelevant parts, is critical to the performance gains observed with ASTUTE RAG.

E Comparison with Context Compression

Context compression (Wang et al., 2023b; Yoon et al., 2024) is also a related direction. We further
<table>
  <tr>
    <th>Model</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>Self-Route</td>
    <td>47.5</td>
    <td>79.9</td>
    <td>58.0</td>
    <td>38.2</td>
    <td>57.6</td>
  </tr>
  <tr>
    <td>Answer Refinement</td>
    <td>47.1</td>
    <td>79.9</td>
    <td>58.0</td>
    <td>38.2</td>
    <td>57.5</td>
  </tr>
  <tr>
    <td><b>ASTUTE RAG</b></td>
    <td>50.2</td>
    <td>81.6</td>
    <td>58.0</td>
    <td>40.5</td>
    <td>59.2</td>
  </tr>
</table>

Table 4: Comparison of Astute RAG with routing and refinement-based baselines on Gemini.

<table>
  <tr>
    <th>Model</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>RAG</td>
    <td>42.7</td>
    <td>76.0</td>
    <td>55.2</td>
    <td>33.7</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td>Context Filtering (retrieved)</td>
    <td>43.7</td>
    <td>77.0</td>
    <td>57.0</td>
    <td>34.3</td>
    <td>54.8</td>
  </tr>
  <tr>
    <td>Context Filtering (generated and retrieved)</td>
    <td>49.2</td>
    <td>79.2</td>
    <td>56.6</td>
    <td>40.4</td>
    <td>57.9</td>
  </tr>
  <tr>
    <td><b>Astute RAG</b></td>
    <td>50.2</td>
    <td>81.6</td>
    <td>58.0</td>
    <td>40.5</td>
    <td>59.2</td>
  </tr>
</table>

Table 5: Comparison with context filtering baselines.

conduct experiments comparing our method with CompAct (Yoon et al., 2024). The results in Tab. 6 and Tab. 7 show that context compression is ineffective in handling the challenges of imperfect context and knowledge conflicts. Notably, it even performs worse than the No RAG and RAG baselines in this context. The primary reason for this underperformance lies in the limitations of context compression. It struggles to effectively identify incorrect information when there are conflicts in context and often removes or reduces the appearance of helpful information in the process. This reinforces the importance of our approach, which does not rely solely on compression but instead integrates both internal and external knowledge while handling conflicts in a more nuanced manner.

F Influence of passage ordering.

We apply different ordering strategies (Alessio et al., 2024) on RAG and ASTUTE RAG. As shown in Tab. 8, we find that the improvement with ASTUTE RAG is significantly larger than the gap between different ordering strategies. Moreover, the consolidation process makes ASTUTE RAG less sensitive to it.
<table>
  <tr>
    <th>Method</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>47.1</td>
    <td>82.0</td>
    <td>50.4</td>
    <td>29.8</td>
    <td>54.5</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>44.4</td>
    <td>76.7</td>
    <td>58.0</td>
    <td>36.0</td>
    <td>55.5</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>38.6</td>
    <td>68.9</td>
    <td>49.3</td>
    <td>30.3</td>
    <td>48.4</td>
  </tr>
  <tr>
    <td>ASTUTE RAG</td>
    <td>52.2</td>
    <td>84.1</td>
    <td>60.1</td>
    <td>44.4</td>
    <td>61.7</td>
  </tr>
</table>

Table 6: Comparison with context compression on Claude.

<table>
  <tr>
    <th>Method</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>No RAG</td>
    <td>44.8</td>
    <td>80.2</td>
    <td>45.8</td>
    <td>25.3</td>
    <td>51.3</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>42.7</td>
    <td>76.0</td>
    <td>55.2</td>
    <td>33.7</td>
    <td>53.7</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>35.3</td>
    <td>65.0</td>
    <td>47.6</td>
    <td>30.9</td>
    <td>46.0</td>
  </tr>
  <tr>
    <td>ASTUTE RAG</td>
    <td>50.2</td>
    <td>81.6</td>
    <td>58.0</td>
    <td>40.5</td>
    <td>59.2</td>
  </tr>
</table>

Table 7: Comparison with context compression on Gemini.

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th rowspan="2">Ordering Strategy</th>
    <th colspan="5">NQ TriviaQA BioASQ PopQA Overall</th>
  </tr>
  <tr>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>BioASQ</th>
    <th>PopQA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td rowspan="5">RAG</td>
    <td>Random</td>
    <td>43.39</td>
    <td>76.33</td>
    <td>56.99</td>
    <td>34.83</td>
    <td>54.61</td>
  </tr>
  <tr>
    <td>Ascending</td>
    <td>43.05</td>
    <td>75.62</td>
    <td>57.69</td>
    <td>34.83</td>
    <td>54.51</td>
  </tr>
  <tr>
    <td>Descending</td>
    <td>44.41</td>
    <td>76.68</td>
    <td>58.04</td>
    <td>35.96</td>
    <td>55.47</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Top-to-bottom</td>
    <td>44.75</td>
    <td>77.39</td>
    <td>57.69</td>
    <td>35.96</td>
    <td>55.66</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Bottom-to-top</td>
    <td>44.41</td>
    <td>75.62</td>
    <td>58.04</td>
    <td>35.96</td>
    <td>55.18</td>
  </tr>
  <tr>
    <td rowspan="5">AstuteRAG</td>
    <td>Random</td>
    <td>51.86</td>
    <td>84.81</td>
    <td>61.19</td>
    <td>41.57</td>
    <td>61.61</td>
  </tr>
  <tr>
    <td>Ascending</td>
    <td>51.86</td>
    <td>85.51</td>
    <td>59.79</td>
    <td>42.13</td>
    <td>61.52</td>
  </tr>
  <tr>
    <td>Descending</td>
    <td>52.20</td>
    <td>84.10</td>
    <td>60.14</td>
    <td>44.38</td>
    <td>61.71</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Top-to-bottom</td>
    <td>52.20</td>
    <td>84.45</td>
    <td>59.09</td>
    <td>43.82</td>
    <td>61.42</td>
  </tr>
  <tr>
    <td>Ping-pong Descending Bottom-to-top</td>
    <td>51.19</td>
    <td>85.16</td>
    <td>61.54</td>
    <td>43.82</td>
    <td>62.00</td>
  </tr>
</table>

Table 8: Performance by Ordering Strategies.

Reviewer paper summaries: The ASTUTE RAG framework enhances the performance of large language models (LLMs) by dynamically integrating internal and external knowledge, resolving conflicts, and generating reliable answers based on information consistency and source reliability. It requires no additional training, making it highly flexible and suitable for black-box LLMs. Experiments across various datasets and models demonstrate its significant advantages over existing methods, particularly in worst-case scenarios, where it surpasses retrieval-free modes in accuracy and robustness. However, its reliance on advanced LLM capabilities limits applicability to weaker models. It primarily focuses on short-answer tasks, with unverified performance on long texts or complex contexts. Additionally, the adaptive generation mechanism may increase computational burden, and the datasets used lack problem diversity, leaving multi-answer tasks unexplored. The conflict resolution mechanism also requires more detailed analysis of how reliability is assessed and conflicts are resolved.

The system dynamically generates internal passages based on the LLM's pre-trained knowledge, ensuring reliability and relevance with Iterative Knowledge Consolidation: a source-aware mechanism that consolidates information from both internal and external sources, resolving conflicts and excluding irrelevant data.

The paper proposes an iterative training-free technique to combine parametric and external knowledge for Retrieval Augmented Generation.
Summary of strengths: 1. The ASTUTE RAG framework dynamically extracts internal knowledge from LLMs to supplement external information, reducing the impact of errors or irrelevant retrievals. It generates final answers based on information consistency and source reliability, ensuring accurate responses.
2. The proposed ASTUTE RAG method does not rely on additional training, making it suitable for black-box LLMs, with high applicability and flexibility.
3. Experiments across multiple datasets and models demonstrate that ASTUTE RAG significantly outperforms existing methods in terms of accuracy and robustness. In worst-case scenarios (where all retrieved content is invalid), ASTUTE RAG is the only method capable of matching or surpassing the performance of retrieval-free modes.

1. The paper introduces ASTUTE RAG, a robust Retrieval-Augmented Generation (RAG) framework designed to overcome the critical issues of imperfect retrieval and knowledge conflicts. 
3. The paper systematically investigates the challenges of imperfect retrieval in real-world scenarios.

S1. The paper analyzes the effect of imperfect retrieval and the contribution of internal knowledge to mitigate this.

S2. This work proposes a simple prompt-based technique that shows better performance over the baselines.
Summary of weaknesses: 1. The effectiveness of ASTUTE RAG relies on the strong instruction-following and reasoning capabilities of the LLMs used, making it potentially unsuitable for less capable models.
2. The paper focuses primarily on short-answer scenarios and does not delve into handling long texts or complex contexts. The performance and efficiency of ASTUTE RAG in more complex scenarios, such as long document retrieval or multi-turn question answering, remain unverified.
3. The adaptive generation mechanism may result in an excessive number of generated passages, further increasing the computational burden during the knowledge consolidation stage. While the model dynamically determines the number of generated passages, the paper does not explain how the model evaluates the need for additional passages or avoids redundant generation.
4. Although the experimental datasets cover both general and domain-specific tasks, the complexity and diversity of the problems are limited. For instance, most questions involve single-answer verification, with no testing of multi-answer generation tasks.

1. While leveraging LLM-internal knowledge is a strength, it introduces risks when the internal knowledge is outdated or incomplete. The paper does not thoroughly address scenarios where LLM-internal knowledge itself is a primary source of error, potentially reducing robustness in such cases.
2. While the paper demonstrates minimal token consumption for a single iteration of knowledge consolidation, it does not provide a detailed analysis of computational overhead for multiple iterations, especially with large datasets or queries requiring extensive refinement. The trade-offs between performance improvement and efficiency at scale are not fully quantified.

W1. After the internal document generation, the rest of the “knowledge consolidation” process is very similar to evidence summarization/filtering [1,2]. This work does not discuss/compare these existing methods in the literature.

W2. Sec 5.1 mentions about RGB benchmark which has been reported to have quality issues in the original version. It is unclear whether the original or the refined version of the benchmark is used [3].

[1] Yoon, Chanwoong, et al. "Compact: Compressing retrieved documents actively for question answering." arXiv preprint arXiv:2407.09014 (2024).

[2] Wang, Zhiruo, et al. "Learning to filter context for retrieval-augmented generation." arXiv preprint arXiv:2311.08377 (2023).

[3] https://github.com/chen700564/RGB?tab=readme-ov-file#news
Comments, suggestions and typos: The core of the conflict resolution mechanism relies on the LLM’s understanding of the sources and conflicts in the passages, but the paper does not provide a detailed analysis of how the LLM assesses the reliability of conflicting information. When analyzing the relationship between conflict rates and accuracy, the paper only shows the trend of conflict rates varying with retrieval precision without further examining the specific categories of conflict sources.

Comments after Author Discussion

Thanks for your response; I boosted your score.

N/A

------
## Comments after Author Discussion
I have increased my rating as some of my concerns have been clarified regarding the novelty of their approach with evidence filtering.
Overall assessment scores: 3

3

3