Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

## Abstract

Abstract Retrieval-Augmented Generation (RAG), while effective in integrating external knowledge to address the limitations of large language models (LLMs), can be undermined by imperfect retrieval, which may introduce irrelevant, misleading, or even malicious information. Despite its importance, previous studies have rarely explored the behavior of RAG through joint analysis on how errors from imperfect retrieval attribute and propagate, and how potential conflicts arise between the LLMs’ internal knowledge and external sources. We find that imperfect retrieval augmentation might be inevitable and quite harmful, through controlled analysis under realistic conditions. We identify the knowledge conflicts between LLM-internal and external knowledge from retrieval as a bottleneck to overcome in the post-retrieval stage of RAG. To render LLMs resilient to imperfect retrieval, we propose Astute RAG , a novel RAG approach that adaptively elicits essential information from LLMs’ internal knowledge, iteratively consolidates internal and external knowledge with source-awareness , and finalizes the answer according to information reliability. Our experiments using Gemini and Claude demonstrate that Astute RAG significantly outperforms previous robustness-enhanced RAG methods. Notably, Astute RAG is the only approach that matches or exceeds the performance of LLMs without RAG under worst-case scenarios. Further analysis reveals that Astute RAG effectively resolves knowledge conflicts, improving the reliability and trustworthiness of RAG systems.

## 1 Introduction

Retrieval augmented generation (RAG) has become the standard approach for large language models (LLMs) to tackle knowledge-intensive tasks (Guu et al., ; Lewis et al., ).
Prior works mainly leverage RAG to address the inherent knowledge limitations of LLMs, effectively integrating missing information and grounding to reliable sources.
However, recent research has highlighted a significant drawback that RAG might rely on imperfect retrieval results, including irrelevant, misleading, or even malicious information, which eventually leads to inaccurate LLM responses (Chen et al., ; Xiang et al., ; Zou et al., ).
For example, when asked about the practice of eating rocks, LLMs might cite misleading information, such as a satirical news source claiming that one should consume at least one rock per day.(^1^11[https://www.bbc.com/news/articles/cd11gzejgz4o](https://www.bbc.com/news/articles/cd11gzejgz4o).) The occurrence of imperfect retrieval augmentation is inevitable, driven by factors such as corpus quality limitations (Shao et al., ), the reliability of retrievers (Dai et al., ), and the complexity of the queries (Su et al., ).
This poses a significant challenge to the trustworthiness of RAG.

While there have been independent analyses of information retrieval and RAG in the context of LLMs (Su et al., ; Mallen et al., ), previous studies have rarely connected the behaviors of retrieval and subsequent generation,
particularly regarding the propagation of information retrieval errors, which may lead to knowledge conflicts (Longpre et al., ; Wang et al., ; Xu et al., ) between LLMs and context. To this end, we conduct comprehensive analyses on the occurrence of imperfect retrieval augmentation and its impact on LLM behavior under realistic conditions (Section 2). We conduct controlled experiments on a diverse range of general, domain-specific, and long-tail questions from NQ (Kwiatkowski et al., ), TriviaQA (Joshi et al., ), BioASQ (Tsatsaronis et al., ), and PopQA (Mallen et al., ).
We observe that imperfect retrieval augmentation is widespread even with adept real-world search engine (such as Google Search with Web as corpus) – roughly 70% retrieved passages do not directly contain true answers, leading to the impeded performance of LLM with RAG augmentation.(^2^22Note that some passages may contain information indirectly relevant to the answer, but may unintentionally mislead or distract LLMs.)

These findings underscore the potential severity of the imperfect retrieval issue in real-world RAG and highlight the widespread existence of knowledge conflicts as the bottleneck to overcome it (Figure 1). Recent studies demonstrate that LLM-internal and external knowledge offer distinct advantages, but LLMs often struggle to consolidate conflicting information reliably, failing to respond based on collective knowledge (Mallen et al., ; Tan et al., ; Xie et al., ; Jin et al., ). This raises the following research question: Is there an effective method to combine internal (from LLMs’ pretrained weights) and external (from specific corpora or knowledge bases) knowledge for more reliable RAG? Previous work has widely explored using external knowledge to enhance LLMs through RAG. We seek to further leverage LLMs’ internal knowledge to recover from RAG failures

Figure: Figure 1: Knowledge conflicts between the LLMs’ internal knowledge and retrieved knowledge from external sources. We report the overall results with Claude under the setting in Section 4.1.
Refer to caption: extracted/5914248/figure/problem_3.png

Motivated by these important real-world challenges, we propose Astute RAG (Section 3), a novel RAG approach designed to be resilient to imperfect retrieval augmentation, while preserving RAG grounding effect when RAG is reliable. To this end, Astute RAG needs effectively differentiate the reliability of the LLM’s intrinsic knowledge and the external information retrieved in RAG, utilizing each only when trustworthy and ensuring proper integration. Specifically, Astute RAG initially elicits information from LLMs’ internal knowledge to explicitly complement the passages retrieved from external sources. Then, Astute RAG conducts source-aware knowledge consolidation of information from various internal and external sources. The desiderata is combining consistent information, identifying conflicting information, and filtering out irrelevant information. Finally, Astute RAG proposes answers based on each group of consistent passages and compares the answers from different passage groups to determine the final answer. Our experiments involving Gemini and Claude(^3^33[https://www.anthropic.com/claude](https://www.anthropic.com/claude)) on various datasets (Section 4) demonstrate the superior performance of Astute RAG compared to previous RAG approaches designed to be robust against retrieval corruptions. Moreover, Astute RAG consistently outperforms baselines across different retrieval quality levels. Notably, Astute RAG is the only RAG method that achieves performance comparable to or even surpassing conventional use of LLMs under the worst-case scenario where all retrieved passages are unhelpful. Further analysis reveals the effectiveness of Astute RAG in resolving knowledge conflicts between internal and external knowledge.

To conclude, our core contributions are threefold. First, we analyze RAG under realistic conditions, identifying imperfect retrieval augmentation as a significant contributor to RAG failures and pinpointing knowledge conflicts as the primary bottleneck in overcoming it. Second,
we propose Astute RAG, which explicitly addresses conflicts between LLM-internal and external knowledge, thereby recovering from RAG failures.
Third, experiments with various LLMs and datasets demonstrate the effectiveness of Astute RAG, even in the most challenging scenarios.

## 2 Imperfect Retrieval: The Pitfall of RAG

Figure: Figure 2: Imperfect retrieval (samples with low retrieval precision) is prevalent in real-world RAG.
Refer to caption: extracted/5914248/figure/retrieval.png

To better showcase the common real-world challenges and to make better motivate for improved methodological designs,
we evaluate retrieval quality, end-to-end RAG performance, and knowledge conflicts on a controlled set of data. The selected data encompass a diverse range of general, domain-specific, and long-tail questions from NQ (Kwiatkowski et al., ), TriviaQA (Joshi et al., ), BioASQ (Tsatsaronis et al., ), and PopQA (Mallen et al., ). Our analysis is based on realistic retrieval results with Google Search(^4^44[https://developers.google.com/custom-search/v1/overview](https://developers.google.com/custom-search/v1/overview)) as the retriever and the Web as the corpus. This setting allows us to analyze the severity of imperfect retrieval in real-world RAG. Overall, we sample 1K short-form QA instances from these datasets, and pair each instance with 10 retrieved passages.

Imperfect retrieval is common.
We examine the occurrence of correct answers in retrieved passages as an approximation of retrieval quality. Since we mainly focus on short-form QA which provides most variants of the correct answer for each question, the approximation through string matching can give us a rouge intuition of how precise the retrieval result is. Specifically, we define the retrieval precision as the ratio of passages containing the correct answer for each instance:

$$ $\text{Retrieval Precision}=\frac{\text{\{number of retrieved passages containing correct answer\}}}{\text{\{number of total retrieved passages\}}}$ $$

As shown in Figure 2, although instances from different datasets exhibit different data distributions, imperfect retrieval is prevalent. Specifically, $\sim$20% of the overall data have no mentions of the correct answer within any retrieved passage, including 34% on NQ, 18% on TriviaQA, 24% on BioASQ, and 50% on PopQA.
This finding also aligns with previous observation on information retrieval (Thakur et al., ), that highlights that the number of positive passages can be very limited.

Imperfect retrieval leads to RAG failures.
We further analyze the relation between retrieval quality and RAG performance. We compare the performance of Claude 3.5 Sonnet, with and without RAG and report the results by retrieval precision in Figure 4.
In general, RAG is helpful when the retrieval precision is not lower than 20%. When the retrieval precision is close to 0, the model with RAG performs much worse than without RAG, indicating that imperfect retrieval augmentation can be the cause of RAG failures.
This finding aligns with the previous observation from Yu et al. () that adding more retrieved passages does not necessarily lead to better performance, as the additional passages might reduce the retrieval precision.

Knowledge conflicts widely exist in RAG failures.
We provide an in-depth analyses of knowledge conflicts between LLMs’ internal knowledge and retrieved passages from external sources. With Claude 3.5 Sonnet as the LLM, Figure 1 shows that 19.2% of the overall data exhibit knowledge conflicts, where either the answer with or without RAG is correct. Among the conflicting cases, the internal knowledge is correct on 47.4% of them, while the external knowledge is correct on the remaining 52.6%. These results emphasize the importance of effectively combining the internal and external knowledge to overcome the inherent limitation of relying solely on either source. However, previous work (Tan et al., ; Xie et al., ; Jin et al., ) show that LLMs might respond based on misleading information rather than comprehensive understanding of the conflicting knowledge in this context.

## 3 Astute RAG : Overcoming the Pitfall

Figure: Figure 3: Overview of the proposed Astute RAG framework. Astute RAG is designed to better combine the information from the external sources (e.g. web, domain-specific corpora, knowledge bases) and internal knowledge of the LLMs by employing a consolidation mechanism to address the information conflicts, which eventually leads to better quality generated outputs.
Refer to caption: extracted/5914248/figure/method_2.png

We begin with formulating the problem of imperfect retrieval in RAG (Section 3.1). We then provide an overview of Astute RAG, designed to overcome this problem (Section 3.2). Subsequently, we delve into the three major steps of Astute RAG, including adaptive generation of internal knowledge (Section 3.3), source-aware knowledge consolidation (Section 3.4), and answer finalization (Section 3.5).

### 3.1 Problem Formulation

Our objective is to mitigate the effects of imperfect retrieval augmentation, resolve knowledge conflicts between the LLM’s internal knowledge and external sources (such as custom/public corpora and knowledge bases), and ultimately produce more accurate and reliable responses from LLMs.

Given a set of retrieved passages from external sources $E=[e_{1},\ldots,e_{n}]$, a pre-trained LLM $\mathcal{M}$ (accessible through prediction-only APIs, encompassing commercial black-box ones), and a query $q$, the task is to generate the corresponding correct answer $a^{*}$.
Notably, this setting is orthogonal to prior work on improving the retriever, training LLMs, or conducting adaptive retrieval, which are mainly preliminary steps.

### 3.2 Overview of the Framework

Astute RAG is designed to better leverage collective knowledge from both internal knowledge of LLMs and external corpus, for more reliable responses. As shown in Figure 3 and Algorithm 1, Astute RAG starts from acquiring the most accurate, relevant, and thorough passage set from the LLMs’ internal knowledge. Then, internal and external knowledge are consolidated in an iterative way, by comparing the generated and retrieved passages. Finally, the reliability of conflicting information is compared and the final output is generated according to the most reliable knowledge.

Figure: Algorithm 1 Astute RAG

### 3.3 Adaptive Generation of Internal Knowledge

In the first step, we elicit internal knowledge from LLMs. This LLM-internal knowledge, reflecting the consensus from extensive pre-training and instruction-tuning data, can supplement any missing information from the limited set of retrieved passages and enable mutual confirmation between LLM-internal and external knowledge. This is especially valuable when the majority of retrieved passages might be irrelevant or misleading.
Specifically, we prompt LLMs to generate passages based on the given question $q$, following Yu et al. (). While Yu et al. () primarily focused on generating diverse internal passages, we emphasize the importance of reliability and trustworthiness of generated passages. To achieve this goal, we enhance the original method with constitutional principles and adaptive generation.

Inspired by Constitutional AI (Bai et al., ), we provide constitutional principles indicating the desired properties of internal passages in the prompt $p_{gen}$ (see Appendix A for details) to guide their generation, emphasizing that the generated passages should be accurate, relevant, and hallucination-free. Moreover, we allow the LLM to perform adaptive generation of passages in its internal knowledge. The LLM can decide how many passages to generate by itself. Rather generating a fix number of passages, we request the LLM to generate at most $\hat{m}$ passages, each covering distinct information, and to directly indicate if no more reliable information is available. This adaptive approach allows the LLM to generate fewer passages (or even no passages at all) when the useful information within internal knowledge is limited and more passages when there are multiple feasible answers in the internal knowledge. In this step, the LLM generates $m\leq\hat{m}$ passages based on its internal knowledge:

$$ $I=[i_{1},\ldots i_{m}]=\mathcal{M}(p_{gen},q,\hat{m}).$ $$

### 3.4 Iterative Source-aware Knowledge Consolidation

In the second step, we employ the LLM to explicitly consolidate information from both passages generated from its internal knowledge and passages retrieved from external sources.
Initially, we combine passages from both internal and external knowledge sources $D_{0}=E\oplus I.$

We additionally ensure source-awareness by providing the source of each passage to LLMs when consolidating knowledge. The source information (internal or external, such as a website) is helpful in assessing the reliability of passages. Here, we provide the passage source as
$S_{0}=[\mathbbm{1}_{\{d\in E\}}\text{for}\ d\ \text{in}\ D_{0}].$

To consolidate knowledge, we prompt the LLM (with $p_{con}$ in Appendix A) to identify consistent information across passages, detect conflicting information between each group of consistent passages, and filter out irrelevant information. This step would regroup the unreliable knowledge in input passages into fewer refined passages.
The regrouped passages will also attribute their source to the corresponding one or more input passages

$$ $\langle D_{j+1},S_{j+1}\rangle=\mathcal{M}(p_{con},q,\langle D_{0},S_{0} \rangle,\langle D_{j},S_{j}\rangle).$ $$

We find that this is especially helpful in comparing the reliability of conflicting knowledge and addressing knowledge conflicts.
Moreover, this knowledge consolidation process can run iteratively for $t$ times to improve the context to be more and more useful. Users can assign a larger number of iterations when the context is lengthy.

### 3.5 Answer Finalization

In the last step, we prompt the LLM (with $p_{ans}$ in Appendix A) to generate one answer based on each group of passages ($\langle D_{t},S_{t}\rangle$), and then compare their reliability and select the most reliable one as the final answer. This comparison allows the LLM to comprehensively consider knowledge source, cross-source confirmation, frequency, and information thoroughness when making the final decision.
Notably, this step can be merged into the last knowledge consolidation step to reduce the inference complexity (the amount of prediction API calls) using a combined prompt:

$$ $a=\mathcal{M}(p_{ans},q,\langle D_{0},S_{0}\rangle,\langle D_{t},S_{t}\rangle).$ $$

When $t=1$, the initial passages will be fed into the model directly for knowledge consolidation and subsequent answering: $a=\mathcal{M}(p_{ans},q,\langle D_{0},S_{0}\rangle).$

## 4 Experiments

We evaluate the effectiveness of Astute RAG on overcoming imperfect retrieval augmentation and addressing knowledge conflicts. In this section, we first introduce the experiment setting in detail (Section 4.1). Then, we compare the performance of Astute RAG with various baselines on diverse datasets (Section 4.2). Finally, we provide in-depth analyses (Section 4.3).

**Table 1: Main results on Claude under zero-shot setting, showing the accuracy of different benchmark methods vs. Astute RAG, along with their prediction complexity, in number of prediction API calls. Best scores are in bold.**
| Method | #API Calls | NQ | TriviaQA | BioASQ | PopQA | Overall |
| --- | --- | --- | --- | --- | --- | --- |
| Claude 3.5 Sonnet (20240620) |  |  |  |  |  |  |
| No RAG | 1 | 47.12 | 81.98 | 50.35 | 29.78 | 54.51 |
| RAG | 1 | 44.41 | 76.68 | 58.04 | 35.96 | 55.47 |
| USC (Chen et al., ) | 4 | 48.14 | 80.21 | 61.54 | 37.64 | 58.73 |
| GenRead (Yu et al., ) | 2 | 42.03 | 74.20 | 56.99 | 34.27 | 53.55 |
| RobustRAG (Xiang et al., ) | 11 | 47.80 | 78.09 | 56.29 | 37.08 | 56.53 |
| InstructRAG (Wei et al., ) | 1 | 47.12 | 83.04 | 58.04 | 41.01 | 58.83 |
| Self-Route (Xu et al., ) | 1-2 | 47.46 | 78.80 | 59.09 | 41.01 | 58.06 |
| Astute RAG (t=1) | 2 | 52.20 | 84.10 | 60.14 | 44.38 | 61.71 |
| Astute RAG (t=2) | 3 | 53.22 | 84.45 | 61.89 | 44.94 | 62.67 |
| Astute RAG (t=3) | 4 | 53.56 | 84.45 | 62.24 | 44.94 | 62.86 |

### 4.1 Experimental Settings

Datasets and metrics.
We conduct experiments on the data collected in Section 2 consisting of data from NQ, TriviaQA, BioASQ, and PopQA. For each instance from these datasets, we provide 10 passages collected under a realistic retrieval setting: for each question in our benchmark, we query Google Search to retrieve the top 30 results and select the first 10 accessible websites. From each retrieved website, we extract the paragraph corresponding to the snippet provided in Google Search results as the retrieved passage.. Most of the retrieval results contains natural noise with irrelevant or misleading information. We do not consider enhancements to the retrieval side, such as query rewriting, as such enhancements are typically already incorporated into commercial information retrieval systems. Notably, we do not select questions or annotate answers based on the retrieval results. This setting allows us to analyze the severity of imperfect retrieval in real-world RAG. It distinguishes our benchmark from previous ones that employ synthetic retrieval corruptions or that unintentionally reduce the frequency of imperfect retrieval with biased construction protocols (Chen et al., ; Yang et al., ). We also evaluate our method on RGB (Chen et al., ), a RAG diagnostic benchmark evaluating several crucial RAG abilities. Specifically, we choose the English subset of RGB focusing on noise robustness. The benchmark have positive and negative passage sets for each question. We select five negative documents per question as the context to form a worst-case scenario.
All the data in these datasets are short-form QA. Following previous work (Xiang et al., ; Wei et al., ; Mallen et al., ), a model response is considered correct if it contains the ground-truth answer. To enhance evaluation reliability, we prompt LLMs to enclose the exact answer within special tokens, extracting them as the final responses.

General Settings of LLMs and RAG.
We conduct experiments on two advanced LLMs, including Gemini 1.5 Pro(^5^55https://deepmind.google/technologies/gemini/pro/) (gemini-1.5-pro-002) and Claude 3.5 Sonnet(^6^66https://www.anthropic.com/news/claude-3-5-sonnet) (claude-3-5-sonnet@20240620). The generation temperature is set to 0 and the maximum output tokens is set to 1,024, if not specified otherwise. By default, the passages are presented in the prompt by reversed order. All experiments are under the zero-shot setting for controlled evaluation, where no demonstrations for QA or method-specific steps are provided.

Baselines.
We compare Astute RAG with various RAG methods designed for enhanced robustness and representative inference strategies designed to improve response trustworthiness.
USC (Chen et al., ) is the universal self-consistency method that samples multiple LLM responses given the same context and aggregates the answers. It provides a reference of naive improvements using additional API calls. The temperature for sampling responses in this baseline is set to 0.7.
Genread (Yu et al., ) augments retrieved passages with LLM-generated passages. It provide a reference of presenting passages from both internal and external knowledge in the prompt without effectively combining them.
RobustRAG (Xiang et al., ) aggregates answers from each independent passage to provide certifiable robustness. We use the keyword aggregation variant as it is shown to be the best-performing variant on advanced LLMs.
InstructRAG (Wei et al., ) instructs the LLM to provide a rationale connecting the answer with information in passages. For a fair comparison, we use the instructions without training or in-context learning.
Self-Route (Xu et al., ) adaptively switches between LLMs with and without RAG.(^7^77The original Self-Route switches between RAG and long-context LLMs, while our implementation switches between LLMs with and without RAG to better align with the problem formulation in this paper.) This baseline provides a reference of switching between LLMs’ internal and external knowledge.

Implementation Details of Astute RAG.
The prompt templates for Astute RAG can be found in Appendix A.
By default, we use 2 API calls per query, setting $t=1$ to merge the prompt for knowledge consolidation and answer finalization. For adaptive generation of internal knowledge, we prompt the LLM to generate no more than one passage.

**Table 2: Main results on Gemini under zero-shot setting, showing the accuracy of different benchmark methods vs. Astute RAG, along with their prediction complexity, in number of prediction API calls. Best scores are in bold.**
| Method | #API Calls | NQ | TriviaQA | BioASQ | PopQA | Overall |
| --- | --- | --- | --- | --- | --- | --- |
| Gemini 1.5 Pro (002) |  |  |  |  |  |  |
| No RAG | 1 | 44.75 | 80.21 | 45.80 | 25.28 | 51.34 |
| RAG | 1 | 42.71 | 75.97 | 55.24 | 33.71 | 53.65 |
| USC (Chen et al., ) | 4 | 46.44 | 76.68 | 58.39 | 37.64 | 56.43 |
| GenRead (Yu et al., ) | 2 | 45.08 | 77.39 | 54.90 | 34.27 | 54.70 |
| RobustRAG (Xiang et al., )(^8^88We observe a high refusal rate in responses of RobustRAG.) | 11 | 34.24 | 67.49 | 44.06 | 32.02 | 45.59 |
| InstructRAG (Wei et al., ) | 1 | 46.78 | 80.57 | 54.90 | 34.83 | 56.14 |
| Self-Route (Xu et al., ) | 1-2 | 47.46 | 79.86 | 58.04 | 38.20 | 57.58 |
| Astute RAG (t=1) | 2 | 50.17 | 81.63 | 58.04 | 40.45 | 59.21 |
| Astute RAG (t=2) | 3 | 51.53 | 81.27 | 58.74 | 40.45 | 59.69 |
| Astute RAG (t=3) | 4 | 48.47 | 80.21 | 60.14 | 42.13 | 59.21 |

### 4.2 Main Results

Table 1 and Table 2 presents the results on data with realistic retrieval augmentation for each dataset. By comparing RAG and No RAG, we find that retrieved passages might not always bring benefits to downstream performance – on NQ and TriviaQA, RAG performance lags behind No RAG. We attribute this to that the questions being covered by the LLM’s internal knowledge and the noise in retrieval results misleading the LLM. In contrast, on BioASQ and PopQA, which focus on domain-specific and long-tail questions, RAG significantly improves LLM performance. However, due to imperfect retrieval augmentation, the absolute performance still remains to be unsatisfactory.
Among all baselines, no single method consistently outperforms others across all datasets. This observation highlights that these baselines are tailored to distinct settings and may not be universally applicable.
For instance, InstructRAG is more effective on TriviaQA, achieving the best performance among all baselines with both Claude and Gemini.
In contrast, Self-Route performs better than InstructRAG on both NQ and BioASQ.
Moreover, RobustRAG achieves very different performance when applied to Gemini and Claude. Through in-depth analysis, we find that RobustRAG with Gemini exhibits a high refusal rate (refuse to answer) in responses.
We attribute this instability to the varying method designs of the baselines, which are tailored for different scenarios, resulting in inconsistent improvement across datasets.
Overall, InstructRAG and Self-Route demonstrates the best performance among all baselines when applied to Claude and Gemini respectively.
We also note that increasing the number of API calls does not necessarily correlate with improved performance.

Astute RAG consistently outperforms baselines across all datasets of different properties. The overall improvement compared with the best baseline is relatively 6.85% on Claude and 4.13% on Gemini, and the improvements in domain-specific questions are much higher. These results highlight the effectiveness of Astute RAG in overcoming imperfect retrieval augmentation.
On Claude, adding more iteration of knowledge consolidation leads to consist improvement. The improvement margin becomes lower when $t$ becomes larger. This is because after each iteration, the remaining improvement space for knowledge consolidation becomes smaller.
On Gemini, increasing $t$ primarily benefits BioASQ and PopQA. These two datasets rely more heavily on external knowledge, and iterative knowledge consolidation helps mitigate noise within this external information.
Performance on NQ and TriviaQA does not improve further when $t$ reaches 3. We attribute this to the less critical role of external knowledge in these datasets. For setting consistency and efficiency, we set the parameter $\hat{m}$ to a smaller value, limiting the influence of internal knowledge.

### 4.3 Analyses

Performance by retrieval precision.
We compare the performance of Astute RAG and baselines across different subsets partitioned by their retrieval precision, on our collected data with Claude as the LLM.
As shown in Figure 4, Astute RAG achieves consistently better performance than all baselines across different retrieval precision, indicating its effectiveness in improving RAG trustworthiness in broad scenarios. Notably, Astute RAG does not sacrifice performance gain under high retrieval quality in exchange for improvement under low retrieval quality.
When the retrieval quality is extremely low (close to zero retrieval precision), all other RAG variants underperforms the ’No RAG’ baseline, except for the proposed Astute RAG. This observation aligns with the worst-case results on RGB. It demonstrates the difficulty in overcoming imperfect retrieval augmentation, and verify the effectiveness of Astute RAG in doing so.

Figure: (a) Performance by Retrieval Precision
Refer to caption: extracted/5914248/figure/perf_dist.png

Effectiveness in addressing knowledge conflicts.
We split our collected data in to three subset according to the answers from Claude, with and without RAG. The answers from two inference methods can be both correct, both incorrect, or conflicting with one being correct. These three subsets represents the three situations between internal and external knowledge. The results are shown in Figure 4.
On the conflicting subset, Astute RAG successfully chooses the correct answer in approximately 80% of cases, being the most effective method in addressing knowledge conflicts.
Notably, Astute RAG even brings performance improvement on the subset where neither internal nor external knowledge alone leads to the correct answer. This indicates that Astute RAG can effectively combine partially-correct information from LLM-internal and external knowledge, to achieve the correct answer through collective information across them.

Worst-case performance on RGB..
Figure 4 presents the results under the worst-case setting on RGB where all retrieved documents are negative. It demonstrates the noise robustness of Astute RAG and baseline RAG methods.
The performance gap between RAG and No RAG exceeds 50 points, highlighting the detrimental impact of imperfect retrieval results and emphasizing the importance of providing robust safeguards against worst-case scenarios.
While the baseline RAG methods outperform the original RAG, they still obviously fall behind No RAG. Astute RAG is the only RAG method that reaches a performance close to No RAG under the worst-case scenario, further supporting its effectiveness in addressing imperfect retrieval augmentation.

Qualitative study.
In Figure 5, we present two representative examples showing the intermediate outputs of Astute RAG. In the first example, LLM without RAG generates a wrong answer, while RAG returns a correct answer. Astute RAG successfully identified the incorrect information in its generated passage and an external passage, avoiding confirmation bias Tan et al. (). In the second example, LLM alone is correct, while RAG is incorrect due to the noisy retrieval results. Astute RAG detected the correct answer from noisy retrieved information by checking with its internal knowledge.

Figure: Figure 5: Qualitative examples. Top: Astute RAG identified the error in internal knowledge (i.e., generated passage) by confirming with external sources. Bottom: Astute RAG detected the correct answer from noisy retrieved information by checking with its internal knowledge. Standard RAG does not provide an answer because the retrieved passages are too noisy.
Refer to caption: extracted/5914248/figure/qual_2.png

## 5 Related Work

Retrieval augmented generation (RAG) seeks to address the inherent knowledge limitation of LLMs with passages retrieved from external sources of information such as private corpora or public knowledge bases (Guu et al., ; Lewis et al., ; Borgeaud et al., ).
Given the widespread adoption of RAG in various real-world applications, including risk-sensitive domains, the negative impact of noisy information within retrieved passages has garnered increasing attention (Cuconasu et al., ).
Recent work has sought to enhance the robustness of RAG systems against noise from various perspectives, including training LLMs with noisy context (Yu et al., ; Yoran et al., ; Pan et al., ; Fang et al., ), training small models to filter out irrelevant passages (Wang et al., ; Xu et al., ), passage reranking (Yu et al., ; Glass et al., ), dynamic and iterative retrieval (Jiang et al., ; Asai et al., ; Yan et al., ), query rewriting (Ma et al., ), and speculative drafting (Wang et al., ). These studies focus on distinct modules or stages of RAG systems and are orthogonal to our work.

Our work focuses on enhancing RAG robustness at the post-retrieval stage, after retrieved passages have been provided. On this topic, RobustRAG (Xiang et al., ) aggregates answers from each independent passage to provide certifiable robustness. InstructRAG (Wei et al., ) instructs the LLM to provide a rationale connecting the answer with information in passages. MADRA (Wang et al., ) applies multi-agent debate to select helpful evidence. However, these works do not explicitly incorporate internal knowledge to recover from RAG failures and may therefore collapse when the majority of retrieved passages are negative.
In terms of emphasizing internal knowledge of LLMs in RAG, recent work has explored using LLM-generated passage as context (Yu et al., ; Zhang et al., ), adaptively switching between LLMs with and without RAG (Xu et al., ; Mallen et al., ; Jeong et al., ), and combining answers from internal and external knowledge through contrastive decoding (Zhao et al., ; Jin et al., ).
We focus on a black-box setting where no further training is required, directly addressing knowledge conflicts to combine the helpful information from both sides and achieve more reliable answers.

## 6 Conclusion

Our paper investigates the impact of imperfect retrieval on the performance of RAG systems and identifies knowledge conflicts as a key challenge. To address this, we introduce Astute RAG, a novel approach that leverages the internal knowledge of LLMs and iteratively refines the generated responses by consolidating internal and external knowledge in a source way. Our empirical results demonstrate the effectiveness of Astute RAG in mitigating the negative effects of imperfect retrieval and improving the robustness of RAG systems, particularly in challenging scenarios with unreliable external sources.

Among the limitations, Astute RAG’s effectiveness hinges on the capabilities of advanced LLMs with strong instruction-following and reasoning abilities, hence potentially more limited applicability with less sophisticated LLMs.
As an important future direction, extending the experimental setup to include longer outputs would be important, where the challenges of imperfect retrieval and knowledge conflicts may be even more pronounced. Furthermore, a comprehensive analysis of the impact of various context types (Balachandran et al., ) would enhance the understanding of the proposed method’s effectiveness. Future work can also extend our method beyond LLMs and RAG, such as addressing knowledge conflicts in multimodal settings (Zhu et al., ).

## Acknowledgement

We would like to thank Jinsung Yoon for valuable discussions and insights that helped to improve this paper. We would also like to thank all other colleagues from Google Cloud AI Research for their valuable feedback.

*

## Appendix A Prompt Template for Astute RAG

## Appendix B Data Collection

Encompassing a diverse range of natural questions, our benchmark consists of realistic retrieval results with Google Search(^9^99[https://developers.google.com/custom-search/v1/overview](https://developers.google.com/custom-search/v1/overview)) as the retriever and the Web as the corpus. Notably, we do not select questions or annotate answers based on the retrieval results. This setting allows us to analyze the severity of imperfect retrieval in real-world RAG. It distinguishes our benchmark from previous ones that employ synthetic retrieval corruptions or that unintentionally reduce the frequency of imperfect retrieval with biased construction protocols (Chen et al., ; Yang et al., ). Overall, our benchmark contains 1,042 short-form question-answer pairs, each paired with 10 retrieved passages.

Question-answer pairs.
We consider question-answer pairs from four datasets of different properties spanning across general questions, domain-specific questions, and long-tail questions. NQ (Kwiatkowski et al., ) and TriviaQA (Joshi et al., ) are two widely-studied question-answering (QA) datasets in general domains. BioASQ (Tsatsaronis et al., ) is from biomedical domain that has demonstrated significant benefits from RAG when general-purpose LLMs are considered. PopQA (Mallen et al., ) focuses on long-tail knowledge and has been shown to be challenging for even advanced LLMs to solve without external knowledge. All these datasets contain questions with short-form answers and most of them list all valid answer variants. This format can support automatic verification of answer appearance in retrieved passages and model responses, leading to more precise evaluations.

Retrieval process.
For each question in our benchmark, we query Google Search to retrieve the top 30 results and select the first 10 accessible websites. From each retrieved website, we extract the paragraph corresponding to the snippet provided in Google Search results as the retrieved passage. We do not consider enhancements to the retrieval side, such as query rewriting, as such enhancements are typically already incorporated into commercial information retrieval systems.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: While the paper presents valuable contributions and strong empirical results, it falls somewhat short of the technical depth and theoretical foundations:
- Could you elaborate on your choice and implementation of Google Search as the retrieval method: Were there specific criteria for determining which snippets were included in the 10 selected passages? What were the specific reasons for choosing Google Search over other commercial search engines (e.g., Bing)? How did you handle sponsored or advertised content in the search results? 
- How might the results differ with other retrieval approaches? How sensitive is the method to different prompt templates? What is the impact of different parameter settings (e.g., number of iterations, maximum generated passages) on performance and stability?
- How would Astute RAG perform on more complex tasks beyond short-form QA? Could you provide examples of failure cases when the system might not be appropriate to use?
soundness: 3
strengths: -  The paper presents an advanced approach to RAG systems, with a well-structured methodology for combining internal LLM knowledge with external sources. The three-step process demonstrates careful consideration of the challenges in knowledge integration.
- The authors provide comprehensive experimental results using state-of-the-art LLMs (Gemini and Claude) and multiple datasets (NQ, TriviaQA, BioASQ, PopQA), offering valuable insights into the method's performance across different scenarios.
- The work addresses a critical challenge in RAG systems: "the prevalence of imperfect retrieval and knowledge conflicts", backed by solid empirical evidence showing that roughly 70% of retrieved passages don't directly contain true answers.
- The authors' analysis using realistic conditions with Google Search as a retriever provides valuable insights for the research community, particularly in understanding real-world RAG system behavior.
summary: The paper introduces Astute RAG, a novel approach addressing imperfect retrieval and knowledge conflicts in RAG systems. 
The authors first conduct comprehensive analyses demonstrating that imperfect retrieval is prevalent in real-world RAG applications and identify knowledge conflicts between LLMs' internal knowledge and external sources as a key bottleneck. Authors then proposes Astute RAG, which adaptively elicits internal knowledge from LLMs, performs source-aware knowledge consolidation, and finalizes answers based on information reliability. Besides that, the authors demonstrate that Astute RAG outperforms previous robustness-enhanced RAG methods (especially in worst-case scenarios where traditional RAG approaches fail).
weaknesses: - While technically sound, the paper doesn't sufficiently articulate why Astute RAG is particularly advantageous for real-world applications compared to simpler approaches, such as different passage ordering strategies (e.g., chronological, relevance-based, or clustered arrangements), varying quality of knowledge sources (from high-authority to potentially unreliable sources), or different passage selection methods. In fact, authors state that passages are presented by reversed order in the set of experiments, although no further positional dependence has been explored. Recent RAG studies has shown significant differences between distinct arrangement strategies (e.g. Alessio et al. 2024 Improving RAG Systems via Sentence Clustering and Reordering).
- The focus on short-form QA tasks limits understanding of the method's broader applicability. Imho, retriever improvements do not always translate into proportional gains in final answers, particularly in open-domain questions. in specialized tasks, even small improvements in retrieval can significantly boost final answers.
- The method's reliance on advanced LLMs with strong instruction-following capabilities significantly limits its applicability. The paper would benefit from addressing how the approach could be adapted for more resource-constrained or smaller specialized language models. Could you discuss strategies for adapting Astute RAG to resource-constrained environments?

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: see weakness
soundness: 2
strengths: - This paper provides an empirical experimental analysis of the impact of imperfect retrieval and knowledge conflict on the accuracy of RAG systems, thereby validating the significance of its motivation.
- This paper presents an intuitive approach that, through the careful design of a framework and prompts, can alleviate the issues posed by the aforementioned challenges, thereby enhancing the performance of RAG systems without training.
summary: This paper proposes astute RAG, a framework to address imperfect retrieval and knowledge conflict challenges in retrieval-augmented generation.
Specifically, astute RAG consists of 3 steps including adaptive internal knowledge generation, iterative knowledge consolidation and answer finalization.
Experiments demonstrate that astute RAG outperforms previous robustness-enhanced RAG methods.
weaknesses: - **Applicability of the method**: The framework proposed by the authors involves a complex pipeline and prompt design, which necessitates that the target model possesses various capabilities. However, the authors do not provide sufficient experimental evidence regarding the performance of current models in these intermediate stages. Furthermore, the experiments were conducted solely on two close-sourced models with undisclosed details. The lack of relevant details prevents readers from assessing the applicability and limitations of the method. I strongly recommend that the authors include experiments using more open-source models and provide analyses of the intermediate processes.
- **Mismatch between method and experimental design**: In the initial step, the authors employ adaptive generation to extract the model's internal knowledge. However, in the experimental setting, the model is required to generate a maximum of only one passage, which means the design of adaptive generation is not effectively reflected in the experiments. Consequently, the comparison of the number of API calls also lacks significance. Furthermore, I believe that the authors should focus on comparing the total number of API tokens used rather than the number of calls.
- **Novelty of the method**: Although the authors have designed a new framework, similar ideas have appeared in some prior works (e..g, [1]). This factor limits the novelty of this paper.

[1] Merging Generated and Retrieved Knowledge for Open-Domain QA

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. The relationship between retrieval imperfection and query characteristics needs further investigation. Please consider conducting additional experiments to: (1) analyze how different query types affect retrieval performance, (2) evaluate whether the findings from short-form QA can be generalized to other formats,(e.g. Long-form QA) and (3) compare system performance with varying numbers of retrieved documents (e.g., 10, 20, 50) given that modern LLMs can handle larger context windows. These analyses would provide more comprehensive insights into the retrieval mechanism's limitations and potential improvements.

2. In the manuscript, the author mentioned that there were several cases where the LLM refused to provide answers during the experiments. Could you please clarify how these refusal cases were handled in the results presented in Table 1? It would be valuable if you could provide specific statistics on the frequency of LLM refusals, and describe mitigation strategies implemented to handle these refusal scenarios? Furthermore, it appears that these LLM refusal cases could significantly impact the overall Recall of the question-answering , yet there seems to be a lack of discussion regarding this crucial aspect. A more detailed analysis of how these refusals affect the system's would strengthen the evaluation section and provide a more comprehensive understanding of the system's real-world performance  and limitations.

3. Please elaborate on the potential role of reranking in ASTUTE RAG. Specifically, could you: (1) discuss why reranking was not incorporated in the current implementation, (2) analyze how reranking might help address the performance decrease observed in Table 1 when using RAG, and (3) explore whether reranking could be a promising direction for future improvements in reducing noise and enhancing retrieval relevance. This discussion would provide valuable insights into the design choices of ASTUTE and potential avenues for enhancement.

4. Please maintain consistency in the abbreviation format, specifically for 'Retrieval-Augmented Generation (RAG)' in the first sentences of the abstract and  'Retrieval augmented generation (RAG)' in the first sentences of the introduction.
soundness: 3
strengths: 1. The research question, "Is there an effective method to integrate internal and external knowledge for enhancing the reliability of RAG?", is at the cutting edge of the field and holds substantial practical significance for the RAG community. 

2. The essence of this paper is to utilize LLM to conduct a self-reflection on the query (generating internal knowledge documents) first, and then perform a conflict detection on external issues, making full use of the LLM's own capabilities and effectively alleviating the conflict between internal and external support. 

3. The method enhances the robustness in case of retrieval failure. Even when all retrieved information is noisy, it can effectively ensure the bottom-line performance of LLM.
summary: The paper focuses on the problem of imperfect retrieval in RAG for LLMs. It analyzes its prevalence and negative impacts like RAG failures and knowledge conflicts. To address this, ASTUTE RAG is proposed, which adaptively generates internal knowledge, iteratively consolidates knowledge with source-awareness, and finalizes answers based on reliability. Experiments show ASTUTE RAG outperforms baselines in various scenarios, effectively resolving knowledge conflicts.
weaknesses: 1. The performance and efficiency of the method are relatively low. The AstuteRAG proposed by the authors is used in the retrieval (online) stage, where LLM is made to generate as much internal knowledge Q&A as possible. This process is rather time-consuming and consumes tokens. Subsequently, iterative knowledge verification is required, which still uses LLM for conflict detection and paragraph merging. Overall, it is very time-consuming and reduces the practicality of the method.

2. A  major  potential drawback is that the knowledge consolidation process is fully completed by LLM. Although the conflicting information is grouped according to its consistency and the conflicting information is separated and then LLM is made to propose answers and assign confidence scores for each group of documents. Essentially, it is still LLM making judgments based on its own capabilities, and the influence of inherent bias and hallucination cannot be eliminated. A very concerning question is whether the confidence score of LLM shows an obvious bias when there is a conflict between internal and external knowledge? The bias may not necessarily be specifically towards internal knowledge or external knowledge, but rather towards some tokens with a specific high distribution.

3. The authors' verification of the method in this paper is all based on benchmarks with definite factual answers, lacking analysis of some scenarios. For example, both the internal knowledge and the external retrieval content are actually correct, but they form a conflict due to the time range or other specified limiting conditions in the query. From the current method, it is not seen that AstuteRAG has good robustness in this regard. This is also strongly related to the judgment basis mentioned in Weakness 2

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: See Weaknesses above.
soundness: 3
strengths: In general, I think it is critical to resolve  the imperfect retrieval problem and the knowledge conflict problem for better/more robust RAG, and the proposed pipleline(adaptive inner knowledge generation and iterative consolidation) is helpful  for robust RAG. Furthermore, I found the proposed method is training-free.
summary: In this paper, the authors address the imperfect retrieval problem and the knowledge conflict problem in RAG, and propose a AstuteRAG method which firstly adaptively generate passages using its internal parameter knowledge, then consolidate information from both generated passages and retrieved passages by taking their source information into consideration, constructing consistent groups, and addressing the conflicts iteratively. Experimental results show some performance improvements over several RAG baselines.
weaknesses: 1. Both the imperfect retrieval problem and  the knowledge conflict problem have been widely recognized in the RAG field (see Benchmarking large language models in retrieval-augmented generation AAAI 24 for example), therefore it is inappropriate for "previous studies have rarely connected...". The authors should make a more comprehensive review of RAG studies;
2. There are many previous studies address the source/confidence-awareness of RAG,  iterative RAG, and knowledge-conflict consolidation methods. Therefore, to claim this paper's contributions, the authors should explain the differences/advantages of their methods over these studies.
3. Based on the above observations, I found the baselines in this paper are all general RAG baselines rather than RAG methods addressing the noise/conflict RAG problem,  which makes the experimental results not convincing enough. I think the authors should also compare with iterative-based, agent-based, and confidence-aware RAG baselines.